Skip to content

Commit 6116e1e

Browse files
NimRegevNimRegev
authored
Add timeout field to git-clone and parallel step (#838)
* Add timeout field to git-clone and parallel step * Updates * Update content Implemented feedback from SME * Add timeout content to remaining step topics Added timeout field description and timeout section to build, push, freestyle, composition and launch composition steps
1 parent 2f46f08 commit 6116e1e

File tree

8 files changed

+404
-1
lines changed

8 files changed

+404
-1
lines changed

‎_docs/pipelines/advanced-workflows.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,34 @@ The final order of execution will be
9898

9999
This is the recommended way to start using parallelism in your Codefresh pipelines. It is sufficient for most scenarios that require parallelism.
100100

101-
>The step names must be unique within the same pipeline. The parent and child steps should NOT share the same name.
101+
>**NOTE**:
102+
The names of the parallel steps must be unique within the same pipeline. The parent and child steps should NOT share the same name.
103+
104+
### Add timeouts for parallel steps
105+
You can add `timeouts`for parallel steps either by defining a parent `timeout` inherited by the other parallel steps, or by defining individual timeouts for every step.
106+
107+
>**NOTES**:
108+
The `timeout` field in the `deploy` and `pending-approval` steps has a specialized syntax and behavior to match the requirements of these steps.<br><br>
109+
When _both_ parent and step-specific timeouts are defined for parallel steps, the step-specific timeouts override the parent timeout.
110+
111+
{% highlight yaml %}
112+
{% raw %}
113+
version: '1.0'
114+
steps:
115+
parallel:
116+
type: parallel
117+
timeout: 1m
118+
steps:
119+
first:
120+
image: alpine
121+
second:
122+
image: alpine
123+
timeout: 2m
124+
third:
125+
image: alpine
126+
timeout: null
127+
{% endraw %}
128+
{% endhighlight %}
102129

103130
### Example: pushing multiple Docker images in parallel
104131

‎_docs/pipelines/steps/build.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ step_name:
9797
| `no_cf_cache` | Defines if to enable or disable Codefresh build optimization for the build. When set to `false`, the default, enables Codefresh build optimization. See [more info]({{site.baseurl}}/docs/kb/articles/disabling-codefresh-caching-mechanisms/). | |
9898
| `build_arguments` | The set of [Docker build arguments](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables-build-arg){:target="\_blank"} to pass to the build process. | Optional |
9999
| `target` | The target stage at which to stop the build in a multistage build. | Optional |
100+
|`timeout` | The maximum duration permitted to complete step execution in seconds (`s`), minutes (`m`), or hours (`h`), after which to automatically terminate step execution. For example, `timeout: 1.5h`. <br>The timeout supports integers and floating numbers, and can be set to a maximum of 2147483647ms (approximately 24.8 days). <br><br>If defined and set to either `0s/m/h` or `null`, the timeout is ignored and step execution is not terminated.<br>See [Add a timeout to terminate step execution](#add-a-timeout-to-terminate-step-execution). |Optional|
100101
| `fail_fast` | Define the build behavior on step failure. When set to `true`, the default, ff a step fails, the build is stopped as well. To continue the build on step failure, set to `false`. | Default |
101102
| `when` | The set of conditions that need to be satisfied in order to execute this step.<br>For more information, see [Conditional execution of steps]({{site.baseurl}}/docs/pipelines/conditional-execution-of-steps/) . | Optional |
102103
| `metadata` | Annotate the built image with [key-value metadata]({{site.baseurl}}/docs/pipelines/docker-image-metadata/). | Optional |
@@ -115,6 +116,70 @@ step_name:
115116

116117
## Build image step examples
117118

119+
### Add a timeout to terminate step execution
120+
To prevent steps from running beyond a specific duration if so required, you can add the `timeout` flag to the step.
121+
When defined:
122+
* The `timeout` is activated at the beginning of the step, before the step pulls images.
123+
* When the step's execution duration exceeds the duration defined for the `timeout`, the step is automatically terminated.
124+
125+
>**NOTE**:
126+
To define timeouts for parallel steps, see [Adding timeouts for parallel steps]({{site.baseurl}}/docs/pipelines/advanced-workflows/#adding-timeouts-for-parallel-steps).
127+
128+
Here's an example of the `timeout` field in the step:
129+
130+
`YAML`
131+
{% highlight yaml %}
132+
step_name:
133+
type: build
134+
title: Step Title
135+
description: Free text description
136+
working_directory: {% raw %}${{clone_step_name}}{% endraw %}
137+
dockerfile: path/to/Dockerfile
138+
image_name: owner/new-image-name
139+
tag: develop
140+
platform: 'linux/arm64'
141+
buildx: true
142+
build_arguments:
143+
- key=value
144+
cache_from:
145+
- owner/image-name:${{CF_BRANCH}}
146+
- owner/image-name:main
147+
target: stage1
148+
no_cache: false
149+
no_cf_cache: false
150+
tag_policy: original
151+
timeout: 45m
152+
fail_fast: false
153+
metadata:
154+
set:
155+
- qa: pending
156+
when:
157+
condition:
158+
all:
159+
noDetectedSkipCI: "includes('{% raw %}${{CF_COMMIT_MESSAGE}}{% endraw %}', '[skip ci]') == false"
160+
on_success:
161+
...
162+
on_fail:
163+
...
164+
on_finish:
165+
...
166+
retry:
167+
...
168+
{% endhighlight %}
169+
170+
171+
**Timeout info in logs**
172+
Timeout information is displayed in the logs, as in the example below.
173+
174+
{% include image.html
175+
lightbox="true"
176+
file="/images/steps/timeout-messages-in-logs.png"
177+
url="/images/steps/timeout-messages-in-logs.png"
178+
caption="Step termination due to timeout in logs"
179+
alt="Step termination due to timeout in logs"
180+
max-width="60%"
181+
%}
182+
118183
### Using Dockerfile in root project folder
119184

120185
`codefresh.yml`

‎_docs/pipelines/steps/composition.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ The following describes the fields available in a step of type `composition`
102102
| `registry_contexts` | Advanced property for resolving Docker images when [working with multiple registries with the same domain]({{site.baseurl}}/docs/ci-cd-guides/working-with-docker-registries/#working-with-multiple-registries-with-the-same-domain) | Optional |
103103
| `volumes` (service level) | Extra volumes for individual services. Used for transferring information between your steps. Explained in detail later in this page. | Optional |
104104
| `composition_variables` | A set of environment variables to substitute in the composition. Notice that these variables are docker-compose variables and **NOT** environment variables | Optional |
105+
|`timeout` | The maximum duration permitted to complete step execution in seconds (`s`), minutes (`m`), or hours (`h`), after which to automatically terminate step execution. For example, `timeout: 1.5h`. <br>The timeout supports integers and floating numbers, and can be set to a maximum of 2147483647ms (approximately 24.8 days). <br><br>If defined and set to either `0s/m/h` or `null`, the timeout is ignored and step execution is not terminated.<br>See [Add a timeout to terminate step execution](#add-a-timeout-to-terminate-step-execution). |Optional|
105106
| `fail_fast` | If a step fails, and the process is halted. The default value is `true`. | Default |
106107
| `when` | Define a set of conditions which need to be satisfied in order to execute this step.<br>You can find more information in the [conditional execution of steps]({{site.baseurl}}/docs/pipelines/conditional-execution-of-steps/) article. | Optional |
107108
| `on_success`, `on_fail` and `on_finish` | Define operations to perform upon step completion using a set of predefined [post-step operations]({{site.baseurl}}/docs/pipelines/post-step-operations/). | Optional |
@@ -157,6 +158,67 @@ If you run this composition, you will see in the logs that the alpine image will
157158
my_service_1 | /tmp
158159
my_test_service_1 | /root
159160
```
161+
## Add a timeout to terminate step execution
162+
To prevent steps from running beyond a specific duration if so required, you can add the `timeout` flag to the step.
163+
When defined:
164+
* The `timeout` is activated at the beginning of the step, before the step pulls images.
165+
* When the step's execution duration exceeds the duration defined for the `timeout`, the step is automatically terminated.
166+
167+
>**NOTE**:
168+
To define timeouts for parallel steps, see [Adding timeouts for parallel steps]({{site.baseurl}}/docs/pipelines/advanced-workflows/#adding-timeouts-for-parallel-steps).
169+
170+
Here's an example of the `timeout` field in the step:
171+
172+
`codefresh.yml`
173+
{% highlight yaml %}
174+
step_name:
175+
type: composition
176+
title: Step Title
177+
description: Free text description
178+
working_directory: {% raw %}${{a_clone_step}}{% endraw %}
179+
composition:
180+
version: '2'
181+
services:
182+
db:
183+
image: postgres
184+
composition_candidates:
185+
test_service:
186+
image: {% raw %}${{build_step}}{% endraw %}
187+
command: gulp integration_test
188+
working_dir: /app
189+
environment:
190+
- key=value
191+
composition_variables:
192+
- key=value
193+
timeout: 45m
194+
fail_fast: false
195+
when:
196+
condition:
197+
all:
198+
notFeatureBranch: 'match("{% raw %}${{CF_BRANCH}}{% endraw %}", "/FB-/", true) == false'
199+
on_success:
200+
...
201+
on_fail:
202+
...
203+
on_finish:
204+
...
205+
retry:
206+
...
207+
{% endhighlight %}
208+
209+
210+
**Timeout info in logs**
211+
Timeout information is displayed in the logs, as in the example below.
212+
213+
{% include image.html
214+
lightbox="true"
215+
file="/images/steps/timeout-messages-in-logs.png"
216+
url="/images/steps/timeout-messages-in-logs.png"
217+
caption="Step termination due to timeout in logs"
218+
alt="Step termination due to timeout in logs"
219+
max-width="60%"
220+
%}
221+
160222

161223
## Composition networking
162224

‎_docs/pipelines/steps/freestyle.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ step_name:
9393
| `entry_point` | Override the default container entry point. can be string or array of strings. | Optional |
9494
| `shell` | Explicitly set the executing shell to bash or sh. If not set the default will be sh. Note the `bash` option requires that you specify an `image` that includes `/bin/bash`; many images do not. | Optional |
9595
| `environment` | A set of environment variables for the container. | Optional |
96+
|`timeout` | The maximum duration permitted to complete step execution in seconds (`s`), minutes (`m`), or hours (`h`), after which to automatically terminate step execution. For example, `timeout: 1.5h`. <br>The timeout supports integers and floating numbers, and can be set to a maximum of 2147483647ms (approximately 24.8 days). <br><br>If defined and set to either `0s/m/h` or `null`, the timeout is ignored and step execution is not terminated.<br>See [Add a timeout to terminate step execution](#add-a-timeout-to-terminate-step-execution). |Optional|
9697
| `fail_fast` | If a step fails, and the process is halted. The default value is `true`. | Default |
9798
| `registry_context` | Advanced property for resolving Docker images when [working with multiple registries with the same domain]({{site.baseurl}}/docs/ci-cd-guides/working-with-docker-registries/#working-with-multiple-registries-with-the-same-domain) | Optional |
9899
| `volumes` | One or more volumes for the container. All volumes must be mounted from the existing shared volume (see details below) |Optional
@@ -103,6 +104,68 @@ step_name:
103104
**Exported resources:**
104105
- Working Directory.
105106

107+
## Add a timeout to terminate step execution
108+
To prevent steps from running beyond a specific duration if so required, you can add the `timeout` flag to the step.
109+
When defined:
110+
* The `timeout` is activated at the beginning of the step, before the step pulls images.
111+
* When the step's execution duration exceeds the duration defined for the `timeout`, the step is automatically terminated.
112+
113+
>**NOTE**:
114+
To define timeouts for parallel steps, see [Adding timeouts for parallel steps]({{site.baseurl}}/docs/pipelines/advanced-workflows/#adding-timeouts-for-parallel-steps).
115+
116+
Here's an example of the `timeout` field in the step:
117+
118+
{% highlight yaml %}
119+
{% raw %}
120+
step_name:
121+
title: Step Title
122+
description: Step description
123+
image: image/id
124+
working_directory: ${{step_id}}
125+
commands:
126+
- bash-command1
127+
- bash-command2
128+
cmd:
129+
- arg1
130+
- arg2
131+
environment:
132+
- key=value
133+
entry_point:
134+
- cmd
135+
- arg1
136+
timeout: 45m
137+
shell: sh
138+
fail_fast: false
139+
volumes:
140+
- ./relative-dir-under-cf-volume1:/absolute-dir-in-container1
141+
- ./relative-dir-under-cf-volume2:/absolute-dir-in-container2
142+
when:
143+
branch:
144+
only: [ master ]
145+
on_success:
146+
...
147+
on_fail:
148+
...
149+
on_finish:
150+
...
151+
retry:
152+
...
153+
{% endraw %}
154+
{% endhighlight %}
155+
156+
157+
**Timeout info in logs**
158+
Timeout information is displayed in the logs, as in the example below.
159+
160+
{% include image.html
161+
lightbox="true"
162+
file="/images/steps/timeout-messages-in-logs.png"
163+
url="/images/steps/timeout-messages-in-logs.png"
164+
caption="Step termination due to timeout in logs"
165+
alt="Step termination due to timeout in logs"
166+
max-width="60%"
167+
%}
168+
106169
## Examples
107170

108171
Here are some full pipelines with freestyle steps. Notice that in all cases the pipelines are connected to [Git repositories]({{site.baseurl}}/docs/pipelines/pipelines/#loading-codefreshyml-from-version-control)

‎_docs/pipelines/steps/git-clone.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ step_name:
5959
| `depth` | The number of commits to pull from the repo to create a shallow clone. Creating a shallow clone truncates the history to the number of commits specified, instead of pulling the entire history. | Optional |
6060
| `use_proxy` | If set to true the Git clone process will honor `HTTP_PROXY` and `HTTPS_PROXY` variables if present for [working via a proxy](#using-git-behind-a-proxy). Default value is `false`. | Default |
6161
| `credentials` | Credentials to access the repository, if it requires authentication. It can an object containing `username` and `password` fields. Credentials are optional if you are using the [built-in Git integrations]({{site.baseurl}}/docs/integrations/git-providers/) . | Optional |
62+
|`timeout` | The maximum duration permitted to complete step execution in seconds (`s`), minutes (`m`), or hours (`h`), after which to automatically terminate step execution. For example, `timeout: 1.5h`. <br>The timeout supports integers and floating numbers, and can be set to a maximum of 2147483647ms (approximately 24.8 days). <br><br>If defined and set to either `0s/m/h` or `null`, the timeout is ignored and step execution is not terminated.<br>See [Add a timeout to terminate step execution](#add-a-timeout-to-terminate-step-execution). |Optional|
6263
| `fail_fast` | If a step fails and the process is halted. The default value is `true`. | Default |
6364
| `when` | Define a set of conditions that need to be satisfied in order to execute this step. You can find more information in the [Conditional execution of steps]({{site.baseurl}}/docs/pipelines/conditional-execution-of-steps/) article. | Optional |
6465
| `on_success`, `on_fail` and `on_finish` | Define operations to perform upon step completion using a set of predefined [Post-Step Operations]({{site.baseurl}}/docs/pipelines/post-step-operations/). | Optional |
@@ -267,6 +268,60 @@ steps:
267268
...
268269
```
269270
271+
## Add a timeout to terminate step execution
272+
To prevent steps from running beyond a specific duration if so required, you can add the `timeout` flag to the step.
273+
When defined:
274+
* The `timeout` is activated at the beginning of the step, before the step pulls images.
275+
* When the step's execution duration exceeds the duration defined for the `timeout`, the step is automatically terminated.
276+
277+
>**NOTE**:
278+
To define timeouts for parallel steps, see [Adding timeouts for parallel steps]({{site.baseurl}}/docs/pipelines/advanced-workflows/#adding-timeouts-for-parallel-steps).
279+
280+
Here's an example of the `timeout` field in the step:
281+
282+
{% highlight yaml %}
283+
{% raw %}
284+
step_name:
285+
type: git-clone
286+
title: Step Title
287+
description: Step description
288+
working_directory: /path
289+
repo: owner/repo
290+
git: my-git-provider
291+
revision: abcdef12345'
292+
use_proxy: false
293+
timeout: 45m
294+
credentials:
295+
username: user
296+
password: credentials
297+
fail_fast: false
298+
when:
299+
branch:
300+
ignore: [ develop ]
301+
on_success:
302+
...
303+
on_fail:
304+
...
305+
on_finish:
306+
...
307+
retry:
308+
...
309+
{% endraw %}
310+
{% endhighlight %}
311+
312+
313+
**Timeout info in logs**
314+
Timeout information is displayed in the logs, as in the example below.
315+
316+
{% include image.html
317+
lightbox="true"
318+
file="/images/steps/timeout-messages-in-logs.png"
319+
url="/images/steps/timeout-messages-in-logs.png"
320+
caption="Step termination due to timeout in logs"
321+
alt="Step termination due to timeout in logs"
322+
max-width="60%"
323+
%}
324+
270325
## Reuse a Git token from Codefresh integrations
271326

272327
You also have the capability to use one of your existing [Git integrations]({{site.baseurl}}/docs/integrations/git-providers/)

0 commit comments

Comments
 (0)