Skip to content

Commit 5fa06d1

Browse files
oboehmeroboehmer
authored
Enable continuable failures via robot:continue-on-failure test or kw tag (#3925)
Also supports `robot:continue-on-failure-recursive` to enable the mode recursively. See #2285 for details. Some tests are failing due to recent changes to master that aren't taken into account. They will be fixed separately.
1 parent b3ac511 commit 5fa06d1

14 files changed

+468
-19
lines changed

‎atest/robot/running/continue_on_failure_tag.robot

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
*** Settings ***
2+
Suite Setup Run Tests ${EMPTY} running/continue_on_failure_tag.robot
3+
Resource atest_resource.robot
4+
5+
*** Test Cases ***
6+
Continue in test with tag
7+
Check Test Case ${TESTNAME}
8+
9+
Continue in test with Set Tags
10+
Check Test Case ${TESTNAME}
11+
12+
Continue in user keyword with tag
13+
Check Test Case ${TESTNAME}
14+
15+
Continue in test with tag and UK without tag
16+
Check Test Case ${TESTNAME}
17+
18+
Continue in test with tag and nested UK with and without tag
19+
Check Test Case ${TESTNAME}
20+
21+
Continue in test with tag and two nested UK with tag
22+
Check Test Case ${TESTNAME}
23+
24+
Continue in FOR loop with tag
25+
Check Test Case ${TESTNAME}
26+
27+
Continue in FOR loop with Set Tags
28+
Check Test Case ${TESTNAME}
29+
30+
No continue in FOR loop without tag
31+
Check Test Case ${TESTNAME}
32+
33+
Continue in FOR loop in UK with tag
34+
Check Test Case ${TESTNAME}
35+
36+
Continue in FOR loop in UK without tag
37+
Check Test Case ${TESTNAME}
38+
39+
Continue in IF with tag
40+
Check Test Case ${TESTNAME}
41+
42+
Continue in IF with set and remove tag
43+
Check Test Case ${TESTNAME}
44+
45+
No continue in IF without tag
46+
Check Test Case ${TESTNAME}
47+
48+
Continue in IF in UK with tag
49+
Check Test Case ${TESTNAME}
50+
51+
No continue in IF in UK without tag
52+
Check Test Case ${TESTNAME}
53+
54+
Continue in Run Keywords with tag
55+
Check Test Case ${TESTNAME}
56+
57+
Recursive continue in test with tag and two nested UK without tag
58+
Check Test Case ${TESTNAME}
59+
60+
Recursive continue in test with Set Tags and two nested UK without tag
61+
Check Test Case ${TESTNAME}
62+
63+
Recursive continue in test with tag and two nested UK with and without tag
64+
Check Test Case ${TESTNAME}
65+
66+
Recursive continue in user keyword
67+
Check Test Case ${TESTNAME}
68+
69+
No recursive continue in user keyword
70+
Check Test Case ${TESTNAME}

‎atest/robot/tags/tag_stat_include_and_exclude.robot

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ Resource atest_resource.robot
55
*** Variables ***
66
${DATA SOURCE} tags/include_and_exclude.robot
77
${F} force
8+
${INTERNAL} robot:just-an-example
89
${I1} incl1
910
${I2} incl 2
1011
${I3} incl_3
@@ -32,6 +33,14 @@ Include With Patterns
3233
--TagStatInc incl_? @{INCL}
3334
--TagStatInc *cl3 --TagStatInc i*2 ${E3} ${I2} ${I3}
3435

36+
Include to show internal tags
37+
--tagstatinclude incl1 --tagstatinclude robot:* ${I1} ${INTERNAL}
38+
--tagstatinclude robot:* ${INTERNAL}
39+
--tagstatinclude * @{ALL} ${INTERNAL}
40+
41+
Include and exclude internal
42+
--tagstatinclude incl1 --tagstatinclude robot:* --tagstatexclude robot:* ${I1}
43+
3544
One Exclude
3645
--tagstatexclude excl1 ${E2} ${E3} ${F} @{INCL}
3746

‎atest/robot/tags/tag_stat_include_and_exclude_with_rebot.robot

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Resource rebot_resource.robot
88
${DATA SOURCE} tags/include_and_exclude.robot
99
${INPUT FILE} %{TEMPDIR}${/}robot-test-tagstat.xml
1010
${F} force
11+
${INTERNAL} robot:just-an-example
1112
${I1} incl1
1213
${I2} incl 2
1314
${I3} incl_3
@@ -35,6 +36,14 @@ Include With Patterns
3536
--TagStatInc incl_? @{INCL}
3637
--TagStatInc *cl3 --TagStatInc i*2 ${E3} ${I2} ${I3}
3738

39+
Include to show internal tags
40+
--tagstatinclude incl1 --tagstatinclude robot:* ${I1} ${INTERNAL}
41+
--tagstatinclude robot:* ${INTERNAL}
42+
--tagstatinclude * @{ALL} ${INTERNAL}
43+
44+
Include and exclude internal
45+
--tagstatinclude incl1 --tagstatinclude robot:* --tagstatexclude robot:* ${I1}
46+
3847
One Exclude
3948
--tagstatexclude excl1 ${E2} ${E3} ${F} @{INCL}
4049

‎atest/testdata/running/continue_on_failure_tag.robot

Lines changed: 263 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,263 @@
1+
*** Variables ***
2+
${HEADER} Several failures occurred:
3+
4+
*** Test Cases ***
5+
Continue in test with tag
6+
[Documentation] FAIL ${HEADER}\n\n
7+
... 1) 1\n\n
8+
... 2) 2
9+
[Tags] robot:continue-on-failure
10+
Fail 1
11+
Fail 2
12+
Log This should be executed
13+
14+
Continue in test with Set Tags
15+
[Documentation] FAIL ${HEADER}\n\n
16+
... 1) 1\n\n
17+
... 2) 2
18+
Set Tags robot:continue-on-failure
19+
Fail 1
20+
Fail 2
21+
Log This should be executed
22+
23+
Continue in user keyword with tag
24+
[Documentation] FAIL ${HEADER}\n\n
25+
... 1) kw1a\n\n
26+
... 2) kw1b
27+
Failure in user keyword with tag
28+
Fail This should not be executed
29+
30+
Continue in test with tag and UK without tag
31+
[Documentation] FAIL ${HEADER}\n\n
32+
... 1) kw2a\n\n
33+
... 2) This should be executed
34+
[Tags] robot:continue-on-failure
35+
Failure in user keyword without tag
36+
Fail This should be executed
37+
38+
Continue in test with tag and nested UK with and without tag
39+
[Documentation] FAIL ${HEADER}\n\n
40+
... 1) kw1a\n\n
41+
... 2) kw1b\n\n
42+
... 3) kw2a\n\n
43+
... 4) This should be executed
44+
[Tags] robot:continue-on-failure
45+
Failure in user keyword with tag run_kw=Failure in user keyword without tag
46+
Fail This should be executed
47+
48+
Continue in test with tag and two nested UK with tag
49+
[Documentation] FAIL ${HEADER}\n\n
50+
... 1) kw1a\n\n
51+
... 2) kw1b\n\n
52+
... 3) kw1a\n\n
53+
... 4) kw1b\n\n
54+
... 5) This should be executed
55+
[Tags] robot:continue-on-failure
56+
Failure in user keyword with tag run_kw=Failure in user keyword with tag
57+
Fail This should be executed
58+
59+
Continue in FOR loop with tag
60+
[Documentation] FAIL ${HEADER}\n\n
61+
... 1) loop-1\n\n
62+
... 2) loop-2\n\n
63+
... 3) loop-3
64+
[Tags] robot:continue-on-failure
65+
FOR ${val} IN 1 2 3
66+
Fail loop-${val}
67+
END
68+
69+
Continue in FOR loop with Set Tags
70+
[Documentation] FAIL ${HEADER}\n\n
71+
... 1) loop-1\n\n
72+
... 2) loop-2\n\n
73+
... 3) loop-3
74+
FOR ${val} IN 1 2 3
75+
Set Tags robot:continue-on-failure
76+
Fail loop-${val}
77+
END
78+
79+
No continue in FOR loop without tag
80+
[Documentation] FAIL loop-1
81+
FOR ${val} IN 1 2 3
82+
Fail loop-${val}
83+
END
84+
85+
Continue in FOR loop in UK with tag
86+
[Documentation] FAIL ${HEADER}\n\n
87+
... 1) kw-loop-1\n\n
88+
... 2) kw-loop-2\n\n
89+
... 3) kw-loop-3
90+
FOR loop in in user keyword with tag
91+
92+
Continue in FOR loop in UK without tag
93+
[Documentation] FAIL kw-loop-1
94+
FOR loop in in user keyword without tag
95+
96+
Continue in IF with tag
97+
[Documentation] FAIL ${HEADER}\n\n
98+
... 1) 1\n\n
99+
... 2) 2\n\n
100+
... 3) 3\n\n
101+
... 4) 4
102+
[Tags] robot:continue-on-failure
103+
IF 1==1
104+
Fail 1
105+
Fail 2
106+
END
107+
IF 1==2
108+
No Operation
109+
ELSE
110+
Fail 3
111+
Fail 4
112+
END
113+
114+
Continue in IF with set and remove tag
115+
[Documentation] FAIL ${HEADER}\n\n
116+
... 1) 1\n\n
117+
... 2) 2\n\n
118+
... 3) 3
119+
Set Tags robot:continue-on-failure
120+
IF 1==1
121+
Fail 1
122+
Fail 2
123+
END
124+
Remove Tags robot:continue-on-failure
125+
IF 1==2
126+
No Operation
127+
ELSE
128+
Fail 3
129+
Fail this is not executed
130+
END
131+
132+
No continue in IF without tag
133+
[Documentation] FAIL 1
134+
IF 1==1
135+
Fail 1
136+
Fail This should not be executed
137+
END
138+
139+
Continue in IF in UK with tag
140+
[Documentation] FAIL ${HEADER}\n\n
141+
... 1) kw1a\n\n
142+
... 2) kw1b\n\n
143+
... 3) kw1c\n\n
144+
... 4) kw1d
145+
IF in user keyword with tag
146+
147+
No continue in IF in UK without tag
148+
[Documentation] FAIL kw1a
149+
IF in user keyword without tag
150+
151+
Continue in Run Keywords with tag
152+
[Documentation] FAIL ${HEADER}\n\n
153+
... 1) 1\n\n
154+
... 2) 2
155+
[Tags] robot:continue-on-failure
156+
Run Keywords Fail 1 AND Fail 2
157+
158+
Recursive continue in test with tag and two nested UK without tag
159+
[Documentation] FAIL ${HEADER}\n\n
160+
... 1) kw2a\n\n
161+
... 2) kw2b\n\n
162+
... 3) kw2a\n\n
163+
... 4) kw2b\n\n
164+
... 5) This should be executed
165+
[Tags] robot:continue-on-failure-recursive
166+
Failure in user keyword without tag run_kw=Failure in user keyword without tag
167+
Fail This should be executed
168+
169+
Recursive continue in test with Set Tags and two nested UK without tag
170+
[Documentation] FAIL ${HEADER}\n\n
171+
... 1) kw2a\n\n
172+
... 2) kw2b\n\n
173+
... 3) kw2a\n\n
174+
... 4) kw2b\n\n
175+
... 5) This should be executed
176+
Set Tags robot:continue-on-failure-recursive
177+
Failure in user keyword without tag run_kw=Failure in user keyword without tag
178+
Fail This should be executed
179+
180+
Recursive continue in test with tag and two nested UK with and without tag
181+
[Documentation] FAIL ${HEADER}\n\n
182+
... 1) kw1a\n\n
183+
... 2) kw1b\n\n
184+
... 3) kw2a\n\n
185+
... 4) kw2b\n\n
186+
... 5) This should be executed
187+
[Tags] robot:continue-on-failure-recursive
188+
Failure in user keyword with tag run_kw=Failure in user keyword without tag
189+
Fail This should be executed
190+
191+
Recursive continue in user keyword
192+
[Documentation] FAIL ${HEADER}\n\n
193+
... 1) kw1a\n\n
194+
... 2) kw1b\n\n
195+
... 3) kw2a\n\n
196+
... 4) kw2b
197+
Failure in user keyword with recursive tag run_kw=Failure in user keyword without tag
198+
Fail This should not be executed
199+
200+
No recursive continue in user keyword
201+
[Documentation] FAIL kw2a
202+
Failure in user keyword without tag run_kw=Failure in user keyword with recursive tag
203+
Fail This should not be executed
204+
205+
*** Keywords ***
206+
207+
Failure in user keyword with tag
208+
[Arguments] ${run_kw}=No Operation
209+
[Tags] robot:continue-on-failure
210+
Fail kw1a
211+
Fail kw1b
212+
Log This should be executed
213+
Run Keyword ${run_kw}
214+
215+
Failure in user keyword without tag
216+
[Arguments] ${run_kw}=No Operation
217+
Fail kw2a
218+
Fail kw2b
219+
Run Keyword ${run_kw}
220+
221+
Failure in user keyword with recursive tag
222+
[Arguments] ${run_kw}=No Operation
223+
[Tags] robot:continue-on-failure-recursive
224+
Fail kw1a
225+
Fail kw1b
226+
Log This should be executed
227+
Run Keyword ${run_kw}
228+
229+
FOR loop in in user keyword with tag
230+
[Tags] robot:continue-on-failure
231+
FOR ${val} IN 1 2 3
232+
Fail kw-loop-${val}
233+
END
234+
235+
FOR loop in in user keyword without tag
236+
FOR ${val} IN 1 2 3
237+
Fail kw-loop-${val}
238+
END
239+
240+
IF in user keyword with tag
241+
[Tags] robot:continue-on-failure
242+
IF 1==1
243+
Fail kw1a
244+
Fail kw1b
245+
END
246+
IF 1==2
247+
No Operation
248+
ELSE
249+
Fail kw1c
250+
Fail kw1d
251+
END
252+
253+
IF in user keyword without tag
254+
IF 1==1
255+
Fail kw1a
256+
Fail kw1b
257+
END
258+
IF 1==2
259+
No Operation
260+
ELSE
261+
Fail kw1c
262+
Fail kw1d
263+
END

‎atest/testdata/tags/include_and_exclude.robot

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
*** Settings ***
2-
Force Tags force
2+
Force Tags force robot:just-an-example
33

44
*** Test Cases ***
55
Incl-1

0 commit comments

Comments
 (0)