Skip to content

Commit 8d92390

Browse files
pekkaklarckyanne
pekkaklarck
authored and
yanne
committed
Skip fine-tuning. robotframework#3622
- Fix logic bug with skip after _one_ continuable failure. - Enhance message with --skiponfailure. - Test tuning.
1 parent f3ef21c commit 8d92390

File tree

5 files changed

+85
-25
lines changed

5 files changed

+85
-25
lines changed

‎atest/robot/running/skip.robot

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,10 @@ Fail in Teardown After Skip In Body
4646
Skip in Teardown After Skip In Body
4747
Check Test Case ${TEST NAME}
4848

49-
Skip with Continuable Failures
49+
Skip with Continuable Failure
50+
Check Test Case ${TEST NAME}
51+
52+
Skip with Multiple Continuable Failures
5053
Check Test Case ${TEST NAME}
5154

5255
Skip with Pass Execution in Teardown

‎atest/testdata/running/skip/skip.robot

Lines changed: 74 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -18,15 +18,17 @@ Skip with Library Keyword
1818
Skip If Keyword with True Condition
1919
[Documentation] SKIP 1 == 1
2020
Skip If 1 == 1
21+
Fail Should not be executed!
2122

2223
Skip If Keyword with True Condition And Custom Message
2324
[Documentation] SKIP Skipped with abandon.
2425
Skip If 1 == 1 Skipped with abandon.
26+
Fail Should not be executed!
2527

2628
Skip If Keyword with False Condition
27-
[Documentation] FAIL AssertionError
29+
[Documentation] FAIL Should be executed!
2830
Skip If 1 == 2
29-
Fail
31+
Fail Should be executed!
3032

3133
Skip Keyword with Custom Message
3234
[Documentation] SKIP Skipped due to reasons
@@ -54,7 +56,12 @@ Remaining teardown keywords aren't run after skip
5456
[Teardown] Skip with keywords before and after
5557

5658
Skip in Teardown After Failure In Body
57-
[Documentation] SKIP Skipped in teardown:\nTeardown skip\n\nEarlier message:\nFailure in body!
59+
[Documentation] SKIP
60+
... Skipped in teardown:
61+
... Teardown skip
62+
...
63+
... Earlier message:
64+
... Failure in body!
5865
Fail Failure in body!
5966
[Teardown] Skip Teardown skip
6067

@@ -64,26 +71,62 @@ Teardown is executed after skip
6471
[Teardown] Log Teardown is executed!
6572

6673
Fail in Teardown After Skip In Body
67-
[Documentation] SKIP Skip in body\n\nAlso teardown failed:\nTeardown failed!
74+
[Documentation] SKIP
75+
... Skip in body
76+
...
77+
... Also teardown failed:
78+
... Teardown failed!
6879
Skip Skip in body
6980
[Teardown] Fail Teardown failed!
7081

7182
Skip in Teardown After Skip In Body
72-
[Documentation] SKIP Skipped in teardown:\nTeardown skip\n\nEarlier message:\nSkip in body
83+
[Documentation] SKIP
84+
... Skipped in teardown:
85+
... Teardown skip
86+
...
87+
... Earlier message:
88+
... Skip in body
7389
Skip Skip in body
7490
[Teardown] Skip Teardown skip
7591

76-
Skip with Continuable Failures
77-
[Documentation] SKIP Skipping should stop execution but test should still fail\n\nAlso failures occurred\n\n1) We can continue!\n\n2) We can continue again!
92+
Skip with Continuable Failure
93+
[Documentation] SKIP
94+
... Skipping should stop execution but test should still fail
95+
...
96+
... Also failure occurred:
97+
... We can continue!
98+
Run Keyword And Continue On Failure
99+
... Fail We can continue!
100+
Skip Skipping should stop execution but test should still fail
101+
Fail Should not be executed!
102+
103+
Skip with Multiple Continuable Failures
104+
[Documentation] SKIP
105+
... Skip after two failures
106+
...
107+
... Also failures occurred:
108+
...
109+
... 1) We can continue!
110+
...
111+
... 2) We can continue again!
78112
Run Keyword And Continue On Failure
79113
... Fail We can continue!
80114
Run Keyword And Continue On Failure
81115
... Fail We can continue again!
82-
Skip Skipping should stop execution but test should still fail
116+
Skip Skip after two failures
83117
Fail Should not be executed!
84118

85119
Skip in Teardown After Continuable Failures
86-
[Documentation] SKIP Skipped in teardown:\nTeardown skip\n\nEarlier message:\nSeveral failures occurred:\n\n1) We can continue!\n\n2) We can continue again!
120+
[Documentation] SKIP
121+
... Skipped in teardown:
122+
... Teardown skip
123+
...
124+
... Earlier message:
125+
... Several failures occurred:
126+
...
127+
... 1) We can continue!
128+
...
129+
... 2) We can continue again!
87130
Run Keyword And Continue On Failure
88131
... Fail We can continue!
89132
Run Keyword And Continue On Failure
@@ -92,32 +135,43 @@ Skip in Teardown After Continuable Failures
92135

93136
Skip with Pass Execution in Teardown
94137
[Documentation] SKIP Skip in body
95-
[Teardown] Pass Execution Thou shall pass
96-
Skip Skip in body
138+
Skip Skip in body
139+
[Teardown] Run Keywords
140+
... Pass Execution Thou shall pass
141+
... AND
142+
... Fail Should not be executed!
97143

98144
Skip in Teardown with Pass Execution in Body
99145
[Documentation] SKIP Then we skip
146+
Pass Execution First we pass
100147
[Teardown] Skip Then we skip
101-
Pass Execution First we pass
102148

103149
Skipped with --skip
104-
[Tags] skip-this
105-
[Documentation] SKIP ${TEST_OR_TASK} skipped with --skip command line option.
150+
[Documentation] SKIP ${TEST_OR_TASK} skipped with '--skip' command line option.
151+
[Tags] skip-this
106152
Fail
107153

108154
Skipped with --SkipOnFailure
109-
[Tags] skip-on-failure
110-
[Documentation] SKIP ${TEST_OR_TASK} skipped with --SkipOnFailure, original error:\nAssertionError
111-
Fail
155+
[Documentation] SKIP
156+
... ${TEST_OR_TASK} failed but its tags matched '--SkipOnFailure' and it was marked skipped.
157+
...
158+
... Original failure:
159+
... Ooops, we fail!
160+
[Tags] skip-on-failure
161+
Fail Ooops, we fail!
112162

113163
--NonCritical Is an Alias for --SkipOnFailure
114-
[Tags] non-crit
115-
[Documentation] SKIP ${TEST_OR_TASK} skipped with --SkipOnFailure, original error:\nAssertionError
164+
[Documentation] SKIP
165+
... ${TEST_OR_TASK} failed but its tags matched '--SkipOnFailure' and it was marked skipped.
166+
...
167+
... Original failure:
168+
... AssertionError
169+
[Tags] non-crit
116170
Fail
117171

118172
--Critical can be used to override --SkipOnFailure
119-
[Tags] dynamic-skip crit
120173
[Documentation] FAIL AssertionError
174+
[Tags] dynamic-skip crit
121175
Fail
122176

123177
Failing Test

‎src/robot/errors.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,9 @@ def _format_message(self, errors):
206206
skip_idx = errors.index([e for e in errors if e.skip][0])
207207
skip_msg = messages[skip_idx]
208208
messages = messages[:skip_idx] + messages[skip_idx+1:]
209-
prefix = '%s\n\nAlso failures occurred' % skip_msg
209+
if len(messages) == 1:
210+
return '%s\n\nAlso failure occurred:\n%s' % (skip_msg, messages[0])
211+
prefix = '%s\n\nAlso failures occurred:' % skip_msg
210212
return '\n\n'.join(
211213
[prefix] +
212214
['%d) %s' % (i, m) for i, m in enumerate(messages, start=1)]

‎src/robot/running/runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ def visit_test(self, test):
131131
if self._skipped_tags.match(test.tags):
132132
status.test_skipped(
133133
test_or_task(
134-
"{Test} skipped with --skip command line option.",
134+
"{Test} skipped with '--skip' command line option.",
135135
self._settings.rpa))
136136
if not status.failed and not test.name:
137137
status.test_failed(

‎src/robot/running/status.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -164,8 +164,9 @@ def test_failed(self, failure):
164164
if hasattr(failure, 'skip') and failure.skip:
165165
self.test_skipped(failure)
166166
elif self._skip_on_failure:
167-
msg = ("%s skipped with --SkipOnFailure, original error:\n%s"
168-
% (test_or_task('{Test}', self._rpa) ,unic(failure)))
167+
msg = ("%s failed but its tags matched '--SkipOnFailure' and it was "
168+
"marked skipped.\n\nOriginal failure:\n%s"
169+
% (test_or_task('{Test}', self._rpa), unic(failure)))
169170
self.failure.test = msg
170171
self.skipped = True
171172
else:

0 commit comments

Comments
 (0)