-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Execution in teardown should continue after keyword timeout #3398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Could you please explain the problem a bit more? I'm on mobile and cannot open the zip file. |
If test teardown is and keyword and if any of it's sub keywords has timeout and fails due to timeout, no other sub keywords will be executed. Here is the same in text format, quite short: *** Keywords ***
Teardown Keyword With Timeout Fail
Log To Console TEARDOWN START
Failing Kw Due To Timeout
Passing KW
Log To Console TEARDOWN ENDS
Teardown Keyword With Fail
Log To Console TEARDOWN START
Failing Kw
Passing KW
Log To Console TEARDOWN ENDS
Failing Kw Due To Timeout
[Timeout] 1 sec
Sleep 2
Failing Kw
Fail This keyword allways fails
Passing KW
Log I am always doing cleanup well and going pass
*** Test Cases ***
Teardown Continues On Failure
[Teardown] Teardown Keyword With Fail
Log Test executed
Teardown Stop In keyword Timeout
[Teardown] Teardown Keyword With Timeout Fail
Log Test executed |
That's by design and ought to also be documented. |
Now that I think about this more, I guess it would be fine to continue teardown after that keyword that has failed due to a timeout. Is someone interested to study how big a change it would be? I probably won't have time for that before RF 3.2. |
https://robotframework.slack.com/archives/C3C28F9DF/p1572513770127500 Test Teardown
Keyword 1
# Which contains:
Keyword 1.1
Keyword 1.2
Keyword 1.3
Keyword 1.4
# Which contains:
Keyword 1.4.1 # <--- This one times out
Keyword 1.5
Keyword 2
# Which contains:
Keyword 2.1
Keyword 3
# Which contains:
Keyword 3.1
Keyword 4
# Which contains:
Keyword 4.1 Keyword 1.4.1 times out, but Keyword 1.5, Keyword 2 and the rest are not executed |
Looked at this quickly and noticed this might be very easy to fix. These changes seem to be enough: --- a/src/robot/errors.py
+++ b/src/robot/errors.py
@@ -146,10 +146,10 @@ class ExecutionStatus(RobotError):
return False
if templated:
return True
- if self.keyword_timeout:
- return False
if teardown:
return True
+ if self.keyword_timeout:
+ return False Could someone encountering this in real environment test the above changes? If the fix works, and there are no problems, this could be included in RF 3.2. Some tests would obviously be needed but adding them shouldn't be overly complicated. I can do them myself, but if someone wants to create a PR that would be even better. |
I executed Robot's acceptance tests with the above changes and the only test that failed explicitly verified that execution in teardown would stop after a timeout. That's great news, because now we know there are tests for this feature and just update them to match the new design and possibly add some new ones. Should also check what the User Guide says about this. |
I just wanted to add comment to ensure there is still ability to add timeout to teardown itself. For instance some of keyword are jumming. But tested your change request with following code and seems it fails due to timeout after 10 sec of executing 'TDKW' E.g.
|
Yeah, you should still be able to add a timeout to the top level teardown keyword to stop its execution for good. With the above simple change that doesn't work fully, though. If you have, for example, a keyword like the one below, keywords after the point where the timeout occurs won't be executed (i.e. the example won't fail with "Should not be executed"). Unfortunately those keywords are run so that they timeout, so the teardown actually fails four times with "Keyword timeout 100 milliseconds exceeded." message. That's not acceptable. If a keyword fails for timeout, remaining keywords in it should be skipped. *** Keywords ***
Example
[Timeout] 0.1 seconds
Sleep 1 second
Fail Should not be executed
Fail Should not be executed
Fail Should not be executed |
^^ sounds like a prime candidate for a negative test |
Any news on this one? We are experiencing this issue in a real environment, with teardown not completing fully. |
Let's see could we get this done in RF 4.1. |
The above commit changes the logic so that keyword where the timeout occurs is stopped but other keywords can continue execution. Would be great if someone who has encountered this issue could validate the behavior in their environment. There's going to be RF 4.1 alpha or beta release somewhat soon, but you can already before that install the latest code directly from GitHub is you want. |
Turning keyword timeouts that occurred in teardowns to normal errors is needed as a fix for #3398. Better to do that explicitly than as a side-effect of `can_continue` call.
Turning keyword timeouts that occurred in teardowns to normal errors is needed as a fix for #3398. Better to do that explicitly than as a side-effect of `can_continue` call.
Version information
Robot Framework version: 3.1.2
Python interpreter type Python and version: Python 2.7.16
Operating system and its version: Centos 7:3.10.0-1062.4.1.el7.x86_64
Steps to reproduce the problem.
See attachment
teardown_not_all_kw_executed.zip
EXPECTED RESULT:
https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#test-setup-and-teardown
"
In addition, all the keywords in the teardown are also executed even if one of them fails. This continue on failure functionality can be used also with normal keywords, but inside teardowns it is on by default.
"
The text was updated successfully, but these errors were encountered: