-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Support custom statuses #3625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This would be nice already in RF 4.1. |
Too big task for RF 4.1. I moved this tentatively to RF 5.0 scope but we haven't yet started more detailed planning of that release and it's not clear will this one be actually included or not. If you find this important, comment here and explain your use case! |
Oooops, there's PR #3869 adding this functionality. If the PR implements what's needed and doesn't have problems, this certainly can be included into RF 4.1 I don't have time to review the PR right now, but I moved this issue back to RF 4.1 to make sure I don't forget about it. |
PR #3869 is a good start. There is, however, so much work still to be done both internally to handle custom statuses and in log report to show them nicely in statistics and elsewhere that I move this issue to RF 5.0 scope. |
I would like to add here a wish that custom statuses would be recorded in the Given RF Jenkins Plugin is in Java, meaning we cannot leverage the parser, it means we have to support this by parsing the output.xml ourselves. |
I just rechecked PR #3869 and noticed that there's even more work, including big design decisions, to be made for this to be reasonable enhancement in RF 5.0. In fact, it's way too big even for RF 5.1 that is going to b a relatively small release. On top of that, there hasn't been too much interest towards this issue. I still consider this a nice enhancement but we can return to it later if there's more interest. |
Hi Pekka Klärck ! |
This feature isn't currently targeted for any planned release. If someone is interested to look at developing it, I'd be happy to help. Paying for the development is possible as well. The best way to get started with that would be joining the Robot Framework Foundation. |
It would be convenient to be able to configure Robot Framework to have custom statuses in addition to the current PASS and FAIL and the forthcoming SKIP (#3622). Currently tests can have tags to differentiate them, but explicit statuses would make the distinction more clear.
This functionality could be implemented so that new statuses are defined based on old statuses and tags. The specification could be given from the command like with a command line option like
--addstatus
with an argument in format such asNEWSTATUS:OLDSTATUS:TAGPATTERN:COLOR
. In this specNEWSTATUS
would be the name of the new status to add,OLDSTATUS
would be the old status to base this status on,TAGPATTERN
would be the tag or tag pattern (e.g.tag
,tag:*
ortag1ORtag2
) matching test must have, andCOLOR
would be the color associated with the status (possibly optional). Example:One use case for this feature would be separating tests skipped explicitly during execution and tests skipped by the forthcoming
--skiponfailure
feature. This would be convenient for those who oppose removing the criticality concept (#3624):The text was updated successfully, but these errors were encountered: