-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Description
Currently long failure messages (over 40 lines by default, configurable with --max-error-lines
(#2576)) are cut from the middle. This is done to avoid huge messages messing up logs and reports, but the problem is that some valuable information may be lost. Another issue is that even the resulting messages are somewhat long and take lot of space.
The above is an old problem, but the situation is getting worse in RF 7.0 due failure messages being shown not only with tests,but also with each keyword and control structure. Earlier keywords and control structures in the result model didn't have a message at all, but it was added as part of the result model cleanup (#4883). The motivation was this:
- We are adding JSON representation to the result model (Support JSON serialization with result model #4847) and want the model to be as stable and future-proof as possible.
- We likely want to in the future allow running individual keywords outside Robot core. At that point we want the result model to have a message, not only status as earlier.
- In some special cases (at least with
--flatten-keywords
and--remove-keywords
) we want to add some extra notes to result objects. Earlier we used documentation for that, but it was odd because control structures such as FOR loops cannot otherwise have a documentation. Using the message for this purpose works much better.
Now that also keywords and control structures also have a message, the same message can be shown on multiple levels in the log file. That's rather annoying in general, but gets especially irritating if the message is long. To mitigate this issue, and to fix the old issue with long messages, I propose we do the following:
- Show only the beginning of long failure messages in log and report by default. I believe we should show so much that typical messages are shown fully, but considerably less than 40 lines that is the current maximum. We could possibly also show more with tests than with keywords and control structures.
- Have some way to show the full message. Probably a simple "Show more." button/link would be fine.
- Stop cutting long messages otherwise. This can increase output file sizes, but I doubt the difference is too big.