Skip to content

Add memoization to --dryrun #3241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
myjniak opened this issue Jul 12, 2019 · 3 comments
Closed

Add memoization to --dryrun #3241

myjniak opened this issue Jul 12, 2019 · 3 comments

Comments

@myjniak
Copy link

myjniak commented Jul 12, 2019

In case when tests are getting pretty complicated, a test dryrun may take even several seconds.
When multiple tests of similar content are added, it can even exceed several minutes.

In my case, dry run check is performed on the same part of code lots of times.
If a keyword has already been verified, you can copy the results each time it's called again.

It would greatly improve dryrun performance.
A 7-minute dryrun added to attachments.
log.zip

@pekkaklarck
Copy link
Member

I've never heard dry-run taking that much time but I believe it can happen with complex enough data. Not validating keywords that have already been validated sounds like a good idea, but I'm not sure how easy it would be. Some challenges listed below:

  • How to reliably that detect a keyword in one suite is same as a keyword in another suite? Keywords and even resource files where they are can have same names. Resource files having a references to the original source path would help but I don't remember do they have it. Could definitely be added if not.

  • What should actually be done if we detect that a keyword has already been validated? Is it enough to only add a note to the log about it or should the whole validation be copied there?

Problems above are likely solvable and prototyping this would be great. I personally won't have time for it in the foreseeable future, but perhaps you @myjniak could look at it yourself. Or perhaps @mkorpela would be interested because, AFAIK, pabot also uses dry-run internally and would benefit from speedups.

@mkorpela
Copy link
Member

Pabot internally hacks away most of the data in dryrun as pabot is only interested about the names of the test cases that will be executed.
This was actually done because some people complaint about the operation taking so much time. And the hack improved the situation.

@pekkaklarck
Copy link
Member

No updates since 2019 so this doesn't seem to be too critical. New issue can be opened, or this one reopened, later if needed. We'd most importantly needed a PR implementing this.

@pekkaklarck pekkaklarck closed this as not planned Won't fix, can't repro, duplicate, stale May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants