runtests.py looks like it's hanging at first (though it's not) #3393
Comments
|
A simple fix might be to count the pytest run as 4 tasks (or whatever level of parallelism we are using). |
|
Yeah -- though that wouldn't help with the "nothing changes for a long time" problem, which I think is probably the bigger one. A simple fix that would also address that part would be to revert the bits of #3019 that added a |
|
I wonder if the README should focus more on running pytest and less on runtests.py. That's the advice I give beginners. pytest is something they already know, and it's easy to select tests and to run pdb on tests (once you have discovered |
|
We should probably also mention |
|
Agreed on all counts re the README. I'll open an issue or two for those. |
|
I thought @JukkaL didn't want to run pytest concurrently with other tests because of the very large peak memory usage. Other than that, the performance impact is not necessarily on the total time, but on the responsiveness of the system while the tests are running: with too many CPU-intensive tasks running concurrently, Windows lags on any user operations. I'm guessing Linux handles it better? Regardless, do we want to pass through pytest progress bar to the console? |
|
For what it's worth, I've also noticed Mac OS becoming unresponsive while I run the mypy test suite. |
|
@pkch Earlier we had multiple pytest runs running in parallel (each with multiple processes) and that caused Travis build failures due to very high memory use. Just running a single parallel pytest run together with other, single-process tasks might work, though I share the concern that running many processes will likely cause system unresponsiveness. |
Fix #3393 This results in pytest output being shown as a passthrough at the beginning of runtests.py.
Starting our test suite, it looks like this and stays this way for many seconds:
When @Nurdok saw this on first trying to run tests on his Windows laptop, he figured the tests were hanging -- and when I first looked at it with him, I thought the same thing too.
There are probably two confusing things here:
Probably the first of these is most important.
The root cause of both issues is that that 1 "task" which is running is actually a
py.testprocess, which is running many test cases and doing its own parallelism among those -- but neither the parallelism nor the ongoing progress of finishing those test cases is being reported. Before #3019 we ran the bigpy.testprocess in parallel with other tasks, some of which finished quickly, which avoided both of these.The real fix for this will be to complete the migration from our custom
runtests.pyrunner to plainpy.test. In the meantime, we should do something simple to make this not confusing.The text was updated successfully, but these errors were encountered: