tests/micropython: Make tests behave in low memory condition. #17955
Conversation
This commit changes the "viper_ptr*_store_boundary" tests to make them fail more gracefully in low memory conditions. The original version of the tests compiled viper code blocks on the fly when it needed them, making them fail at runtime on some boards that do not come with enough memory for this test. This clashes with "run-tests.py"'s ability to look for a particular signature to mark tests as skipped due to not enough memory. Now compiled code blocks are generated at the beginning of the test inside an appropriate exception handler. In case of a memory error when pre-compiling a code block, the running test exits reporting a low memory condition to the test runner. This allows to have clean test runs on all platforms when it comes to viper pointer tests. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #17955 +/- ##
=======================================
Coverage 98.38% 98.38%
=======================================
Files 171 171
Lines 22296 22296
=======================================
Hits 21937 21937
Misses 359 359 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thanks for the PR! I ran Octoprobe on this PR, and it still fails sometimes with
|
|
This is odd. The error reported by Octoprobe is for an empty line in the source file, but that's right above a That said, I thought compile-time memory errors were caught by the test runner, but no worries. I can move the declaration for And if the remaining |
Summary
This PR changes the "viper_ptr*_store_boundary" tests to make them fail more gracefully in low memory conditions.
The original version of the tests compiled viper code blocks on the fly when it needed them, making them fail at runtime on some boards that do not come with enough memory for this test. This clashes with "run-tests.py"'s ability to look for a particular signature to mark tests as skipped due to not enough memory.
Now compiled code blocks are generated at the beginning of the test inside an appropriate exception handler. In case of a memory error when pre-compiling a code block, the running test exits reporting a low memory condition to the test runner. This allows to have clean test runs on all platforms when it comes to viper pointer tests.
Testing
Modified tests were run on an ESP8266 without any parameters to make sure they were reported as too large, and then with an appropriate prologue file to make sure the output is correct. The same tests were also executed on the Unix port on x64, just to be sure.
Trade-offs and Alternatives
I've tried to simplify certain parts of the test to counter the added complexity of generating blocks upfront, although it came with a test output data rearrangement to make it work. On the other hand, now those tests share most of their code except for the typed getters and setters.