Skip to content

Conversation

jepler
Copy link
Contributor

@jepler jepler commented Aug 27, 2025

Summary

There are no new findings, but this ensures the code paths unique to mpy-cross are checked with the sanitizers.

I think that new coverage may turn up as well (e.g., MICROPY_DYNAMIC_COMPILER) if I've got the CI rules right.

Testing

I ran the coverage, ubsan, and asan commands in ci.sh locally.

Trade-offs and Alternatives

Not a trade-off per se, but specifically adding mpy-cross tests that would exercise all emitters when built with coverage & sanitizers would potentially have benefits.

There are no new findings, but this ensures the code paths unique to
mpy-cross are checked with the sanitizers.

Signed-off-by: Jeff Epler <jepler@gmail.com>
Copy link

codecov bot commented Aug 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.75%. Comparing base (8c47e44) to head (4454c9c).
⚠️ Report is 4 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (8c47e44) and HEAD (4454c9c). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (8c47e44) HEAD (4454c9c)
2 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master   #17997       +/-   ##
===========================================
- Coverage   98.38%   87.75%   -10.64%     
===========================================
  Files         171      183       +12     
  Lines       22296    25149     +2853     
===========================================
+ Hits        21937    22070      +133     
- Misses        359     3079     +2720     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jepler jepler force-pushed the sanitize-mpy-cross branch from c588a2c to 5e1e39b Compare August 27, 2025 21:19
Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Signed-off-by: Jeff Epler <jepler@gmail.com>
@jepler jepler force-pushed the sanitize-mpy-cross branch from 5e1e39b to 4454c9c Compare August 27, 2025 21:43
@jepler
Copy link
Contributor Author

jepler commented Aug 27, 2025

hmm does anyone spot what I messed up about the codecov upload?

@dpgeorge
Copy link
Member

I don't understand how this works. It looks like it's merging results from running mpy-cross into the coverage results from running the unix port, is that right? If so I don't think that's correct because mpy-cross is built with different compile-time options to the unix port, and so different #if's are active and potentially other more complex differences. So won't that confuse gcov/codecov?

@jepler
Copy link
Contributor Author

jepler commented Aug 28, 2025

I had hoped that it would be able to integrate line coverage data from both mpy-cross and micropython in a single report. It does appear to ingest two "gcno" files for sources that appear in both,

warning - 2025-08-27 21:50:37,958 -- Running gcov on the following list of files:…
warning - 2025-08-27 21:50:37,962 -- /home/runner/work/micropython/micropython/mpy-cross/build/py/compile.gcno
warning - 2025-08-27 21:50:37,982 -- /home/runner/work/micropython/micropython/ports/unix/build-coverage/py/compile.gcno

but unfortunately it looks like the earlier one is used and the latter one is ignored(?)

Locally using "gcovr" I can create merged coverage .. in this case, the line coverage of py/compile.c increased from 1761 to 1766 when combining (though the coverage percentage went down as the total coverable lines rose from 1765 to 1899)

But if it's not going to work with the codecov website then it's not super useful; and the line coverage increase is only about 150 lines anyway, in my local testing. If we could get it to combine in codecov, AND we added tests that f.e. covered all the emitters, there would be some benefit. As it is, there doesn't seem to be.

I'll close this up for now, but might reopen a similar PR someday if I work out how to get codecov to see all the coverage data, and can actually cover any significant additional code.

@jepler jepler closed this Aug 28, 2025
@dpgeorge
Copy link
Member

An alternative would be to enable the dynamic compiler option on the unix coverage build, and add some mpy-cross behaviour to unix coverage, then use unix coverage to build all tests using all architectures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants