Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.38%. Comparing base (b5fcb33) to head (6a61356).

@@           Coverage Diff           @@
##           master   #17960   +/-   ##
=======================================
  Coverage   98.38%   98.38%           
=======================================
  Files         171      171           
  Lines       22296    22296           
=======================================
  Hits        21937    21937           
  Misses        359      359           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@projectgus I built 6a61356, and will test when I have access to the boards tomorrow.

FWIW: my steps were

• checking out 6a61356
• cd ports/esp32
• make submodules
• make BOARD=...

If there is more to it, please kindly let me know, as I am not yet very familiar with the build process.

However, this still doesn't provide any tracking of when the ESP-IDF version has changed from whatever the current recommended version is... [...] Paying more attention to exact component versions means from now that we'll need to also pay attention to updating components (and the lock file) when it's relevant to do so.

Keep in mind that it's not an uncommon occurrence for Espressif to yank published component versions. That can make pro and con arguments both for keeping track of versions manually and for letting ESP-IDF resolve versions on its own so I won't weigh on this :)

Keep in mind that it's not an uncommon occurrence for Espressif to yank published component versions. That can make pro and con arguments both for keeping track of versions manually and for letting ESP-IDF resolve versions on its own

Although I agree there's pros and cons, I still think explicitly tracking dependencies is a win for this kind of scenario. We don't currently have any way to tell if someone is building with a local yanked component when they submit a bug report, but with this change they'd have to have explicitly chosen to do that.

Also, if we're building with a yanked component and this PR then CI should fail and we get the hint to go and update it.

The more complex case, as I see it, is that we won't automatically build MicroPython with bugfix releases of components, without manually running idf.py -B [...] update-components. In this aspect I agree it's a mixed bag, things won't quietly break but they won't quietly fix themselves either...

so I won't weigh on this :)

Overall I am very interested in your thoughts on this approach, both for this specific TinyUSB problem and generally - so feel free to weigh in if you think it's a step in the wrong direction.

If there is more to it, please kindly let me know, as I am not yet very familiar with the build process.

Thanks @gohai, those are the steps! To flash in the same step you can do make deploy BOARD=... PORT=... (more info in the README). Look forward to hearing if it changes anything (or not).

Although the lock files add complexity (at least, in the sense of adding more files) it sounds like it's worth it to keep proper track of which component versions are in use.

@projectgus This works fine on my USB-OTG-connected ESP32-S3-Zero. (This one had issues copying files earlier.)

However, I am still seeing reproducible repl hangs on the ESP32S3-S3-DEV-KIT-NXR8. See e.g. below (just starting repl made it hang):

@gohai So you get this immediately from simply running mpremote?

@projectgus I'd say I don't have a preference on how the tracking is performed, as long as progress is made :)

I think it all boils down on how much of a maintenance burden it can be to track versions manually. From my past experiences I can sum it all up as this (virtually none of these can be MicroPython's concern, I'm dumping them here in the hope they may help make a more informed decision): how often dependencies need to be updated, how much time it takes to validate those changes, and what's the turnaround time to get an emergency fix from Espressif into MicroPython (eg. potential security implications, that's yet another Pandora's box that should better stay shut) or how quickly a yanked component can be gotten rid of in the published repo lockfiles.

For emergency changes, if people in charge for releasing the Espressif components you rely on are based out of Shanghai, then you folks being a couple hours ahead of them is not a big deal, otherwise you may want to keep that in mind too.

Finally, I wonder if this approach is compatible with https://github.com/agatti/micropython-idf-component. Granted, it's up to me to work around your choices in this case if the end result is not compatible so I'm just talking out loud here.

Incidentally, @dpgeorge if you ever plan to release MicroPython as an ESP-IDF component and put it up into Espressif's registry, feel free to take as much code/ideas/whatnot from that repo. I've licensed that as MIT in order to make the process easier on your end if an official component is an option you want to consider.

However, I am still seeing reproducible repl hangs on the ESP32S3-S3-DEV-KIT-NXR8. See e.g. below (just starting repl made it hang):

Just to be extra sure, does that board with the official 1.26 release firmware and with a firmware with updated IDF components work if you connect to it from Linux, Windows, or a virtualised Linux (so the full USB device is passed through)?

@gohai So you get this immediately from simply running mpremote?

@projectgus It behaves identical to what I saw on 1.26 with that board... it sometimes hangs at startup, but if it does, running help() once or twice will definitely freeze it.