Skip to content

esp32: Add support for IDF version v5.2. #13775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 8, 2024

Conversation

DvdGiessen
Copy link
Contributor

@DvdGiessen DvdGiessen commented Feb 28, 2024

This MR adds support for building the ESP32 port with ESP-IDF v5.2. I've been using these for a little bit the past few weeks on three different ESP32 variants and it seems to work well so far. These changes are implemented in the second commit of this MR.

The first commit in this MR makes the build work with IDF v5.0.5 and later. Included here since all that needed was a small touch-up on the version ranges in existing version checks.

The new IDF v5.2 deprecated the task cleanup callback we use, so I implemented support for the new option. This also requires a change in the sdkconfig, for the third and last commit in this MR implements an new variable ${SDKCONFIG_IDF_VERSION_SPECIFIC} which is used in all mpconfigboard.cmake files to include an extra sdkconfig file based on the IDF version in use.

EDIT: Added IDF-dependant sdkconfig file as suggested by @dpgeorge.

@dpgeorge
Copy link
Member

See related #11869 and #13364.

@dpgeorge
Copy link
Member

But because I don't think we can conditionally configure these based on a detected IDF version it is not part of the patch.

This might be possible with some cmake code like this:

if(IDF_VERSION_MINOR >= 2)
    set(SDKCONFIG_DEFAULTS
        ${SDKCONFIG_DEFAULTS}
        boards/sdkconfig.idf52
    )
else()
    set(SDKCONFIG_DEFAULTS
        ${SDKCONFIG_DEFAULTS}
        boards/sdkconfig.idf50
    )
endif()

@DvdGiessen DvdGiessen force-pushed the esp32_idf52 branch 2 times, most recently from 91fb390 to 4db0f16 Compare February 29, 2024 13:05
@DvdGiessen
Copy link
Contributor Author

This might be possible with some cmake code

Ah, yes, that does (with some changes) work nicely.

I ended up implementing it slightly different: Instead of adding always the extra configuration file in the CMake code, I opted to make a specific variable ${SDKCONFIG_IDF_VERSION_SPECIFIC} that is used in all the mpconfigboard.cmake files, similar to how we also explicitly specify the sdkconfig.base file in every one of those.

Pushed an extra commit that implements this change.

I think that makes it more explicit and clear for developers which sdkconfig files are being used for their board, without also having to go through the CMakeList.txt file to see if any additional ones are being appended there. Let me know if you'd prefer the solution where we don't modify all board files.

Also updated MR description and added explicit commit that adds compatibility with v5.0.5 and v5.0.6, since that was just a matter of setting the correct version range in an existing check.

@dpgeorge
Copy link
Member

dpgeorge commented Mar 1, 2024

I think that makes it more explicit and clear for developers which sdkconfig files are being used for their board, without also having to go through the CMakeList.txt file to see if any additional ones are being appended there.

I agree, I like this approach of being explicit.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
The new IDF v5.2 deprecated the task cleanup callback we use, so support
for the new option has been implemented in the previous commit.  This also
requires a change in the sdkconfig, via a new variable
${SDKCONFIG_IDF_VERSION_SPECIFIC} which is used in all mpconfigboard.cmake
files to include an extra sdkconfig file based on the IDF version in use.

Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
@dpgeorge dpgeorge merged commit 27279e6 into micropython:master Mar 8, 2024
@kdschlosser
Copy link

kdschlosser commented Mar 28, 2024

I had thought that MIcroPython was no longer going to support more than a single version of the IDF at any given time. It would eliminate all of the IDF version checking that needs to be done.... what happened there?

This also doesn't add support for 5.2 properly. There is a bunch of things that have been deprecated between 5.1 and 5.2 and yet those things are still being used. I2C is one of the very large changes.. There are others as well. This is the bare bones minimum in order to get IDF5.2 to run. Doing this same kind of thing is what caused MicroPython to lag behind with the ESP-IDF versions in the past. In order to keep up with version changes anything that is deprecated needs to be removed/changed to make it fully compliant with the version of the IDF that is being supported.

@andrewleech
Copy link
Contributor

@kdschlosser I'd say this is a case of don't let perfect get in the way of good enough.

While it'd be nice to not need any version checking in the code and stick to just one version, the "latest" version isn't always stable / tested for all platforms so there's a reason to keep it build-able at a tested working version.
But then newer versions are needed for newer chip support.

The reality is I'm sure there's not enough paid staff time to perfectly keep up with all the idf changes at all times. That's where community submissions like this come in.

I for one am glad this was merged as is because it let me progress with the C6 support. If this MR was blocked until every change in IDF was accounted for and thoroughly tested it likely would have never been finished!

That being said, it would be helpful to have a checklist somewhere of all the remaining changed / deprecated APIs that eventually will need updating, though even that requires someone to spend the time going through all the changelogs for each version change.

This is the problem with trying to keep up with a platform as fast moving as the idf where there's a lot of api and platform changes.

@andrewleech
Copy link
Contributor

@DvdGiessen could you check if master still builds with v5.2.1 for you?

It's fine for S3 etc but ESP32_GENERIC build is failing with:

ld: region `iram0_0_seg' overflowed by 84
 bytes
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
HINT: The applications static IRAM usage is larger than th
e available IRAM size.
For more information on how to reduze IRAM usage run 'idf.
py docs -sp api-guides/performance/ram-usage.html#optimizi
ng-iram-usage'
ninja failed with exit code 1, output of the command is in
 the /micropython/ports/esp32/build-ESP32_GENERIC/log/idf_
py_stderr_output_774 and /micropython/ports/esp32/build-ES
P32_GENERIC/log/idf_py_stdout_output_774

@DvdGiessen
Copy link
Contributor Author

DvdGiessen commented Mar 28, 2024

There is a bunch of things that have been deprecated between 5.1 and 5.2 and yet those things are still being used. I2C is one of the very large changes.. There are others as well.

Correct. I did see those changes and looked into how much work it would be to rework the current code to use those new interfaces, and estimated doing so would result in a lot more work (both for me in rewriting a lot of code, and for the maintainers needing to review a much larger PR). The old interfaces, although marked as deprecated, are not removed and work fine (they might not contain the improvements made in the newer interfaces, but it's usually not going to be worse than things already were by being on an older IDF version all together).

So while indeed it might indeed be nice to move to these newer interfaces, I concluded that shouldn't delay or block me from making an PR for the smaller changes needed to run on the lastest IDF (which aside from these big interface changes also includes a lot of other stuff that is useful to have in the meantime).

@DvdGiessen could you check if master still builds with v5.2.1 for you?

Hm, the ESP32_GENERIC does indeed seem to slightly overflow the static IRAM space:

Total sizes:
Used static DRAM:   55704 bytes (  68876 remain, 44.7% used)
      .data size:   23280 bytes
      .bss  size:   32424 bytes
Used static IRAM:  131154 bytes (    -82 remain, 100.1% used) Overflow detected! You can run idf.py size-files for more information.
      .text size:  130127 bytes
   .vectors size:    1027 bytes
Used Flash size : 1561271 bytes
           .text: 1285067 bytes
         .rodata:  275948 bytes
Total image size: 1715705 bytes (.bin may be padded larger)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants