Skip to content

[ESP32] Errno mismatch between esp-idf and mpy #5752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tve opened this issue Mar 11, 2020 · 5 comments
Closed

[ESP32] Errno mismatch between esp-idf and mpy #5752

tve opened this issue Mar 11, 2020 · 5 comments

Comments

@tve
Copy link
Contributor

tve commented Mar 11, 2020

The esp32 port sets MICROPY_USE_INTERNAL_ERRNO which causes the built-in list of error codes in py/mperrno.h to be used. That's mostly OK, but there are discrepancies WRT esp-idf (see newlib's errno.h) and lots of missing error codes as a result. Examples:

  • MP uses 115 for EINPROGRESS while esp-idf uses 119, this particular one is patched up in exception_from_errno in https://github.com/micropython/micropython/blob/master/ports/esp32/modsocket.c#L156, however esp-idf uses 115 for ENETDOWN, which is now essentially undetectable
  • socket operations can return errno 23, which is not defined in MP's list, which means the user has to go on a hunt for where that may be defined to see that it's ENFILE /* Too many open files in system */ (happens, for example, if one doesn't close sockets on error somewhere).
  • there are many other useful socket errors which are not in MP's list

Q: is there a reason the esp32 port sets the insufficient MICROPY_USE_INTERNAL_ERRNO other than expediency at the time?

@dpgeorge
Copy link
Member

is there a reason the esp32 port sets the insufficient MICROPY_USE_INTERNAL_ERRNO other than expediency at the time?

The main reason is for consistency with other (bare-metal) ports. So if you get an OSError with just a numeric value for the errno then that numeric value matches other ports and the value itself can be looked up in py/mperrno.h, and you could write a constant in your Python code, eg ENOTDIR = 20.

Also, some tests check for specific errno values because the symbol does not exist in the standard uerrno module (eg uerrno.ENOTDIR doesn't exist).

however esp-idf uses 115 for ENETDOWN, which is now essentially undetectable

That could be fixed by converting newlib's 115 to uPy's "standard" number. Really there should be a table doing the mapping that can be use everywhere to convert IDF errno to uPy errno. It's just that this wasn't done yet.

socket operations can return errno 23, which is not defined in MP's list

I'm not sure what you mean by "not defined in MP's list". In py/mperrno.h there's a definition of MP_ENFILE with value 23.

which means the user has to go on a hunt for where that may be defined

Even with MICROPY_USE_INTERNAL_ERRNO disabled one will need to track down errno numbers in header files to see what they mean. With MICROPY_USE_INTERNAL_ERRNO enabled it's a bit easier to do this, just look in py/mperrno.h.

@tve
Copy link
Contributor Author

tve commented Mar 12, 2020

I understand your point of view. Let me try to argue for a different point of view:

  • There are a very large number of error codes pretty cleanly defined in ESP-IDF that vastly exceed what's in mperrno.h and that really should not be added to it, I believe. Examples are error codes from mbedtls.
  • esp-idf provides a function esp_err_to_name that provides a human-readable error string for each of the numbers (I believe it covers all the error codes, but I haven't verified). It would be great to use this function instead of the insufficient table in moduerrno.c
  • if MP code uses hard-coded error number constants then that should be fixed and on the esp32 the appropriate errno.h file should be included

socket operations can return errno 23, which is not defined in MP's list

I'm not sure what you mean by "not defined in MP's list". In py/mperrno.h there's a definition of MP_ENFILE with value 23.

You are correct. I got slightly mixed-up, what I meant to say is that it's not defined in moduerrno.c and thus when printing the exception one just sees "23" and has to start looking at sources to figure out what it is.

which means the user has to go on a hunt for where that may be defined

Even with MICROPY_USE_INTERNAL_ERRNO disabled one will need to track down errno numbers in header files to see what they mean. With MICROPY_USE_INTERNAL_ERRNO enabled it's a bit easier to do this, just look in py/mperrno.h.

You mean: look in mperrno.h and if not found look in esp-idf newlib's errno.h and if you want to be really sure grep the esp32's sources just in case two errors get mapped to the same number. ;-)

@dpgeorge
Copy link
Member

Examples are error codes from mbedtls.

Are these additional error codes ever raised as an OSError? For mbedtls (ussl module) I don't think they are.

esp-idf provides a function esp_err_to_name that provides a human-readable error string for each of the numbers (I believe it covers all the error codes, but I haven't verified). It would be great to use this function instead of the insufficient table in moduerrno.c

But the error messages for standard errno's should match other ports, eg:

>>> raise OSError(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 1] EPERM

For extended errno's that are IDF specific, how would the corresponding error message look?

if MP code uses hard-coded error number constants then that should be fixed

As far as I know there are no uses of hard-coded errno's in the C code. Only in the Python tests, eg extmod/vfs_fat_fileio1.py uses 20 instead of ENOTDIR.


In the end it's not a problem to disable MICROPY_USE_INTERNAL_ERRNO and use the IDF provided numbers. User code should not notice any difference (unless it uses hard-coded values...). But the tests should still pass, and error messages remain the same.

@tve
Copy link
Contributor Author

tve commented Mar 12, 2020

I also realized that for native modules it can be awkward if one wants to use EXXX symbols: which .h file do you include and what if you want to test against an esp-idf EXXX and raise an MPY EYYY...

Overall it's clearly not something that prevents getting work done, but clean it is not. Thanks for responding to my questions!

@dpgeorge
Copy link
Member

MICROPY_USE_INTERNAL_ERRNO was disabled for esp32 in 902da05

See this comment for additional reasoning: #6569 (comment)

tannewt pushed a commit to tannewt/circuitpython that referenced this issue Dec 29, 2021
make github-ci install python version "3.x" everywhere we use actions/setup-python (and upgrade to setup-python@v2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants