Skip to content

3.12+: compile() and some wrappers of it don't accept non-bytes buffers for filename argument at runtime #13881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Sachaa-Thanasius opened this issue Apr 24, 2025 · 3 comments
Labels
stubs: false negative Type checkers do not report an error, but should

Comments

@Sachaa-Thanasius
Copy link
Contributor

Sachaa-Thanasius commented Apr 24, 2025

The underlying C API that compile() uses to convert/validate its filename argument, PyUnicode_FSConverter, doesn't accept non-bytes buffers anymore (see python/cpython#98393), but typeshed still has that parameter typed with (collections.abc|typing_extensions).Buffer instead of bytes. Having a 3.12+ version if branch for the updated function definition would make sense, imo.

Min. Repro. Example

(.venv) thanos@DESKTOP-DERJ7DF:~$ python3.11 --version
Python 3.11.11
(.venv) thanos@DESKTOP-DERJ7DF:~$ python3.11 -c "compile('a=1', bytearray(b'<bytes>'), 'exec')"
<string>:1: DeprecationWarning: path should be string, bytes, or os.PathLike, not bytearray
(.venv) thanos@DESKTOP-DERJ7DF:~$ python3.12 --version
Python 3.12.9
(.venv) thanos@DESKTOP-DERJ7DF:~$ python3.12 -c "compile('a=1', bytearray(b'<bytes>'), 'exec')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: expected str, bytes or os.PathLike object, not bytearray

At the moment, I can think of a few stdlib functions that pass a filename argument through to compile() and thus also need to be updated:

  • ast.parse()
  • importlib.machinery.SourceFileLoader.source_to_code() (maybe also the corresponding methods in its base classes?)

Not sure what else is affected by this deprecation-turned-error, but there might be more.

@Sachaa-Thanasius Sachaa-Thanasius changed the title 3.12+: compile() and some wrappers of it don't accept non-bytes buffers as filepath argument at runtime 3.12+: compile() and some wrappers of it don't accept non-bytes buffers for filename argument at runtime Apr 24, 2025
@srittau srittau added the stubs: false negative Type checkers do not report an error, but should label Apr 25, 2025
@vidhyavijayan3
Copy link
Contributor

vidhyavijayan3 commented Apr 28, 2025

Hi @Sachaa-Thanasius

I’d like to work on this. Can I take it?

@Sachaa-Thanasius
Copy link
Contributor Author

I'm not a maintainer here, but for what my word is worth, have at it. :D

@vidhyavijayan3
Copy link
Contributor

Thank you! @Sachaa-Thanasius

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stubs: false negative Type checkers do not report an error, but should
Projects
None yet
Development

No branches or pull requests

3 participants