Skip to content

.pth and sitecustomize modules are not processed on interpreter startup #2319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks done
freakboy3742 opened this issue Mar 25, 2025 · 2 comments
Open
3 tasks done
Labels
needs-triage Issue needs triage type: bug Something isn't working

Comments

@freakboy3742
Copy link
Contributor

Checklist

  • I added a descriptive title
  • I searched for other issues and couldn't find a solution or duplication
  • I already searched in Google and didn't find any good information or help

What happened?

Python has a site module, and provides 2 ways for that site module to be configured: sitecustomize.py, and .pth files in the site-packages path.

However, if a Python package provides a sitecustomize.py, or installs a .pth file when it is installed, the logic contained by those mechanisms will not be triggered when the package is declared as being part of a Pyscript environment.

What browsers are you seeing the problem on? (if applicable)

No response

Console info

N/a - it's behavior that *isn't* happening.

Additional Context

I've logged this as a bug on the basis that it's a feature of Python that Pyscript doesn't support; but I can also see the argument that this is a feature (or a cluster of features) that Pyscript doesn't currently support (or, possibly, shouldn't support, depending on how you interpret the underlying cause).

The underlying problem is that Pyodide (and, by extension, Pyscript) has a startup sequence that is a little different to a desktop Python interpreter. On desktop, you create an environment, install a package, then start the interpreter. Site handling is then performed on a complete site-packages folder; so if your PYTHONPATH contains a sitecustomize.py, or there's a .pth file in your site-packages, they will be processed.

However, Pyodide performs site-packages handling when the interpreter is started, and then installs packages into the active environment. Thus, there is no trigger for running sitecustomize or .pth logic.

An immediate workaround is to invoke site.addsitedir("/lib/python3.12/site-packages") in user-space code; that will process .pth files, but results in site-packages being duplicated in the site packages.

I've reported this upstream to pyodide; pyodide/pyodide#5539 was the immediate response. This hook would allow a pyodide app to inject content into the site-packages folder. That would work for a pure pyodide project; but that feature would then need to be exposed in Pyscript.

It's arguable whether it should be exposed directly as a literal feature (i.e., an "add this file/content to site-packages"), or whether Pyscript's dependency initialisation mechanism needs to be modified (which would likely involve using the underlying feature). After all - the original use case that exposed this is installing Python package that has .pth file in the installed payload - on desktop that doesn't require an explicit step to write a .pth file.

The alternate approach would be to say that site handling is out-of-scope for Pyscript (on the basis that Pyscript's execution model is fundamentally different to desktop). However, if this is the approach taken, it should then be documented as an explicit difference/divergence from "expected" Python behavior.

@mhsmith
Copy link

mhsmith commented Mar 25, 2025

that will process .pth files, but results in site-packages being duplicated in the site packages

Do you mean it'll be duplicated in sys.path? Looking at the site.py source code, it appears there's some attempt to avoid this duplication, though I'm not sure if it'll work in this context.

@freakboy3742
Copy link
Contributor Author

freakboy3742 commented Mar 25, 2025

Do you mean it'll be duplicated in sys.path? Looking at the site.py source code, it appears there's some attempt to avoid this duplication, though I'm not sure if it'll work in this context.

🤦 Yes - I meant sys.path. And yes - it looks adding an existing path (like lib/python/site-packages) might not work, as an existing path isn't re-processed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-triage Issue needs triage type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants