Skip to content

.pth and sitecustomize modules are not processed on interpreter startup #2319

Open
@freakboy3742

Description

@freakboy3742

Checklist

  • I added a descriptive title
  • I searched for other issues and couldn't find a solution or duplication
  • I already searched in Google and didn't find any good information or help

What happened?

Python has a site module, and provides 2 ways for that site module to be configured: sitecustomize.py, and .pth files in the site-packages path.

However, if a Python package provides a sitecustomize.py, or installs a .pth file when it is installed, the logic contained by those mechanisms will not be triggered when the package is declared as being part of a Pyscript environment.

What browsers are you seeing the problem on? (if applicable)

No response

Console info

N/a - it's behavior that *isn't* happening.

Additional Context

I've logged this as a bug on the basis that it's a feature of Python that Pyscript doesn't support; but I can also see the argument that this is a feature (or a cluster of features) that Pyscript doesn't currently support (or, possibly, shouldn't support, depending on how you interpret the underlying cause).

The underlying problem is that Pyodide (and, by extension, Pyscript) has a startup sequence that is a little different to a desktop Python interpreter. On desktop, you create an environment, install a package, then start the interpreter. Site handling is then performed on a complete site-packages folder; so if your PYTHONPATH contains a sitecustomize.py, or there's a .pth file in your site-packages, they will be processed.

However, Pyodide performs site-packages handling when the interpreter is started, and then installs packages into the active environment. Thus, there is no trigger for running sitecustomize or .pth logic.

An immediate workaround is to invoke site.addsitedir("/lib/python3.12/site-packages") in user-space code; that will process .pth files, but results in site-packages being duplicated in the site packages.

I've reported this upstream to pyodide; pyodide/pyodide#5539 was the immediate response. This hook would allow a pyodide app to inject content into the site-packages folder. That would work for a pure pyodide project; but that feature would then need to be exposed in Pyscript.

It's arguable whether it should be exposed directly as a literal feature (i.e., an "add this file/content to site-packages"), or whether Pyscript's dependency initialisation mechanism needs to be modified (which would likely involve using the underlying feature). After all - the original use case that exposed this is installing Python package that has .pth file in the installed payload - on desktop that doesn't require an explicit step to write a .pth file.

The alternate approach would be to say that site handling is out-of-scope for Pyscript (on the basis that Pyscript's execution model is fundamentally different to desktop). However, if this is the approach taken, it should then be documented as an explicit difference/divergence from "expected" Python behavior.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions