Skip to content

Ensure correct site packages are visible to rustpython #4924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 23, 2023

Conversation

itsankitkp
Copy link
Contributor

@itsankitkp itsankitkp commented Apr 22, 2023

This work should resolves #4911

Aim

  • 1. Add rustpython's lib to sys.path
  • 2. Fix typo in rustpython's lib name
  • 3. Add test in CI for setting up venv and pip

Note: Instead of having both python and rustpython, I am including only one as venv/lib/rustpython3.11/site-packages, as venv/lib/python3.11/ severs no purpose and create confusion.

@itsankitkp
Copy link
Contributor Author

itsankitkp commented Apr 22, 2023

venv/lib/rustrustpython3.11 folder gets created and populated by pip (while setting up -m ensurepip during venv initialization)

@itsankitkp
Copy link
Contributor Author

Issue probably lies here:

>>>>> import sysconfig
>>>>> sysconfig.get_paths()
{'stdlib': '/usr/local/lib/rustrustpython3.11', 'platstdlib': '/tmp/venv/lib/rustrustpython3.11', 'purelib': '/tmp/venv/lib/rustrustpython3.11/site-packages', 'platlib': '/tmp/venv/lib/rustrustpython3.11/site-packages', 'include': '/usr/local/include/rustrustpython3.11', 'platinclude': '/usr/local/include/rustrustpython3.11', 'scripts': '/tmp/venv/bin', 'data': '/tmp/venv'}
>>>>>

Notice purelib points to venv/lib/rustrustpython3.11/site-packages

This function eventually gets called in system_sites = _get_system_sitepackages() in _internal/build_env.py in pip module

@itsankitkp
Copy link
Contributor Author

itsankitkp commented Apr 22, 2023

Issue probably lies here:

>>>>> import sysconfig
>>>>> sysconfig.get_paths()
{'stdlib': '/usr/local/lib/rustrustpython3.11', 'platstdlib': '/tmp/venv/lib/rustrustpython3.11', 'purelib': '/tmp/venv/lib/rustrustpython3.11/site-packages', 'platlib': '/tmp/venv/lib/rustrustpython3.11/site-packages', 'include': '/usr/local/include/rustrustpython3.11', 'platinclude': '/usr/local/include/rustrustpython3.11', 'scripts': '/tmp/venv/bin', 'data': '/tmp/venv'}
>>>>>

Notice purelib points to venv/lib/rustrustpython3.11/site-packages

This function eventually gets called in system_sites = _get_system_sitepackages() in _internal/build_env.py in pip module

I think I found the issue:
in sysconfig

# XXX RUSTPYTHON: replace python with rustpython in all these paths
for group in _INSTALL_SCHEMES.values():
    for key in group.keys():
        group[key] = group[key].replace("Python", "RustPython").replace("python", "rustpython")

when create venv from rustpython, it already have correct _INSTALL_SCHEMES (with rustpython in it),
once above line gets executed again in venv, rustpython become rust(rustpython)

@itsankitkp itsankitkp marked this pull request as ready for review April 22, 2023 17:19
@youknowone
Copy link
Member

Thank you! Let me try some tricks to minimize sys_config.py diff

@youknowone
Copy link
Member

Do you know how other distributions handle it? It seems we'd better just read lib/python3.11 rather than lib/rustpython3.11 if possible

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much! Let's fix it first and then think about the better or the correct way.

@youknowone youknowone merged commit cf465bd into RustPython:main Apr 23, 2023
@itsankitkp
Copy link
Contributor Author

Thank you so much! Let's fix it first and then think about the better or the correct way.

Makes sense, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add venv PIP test to CI
2 participants