Skip to content

Implement import by using importlib #958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
8 of 11 tasks
BenLewis-Seequent opened this issue May 11, 2019 · 8 comments
Closed
8 of 11 tasks

Implement import by using importlib #958

BenLewis-Seequent opened this issue May 11, 2019 · 8 comments
Labels
C-compat A discrepancy between RustPython and CPython

Comments

@BenLewis-Seequent
Copy link

BenLewis-Seequent commented May 11, 2019 •
edited by coolreader18
Loading

The standard library for Python contains importlib, which contains a pure* Python implementation of the import machinery. This is lucky as we then don't have to implement it all in Rust. But we have to bootstrap it and implement all of the functionality it uses from builtin modules.

This is blocked by the following:

  • compile should accept bytes as well as strings
  • _os.fspath - https://docs.python.org/3/library/os.html#os.fspath
  • sys.implementation.cache_tag - This just needs to exist we don't support caching bytecode at the moment
  • sys.pycache_prefix - same as above
  • sys.flags
  • str.endswith needs to support tuple argument
  • _io.FileIO __enter__ and __exit__ methods
  • sys.platform
  • os.getcwd
  • os.stat st_mtime attribute
  • _warnings module
  • possibly more

* Uses built-in module _imp to actually create module objects.
@palaviv palaviv mentioned this issue May 11, 2019
Merged
@BenLewis-Seequent BenLewis-Seequent mentioned this issue May 11, 2019
Merged
This was referenced May 11, 2019
Merged
Merged
@BenLewis-Seequent BenLewis-Seequent mentioned this issue May 12, 2019
Merged
@youknowone youknowone mentioned this issue May 13, 2019
Open
@Jongy Jongy mentioned this issue May 14, 2019
Closed
This was referenced May 18, 2019
Merged
Merged
This was referenced May 24, 2019
Merged
Merged
Merged
@palaviv
Copy link
Contributor

palaviv commented Jun 5, 2019

An initial implementation can be found in #1018.

@palaviv
Copy link
Contributor

palaviv commented Jun 7, 2019

Another thing that would help this is supporting {!r} in string format.

@palaviv palaviv mentioned this issue Jun 7, 2019
Merged
@coolreader18 coolreader18 added the C-compat A discrepancy between RustPython and CPython label Jun 9, 2019
@palaviv
Copy link
Contributor

palaviv commented Jun 18, 2019

This can be closed not that #1018 was merged

@apatrushev
Copy link

apatrushev commented Jun 18, 2019 •
edited
Loading

UPD: Related to #55

level and globals argument is not provided and relative import is not working.
As example import unittest is broken at:

from .result import TestResult

Example:

module/__init__.py:

from .second import value

module/second.py:

value = 1

Expected result

python -c 'import module; print(module.value)'
1

Current result

cargo run -- -c 'import module; print(module.value)'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "None", line 1093, in __import__
  File "None", line 1014, in _gcd_import
  File "None", line 991, in _find_and_load
  File "None", line 975, in _find_and_load_unlocked
  File "None", line 686, in _load_unlocked
  File "None", line 677, in _load_unlocked
  File "None", line 671, in _load_unlocked
  File "None", line 778, in exec_module
  File "None", line 219, in _call_with_frames_removed
  File ".../RustPython/module/__init__.py", line 1, in <module>
    from .second import value
  File "None", line 1093, in __import__
  File "None", line 1014, in _gcd_import
  File "None", line 991, in _find_and_load
  File "None", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named '.second'

@apatrushev
Copy link

Additional investigation - CPython does level magic in compile:

python -c "import dis; dis.dis(compile('from .second import value', '', 'exec'))"
  1           0 LOAD_CONST               0 (1)
              2 LOAD_CONST               1 (('value',))
              4 IMPORT_NAME              0 (second)
              6 IMPORT_FROM              1 (value)
              8 STORE_NAME               1 (value)
[...skipped...]

(note second dot)

python -c "import dis; dis.dis(compile('from ..second import value', '', 'exec'))"
  1           0 LOAD_CONST               0 (2)
              2 LOAD_CONST               1 (('value',))
              4 IMPORT_NAME              0 (second)
              6 IMPORT_FROM              1 (value)
              8 STORE_NAME               1 (value)
[...skipped...]

@palaviv
Copy link
Contributor

palaviv commented Jun 21, 2019

@apatrushev please look at the implementation of the relative import at #1050.

@apatrushev
Copy link

@apatrushev please look at the implementation of the relative import at #1050.

I took a short look on the code in PR. Unfortunately I have no strong opinion on where to place this code in compile or in vm execution because I am not confident neither in CPython code nor in RustPython. And I have no idea why CPython doing this in compile time - I just put my investigations here for the history to save the time later.

Besides this disclaimer your code LGTM (and import unittest moved forward!).

@palaviv
Copy link
Contributor

palaviv commented Jun 30, 2019

@Skinny121, @apatrushev can we close this issue?

@palaviv palaviv closed this as completed Jul 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-compat A discrepancy between RustPython and CPython
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@apatrushev @palaviv @BenLewis-Seequent @coolreader18