Skip to content

bpo-42131: Add PEP 451-related methods to zipimport #23187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 13, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 43 additions & 3 deletions Doc/library/zipimport.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,9 @@ doesn't contain :file:`.pyc` files, importing may be rather slow.
follows the specification in :pep:`273`, but uses an implementation written by Just
van Rossum that uses the import hooks described in :pep:`302`.

:pep:`302` - New Import Hooks
The PEP to add the import hooks that help this module work.
:mod:`importlib` - The implementation of the import machinery
Package providing the relevant protocols for all importers to
implement.


This module defines an exception:
Expand Down Expand Up @@ -73,14 +74,49 @@ zipimporter Objects
:exc:`ZipImportError` is raised if *archivepath* doesn't point to a valid ZIP
archive.

.. method:: find_module(fullname[, path])
.. method:: create_module(spec)

Implementation of :meth:`importlib.abc.Loader.create_module` that returns
:const:`None` to explicitly request the default semantics.

.. versionadded:: 3.10


.. method:: exec_module(module)

Implementation of :meth:`importlib.abc.Loader.exec_module`.

.. versionadded:: 3.10


.. method:: find_loader(fullname, path=None)

An implementation of :meth:`importlib.abc.PathEntryFinder.find_loader`.

.. deprecated:: 3.10

Use :meth:`find_spec` instead.


.. method:: find_module(fullname, path=None)

Search for a module specified by *fullname*. *fullname* must be the fully
qualified (dotted) module name. It returns the zipimporter instance itself
if the module was found, or :const:`None` if it wasn't. The optional
*path* argument is ignored---it's there for compatibility with the
importer protocol.

.. deprecated:: 3.10

Use :meth:`find_spec` instead.


.. method:: find_spec(fullname, target=None)

An implementation of :meth:`importlib.abc.PathEntryFinder.find_spec`.

.. versionadded:: 3.10


.. method:: get_code(fullname)

Expand Down Expand Up @@ -126,6 +162,10 @@ zipimporter Objects
qualified (dotted) module name. It returns the imported module, or raises
:exc:`ZipImportError` if it wasn't found.

.. deprecated:: 3.10

Use :meth:`exec_module` instead.


.. attribute:: archive

Expand Down
7 changes: 7 additions & 0 deletions Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,13 @@ Add a :class:`~xml.sax.handler.LexicalHandler` class to the
:mod:`xml.sax.handler` module.
(Contributed by Jonathan Gossage and Zackery Spytz in :issue:`35018`.)

zipimport
---------
Add methods related to :pep:`451`: :meth:`~zipimport.zipimporter.find_spec`,
:meth:`zipimport.zipimporter.create_module`, and
:meth:`zipimport.zipimporter.exec_module`.
(Contributed by Brett Cannon in :issue:`42131`.


Optimizations
=============
Expand Down
71 changes: 53 additions & 18 deletions Lib/test/test_zipimport.py
Original file line number Diff line number Diff line change
Expand Up @@ -450,8 +450,9 @@ def testZipImporterMethods(self):

zi = zipimport.zipimporter(TEMP_ZIP)
self.assertEqual(zi.archive, TEMP_ZIP)
self.assertEqual(zi.is_package(TESTPACK), True)
self.assertTrue(zi.is_package(TESTPACK))

# PEP 302
find_mod = zi.find_module('spam')
self.assertIsNotNone(find_mod)
self.assertIsInstance(find_mod, zipimport.zipimporter)
Expand All @@ -462,25 +463,39 @@ def testZipImporterMethods(self):
mod = zi.load_module(TESTPACK)
self.assertEqual(zi.get_filename(TESTPACK), mod.__file__)

# PEP 451
spec = zi.find_spec('spam')
self.assertIsNotNone(spec)
self.assertIsInstance(spec.loader, zipimport.zipimporter)
self.assertFalse(spec.loader.is_package('spam'))
exec_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(exec_mod)
self.assertEqual(spec.loader.get_filename('spam'), exec_mod.__file__)

spec = zi.find_spec(TESTPACK)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
self.assertEqual(zi.get_filename(TESTPACK), mod.__file__)

existing_pack_path = importlib.import_module(TESTPACK).__path__[0]
expected_path_path = os.path.join(TEMP_ZIP, TESTPACK)
self.assertEqual(existing_pack_path, expected_path_path)

self.assertEqual(zi.is_package(packdir + '__init__'), False)
self.assertEqual(zi.is_package(packdir + TESTPACK2), True)
self.assertEqual(zi.is_package(packdir2 + TESTMOD), False)
self.assertFalse(zi.is_package(packdir + '__init__'))
self.assertTrue(zi.is_package(packdir + TESTPACK2))
self.assertFalse(zi.is_package(packdir2 + TESTMOD))

mod_path = packdir2 + TESTMOD
mod_name = module_path_to_dotted_name(mod_path)
mod = importlib.import_module(mod_name)
self.assertTrue(mod_name in sys.modules)
self.assertEqual(zi.get_source(TESTPACK), None)
self.assertEqual(zi.get_source(mod_path), None)
self.assertIsNone(zi.get_source(TESTPACK))
self.assertIsNone(zi.get_source(mod_path))
self.assertEqual(zi.get_filename(mod_path), mod.__file__)
# To pass in the module name instead of the path, we must use the
# right importer
loader = mod.__loader__
self.assertEqual(loader.get_source(mod_name), None)
loader = mod.__spec__.loader
self.assertIsNone(loader.get_source(mod_name))
self.assertEqual(loader.get_filename(mod_name), mod.__file__)

# test prefix and archivepath members
Expand All @@ -505,17 +520,22 @@ def testZipImporterMethodsInSubDirectory(self):
zi = zipimport.zipimporter(TEMP_ZIP + os.sep + packdir)
self.assertEqual(zi.archive, TEMP_ZIP)
self.assertEqual(zi.prefix, packdir)
self.assertEqual(zi.is_package(TESTPACK2), True)
self.assertTrue(zi.is_package(TESTPACK2))
# PEP 302
mod = zi.load_module(TESTPACK2)
self.assertEqual(zi.get_filename(TESTPACK2), mod.__file__)
# PEP 451
spec = zi.find_spec(TESTPACK2)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
self.assertEqual(spec.loader.get_filename(TESTPACK2), mod.__file__)

self.assertEqual(
zi.is_package(TESTPACK2 + os.sep + '__init__'), False)
self.assertEqual(
zi.is_package(TESTPACK2 + os.sep + TESTMOD), False)
self.assertFalse(zi.is_package(TESTPACK2 + os.sep + '__init__'))
self.assertFalse(zi.is_package(TESTPACK2 + os.sep + TESTMOD))

pkg_path = TEMP_ZIP + os.sep + packdir + TESTPACK2
zi2 = zipimport.zipimporter(pkg_path)
# PEP 302
find_mod_dotted = zi2.find_module(TESTMOD)
self.assertIsNotNone(find_mod_dotted)
self.assertIsInstance(find_mod_dotted, zipimport.zipimporter)
Expand All @@ -524,17 +544,27 @@ def testZipImporterMethodsInSubDirectory(self):
self.assertEqual(
find_mod_dotted.get_filename(TESTMOD), load_mod.__file__)

# PEP 451
spec = zi2.find_spec(TESTMOD)
self.assertIsNotNone(spec)
self.assertIsInstance(spec.loader, zipimport.zipimporter)
self.assertFalse(spec.loader.is_package(TESTMOD))
load_mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(load_mod)
self.assertEqual(
spec.loader.get_filename(TESTMOD), load_mod.__file__)

mod_path = TESTPACK2 + os.sep + TESTMOD
mod_name = module_path_to_dotted_name(mod_path)
mod = importlib.import_module(mod_name)
self.assertTrue(mod_name in sys.modules)
self.assertEqual(zi.get_source(TESTPACK2), None)
self.assertEqual(zi.get_source(mod_path), None)
self.assertIsNone(zi.get_source(TESTPACK2))
self.assertIsNone(zi.get_source(mod_path))
self.assertEqual(zi.get_filename(mod_path), mod.__file__)
# To pass in the module name instead of the path, we must use the
# right importer.
loader = mod.__loader__
self.assertEqual(loader.get_source(mod_name), None)
self.assertIsNone(loader.get_source(mod_name))
self.assertEqual(loader.get_filename(mod_name), mod.__file__)

def testGetData(self):
Expand Down Expand Up @@ -655,7 +685,9 @@ def testUnencodable(self):
zinfo = ZipInfo(TESTMOD + ".py", time.localtime(NOW))
zinfo.compress_type = self.compression
z.writestr(zinfo, test_src)
zipimport.zipimporter(filename).load_module(TESTMOD)
spec = zipimport.zipimporter(filename).find_spec(TESTMOD)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)

def testBytesPath(self):
filename = os_helper.TESTFN + ".zip"
Expand Down Expand Up @@ -747,14 +779,17 @@ def _testBogusZipFile(self):

try:
self.assertRaises(TypeError, z.find_module, None)
self.assertRaises(TypeError, z.find_spec, None)
self.assertRaises(TypeError, z.exec_module, None)
self.assertRaises(TypeError, z.load_module, None)
self.assertRaises(TypeError, z.is_package, None)
self.assertRaises(TypeError, z.get_code, None)
self.assertRaises(TypeError, z.get_data, None)
self.assertRaises(TypeError, z.get_source, None)

error = zipimport.ZipImportError
self.assertEqual(z.find_module('abc'), None)
self.assertIsNone(z.find_module('abc'))
self.assertIsNone(z.find_spec('abc'))

self.assertRaises(error, z.load_module, 'abc')
self.assertRaises(error, z.get_code, 'abc')
Expand Down
34 changes: 33 additions & 1 deletion Lib/zipimport.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ class ZipImportError(ImportError):
STRING_END_ARCHIVE = b'PK\x05\x06'
MAX_COMMENT_LEN = (1 << 16) - 1

class zipimporter:
class zipimporter(_bootstrap_external._LoaderBasics):
"""zipimporter(archivepath) -> zipimporter object

Create a new zipimporter instance. 'archivepath' must be a path to
Expand Down Expand Up @@ -115,6 +115,8 @@ def find_loader(self, fullname, path=None):
full path name if it's possibly a portion of a namespace package,
or None otherwise. The optional 'path' argument is ignored -- it's
there for compatibility with the importer protocol.

Deprecated since Python 3.10. Use find_spec() instead.
"""
mi = _get_module_info(self, fullname)
if mi is not None:
Expand Down Expand Up @@ -146,9 +148,37 @@ def find_module(self, fullname, path=None):
instance itself if the module was found, or None if it wasn't.
The optional 'path' argument is ignored -- it's there for compatibility
with the importer protocol.

Deprecated since Python 3.10. Use find_spec() instead.
"""
return self.find_loader(fullname, path)[0]

def find_spec(self, fullname, target=None):
"""Create a ModuleSpec for the specified module.

Returns None if the module cannot be found.
"""
module_info = _get_module_info(self, fullname)
if module_info is not None:
return _bootstrap.spec_from_loader(fullname, self, is_package=module_info)
else:
# Not a module or regular package. See if this is a directory, and
# therefore possibly a portion of a namespace package.

# We're only interested in the last path component of fullname
# earlier components are recorded in self.prefix.
modpath = _get_module_path(self, fullname)
if _is_dir(self, modpath):
# This is possibly a portion of a namespace
# package. Return the string representing its path,
# without a trailing separator.
path = f'{self.archive}{path_sep}{modpath}'
spec = _bootstrap.ModuleSpec(name=fullname, loader=None,
is_package=True)
spec.submodule_search_locations.append(path)
return spec
else:
return None

def get_code(self, fullname):
"""get_code(fullname) -> code object.
Expand Down Expand Up @@ -237,6 +267,8 @@ def load_module(self, fullname):
Load the module specified by 'fullname'. 'fullname' must be the
fully qualified (dotted) module name. It returns the imported
module, or raises ZipImportError if it wasn't found.

Deprecated since Python 3.10. use exec_module() instead.
"""
code, ispackage, modpath = _get_module_code(self, fullname)
mod = sys.modules.get(fullname)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Implement PEP 451/spec methods on zipimport.zipimporter: find_spec(),
create_module(), and exec_module().

This also allows for the documented deprecation of find_loader(),
find_module(), and load_module().
Loading