Skip to content

bpo-34632: Add the importlib.metadata module #9327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions Doc/library/importlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -792,6 +792,96 @@ ABC hierarchy::
itself does not end in ``__init__``.


:mod:`importlib.metadata` -- Metadata
-------------------------------------

.. module:: importlib.metadata
:synopsis: Installed package metadata reading

**Source code:** :source:`Lib/importlib/metadata.py`

--------------

.. versionadded:: 3.8

This module leverages Python's import system to provide access to installed
package metadata, such as the package's version, entry points, and more.

.. note::

This module provides functionality similar to `pkg_resources
<https://setuptools.readthedocs.io/en/latest/pkg_resources.html>`_ `entry
point API
<https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points>`_
and `metadata API
<https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api>`_.

The standalone backport of this module provides more information
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"stand-alone"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want to address this on the next port, as this phrase is local to the CPython port.

on `using importlib.metadata
<https://importlib-metadata.readthedocs.io/en/latest/using.html>`_.

By *installed package* we generally mean a third party package installed into
Python's ``site-packages`` directory via tools such as ``pip``. Specifically,
it means a package with either a discoverable ``dist-info`` or ``egg-info``
directory, and metadata defined by :PEP:`566` or its older specifications. By
default, package metadata can live on the file system or in wheels on
``sys.path``. Through an extension mechanism, the metadata can live almost
anywhere.

Support is provided by special finders on :attr:`sys.meta_path` (see below).
By default, packages installed on the file system and in :PEP:`491` wheel
files are supported.

The following exception is defined.

.. exception:: PackageNotFoundError

Raised when the named package's metadata could not be found.

The following functions are available.

.. function:: entry_points(package):

Returns the `entry points
<https://packaging.python.org/specifications/entry-points/>`_ for the
given package, as a :class:`configparser.ConfigParser` instance. The
sections in this ``ConfigParser`` exactly match the sections in the
distribution package's `entry_points.txt
<https://packaging.python.org/specifications/entry-points/#file-format>`
file. For example, you can get the console script entry points by
extracting the the ``console_scripts`` section.

.. function:: resolve(entry_point):

This is a convenience function for turning an entry point specification
(e.g. ``foo.bar:main``) into the ``foo.bar.main`` object. Raises
:exc:`ValueError` if *entry_point* is not of the proper format.

.. function:: version(package):

Returns the distribution package's version as a string. This is a
convenience function for returning the ``Version`` metadata.

In order to extend the types of distribution packages for which metadata can
be returned, you can implement a special finder that implements a method
called ``find_distribution()``. This function takes a single string argument
which names the distribution package to find, and it returns an instance of as
described below. ``find_distribution()`` can use any algorithm it wants to
find the distribution package's ``dist-info`` or ``egg-info`` directory.

You can add this method to an existing finder, or you can create a simple
finder that onky finds distribution package metadata. In the latter case, you
must add a ``@staticmethod`` called ``find_spec()`` that accepts any arguments
and returns ``None``. Your finder must be added to :attr:`sys.meta_path`.

``find_distribution()`` should return an instance of a class derived from the
abstract base class ``importlib.metadata.Distribution``. The only
additional method you need to implement on your subclass is
``load_metadata()``, which takes the name of the distribution package and
returns the contents of the package's ``dist-info/METADATA`` file as a
``str``, or ``None`` if the file does not exist.


:mod:`importlib.resources` -- Resources
---------------------------------------

Expand Down
1 change: 1 addition & 0 deletions Doc/tools/susp-ignored.csv
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ library/http.client,,:port,host:port
library/http.cookies,,`,!#$%&'*+-.^_`|~:
library/imaplib,,:MM,"""DD-Mmm-YYYY HH:MM:SS"
library/imaplib,,:SS,"""DD-Mmm-YYYY HH:MM:SS"
library/importlib,,:main,(e.g. ``foo.bar:main``) into the ``foo.bar.main`` object. Raises
library/inspect,,:int,">>> def foo(a, *, b:int, **kwargs):"
library/inspect,,:int,"'(a, *, b:int, **kwargs)'"
library/inspect,,:int,'b:int'
Expand Down
271 changes: 271 additions & 0 deletions Lib/importlib/metadata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,271 @@
import os
import re
import abc
import sys
import email
import itertools
import contextlib

from configparser import ConfigParser
from importlib import import_module
from importlib.resources import Package
from pathlib import Path
from types import ModuleType
from typing import Callable
from zipfile import ZipFile


__all__ = [
'Distribution',
'PackageNotFoundError',
'distribution',
'entry_points',
'resolve',
'version',
]


class PackageNotFoundError(ModuleNotFoundError):
"""The package's metadata was not found."""


class Distribution:
"""A Python distribution package."""

@abc.abstractmethod
def load_metadata(self, name):
"""Attempt to load metadata given by the name.

:param name: The name of the distribution package.
:return: The metadata string if found, otherwise None.
"""

@classmethod
def from_name(cls, name):
"""Return the Distribution for the given package name.

:param name: The name of the distribution package to search for.
:return: The Distribution instance (or subclass thereof) for the named
package, if found.
:raises PackageNotFoundError: When the named package's distribution
metadata cannot be found.
"""
for resolver in cls._discover_resolvers():
resolved = resolver(name)
if resolved is not None:
return resolved
else:
raise PackageNotFoundError(name)

@staticmethod
def _discover_resolvers():
"""Search the meta_path for resolvers."""
declared = (
getattr(finder, 'find_distribution', None)
for finder in sys.meta_path
)
return filter(None, declared)

@classmethod
def from_module(cls, module):
"""Discover the Distribution package for a module."""
return cls.from_name(cls.name_for_module(module))

@classmethod
def from_named_module(cls, mod_name):
return cls.from_module(import_module(mod_name))

@staticmethod
def name_for_module(module):
"""Given an imported module, infer the distribution package name."""
return getattr(module, '__dist_name__', module.__name__)

@property
def metadata(self):
"""Return the parsed metadata for this Distribution.

The returned object will have keys that name the various bits of
metadata. See PEP 566 for details.
"""
return email.message_from_string(
self.load_metadata('METADATA') or self.load_metadata('PKG-INFO')
)

@property
def version(self):
"""Return the 'Version' metadata for the distribution package."""
return self.metadata['Version']


def distribution(package: Package) -> Distribution:
"""Get the ``Distribution`` instance for the given package.

:param package: The module object for the package or the name of the
package as a string.
:return: A ``Distribution`` instance (or subclass thereof).
"""
if isinstance(package, ModuleType):
return Distribution.from_module(package)
else:
return Distribution.from_name(package)


def entry_points(package: Package) -> ConfigParser:
"""Return the entry points for the named distribution package.

:param name: The name of the distribution package to query.
:return: A ConfigParser instance where the sections and keys are taken
from the entry_points.txt ini-style contents.
"""
as_string = distribution(package).load_metadata('entry_points.txt')
# 2018-09-10(barry): Should we provide any options here, or let the caller
# send options to the underlying ConfigParser? For now, YAGNI.
config = ConfigParser()
config.read_string(as_string)
return config


def resolve(entry_point: str) -> Callable:
"""Resolve an entry point string into the named callable.

:param entry_point: An entry point string of the form
`path.to.module:callable`.
:return: The actual callable object `path.to.module.callable`
:raises ValueError: When `entry_point` doesn't have the proper format.
"""
path, colon, name = entry_point.rpartition(':')
if colon != ':':
raise ValueError('Not an entry point: {}'.format(entry_point))
module = import_module(path)
return getattr(module, name)


def version(package: Package) -> str:
"""Get the version string for the named package.

:param package: The module object for the package or the name of the
package as a string.
:return: The version string for the package as defined in the package's
"Version" metadata key.
"""
return distribution(package).version


class MetadataPathFinder:
"""A degenerate finder for distribution packages on the file system.

This finder supplies only a find_distribution() method for versions
of Python that do not have a PathFinder find_distribution().
"""
@staticmethod
def find_spec(*args, **kwargs):
return None

@classmethod
def find_distribution(cls, name):
paths = cls._search_paths(name)
dists = map(PathDistribution, paths)
return next(dists, None)

@classmethod
def _search_paths(cls, name):
"""
Find metadata directories in sys.path heuristically.
"""
return itertools.chain.from_iterable(
cls._search_path(path, name)
for path in map(Path, sys.path)
)

@classmethod
def _search_path(cls, root, name):
if not root.is_dir():
return ()
return (
item
for item in root.iterdir()
if item.is_dir()
and str(item.name).startswith(name)
and re.match(rf'{name}(-.*)?\.(dist|egg)-info', str(item.name))
)


class PathDistribution(Distribution):
def __init__(self, path):
"""Construct a distribution from a path to the metadata directory."""
self._path = path

def load_metadata(self, name):
"""Attempt to load metadata given by the name.

:param name: The name of the distribution package.
:return: The metadata string if found, otherwise None.
"""
filename = os.path.join(self._path, name)
with contextlib.suppress(FileNotFoundError):
with open(filename, encoding='utf-8') as fp:
return fp.read()
return None


class WheelMetadataFinder:
"""A degenerate finder for distribution packages in wheels.

This finder supplies only a find_distribution() method for versions
of Python that do not have a PathFinder find_distribution().
"""
@staticmethod
def find_spec(*args, **kwargs):
return None

@classmethod
def find_distribution(cls, name):
try:
module = import_module(name)
except ImportError:
return None
archive = getattr(module.__loader__, 'archive', None)
if archive is None:
return None
try:
name, version = Path(archive).name.split('-')[0:2]
except ValueError:
return None
dist_info = '{}-{}.dist-info'.format(name, version)
with ZipFile(archive) as zf:
# Since we're opening the zip file anyway to see if there's a
# METADATA file in the .dist-info directory, we might as well
# read it and cache it here.
zi = zf.getinfo('{}/{}'.format(dist_info, 'METADATA'))
metadata = zf.read(zi).decode('utf-8')
return WheelDistribution(archive, dist_info, metadata)


class WheelDistribution(Distribution):
def __init__(self, archive, dist_info, metadata):
self._archive = archive
self._dist_info = dist_info
self._metadata = metadata

def load_metadata(self, name):
"""Attempt to load metadata given by the name.

:param name: The name of the distribution package.
:return: The metadata string if found, otherwise None.
"""
if name == 'METADATA':
return self._metadata
with ZipFile(self._archive) as zf:
with contextlib.suppress(KeyError):
as_bytes = zf.read('{}/{}'.format(self._dist_info, name))
return as_bytes.decode('utf-8')
return None


def _install(): # pragma: nocover
"""Install the appropriate sys.meta_path finder for the Python version."""
sys.meta_path.append(MetadataPathFinder)
sys.meta_path.append(WheelMetadataFinder)


_install()
Empty file.
Binary file not shown.
Loading