Skip to content

Commit 9783b46

Browse files
authored
Reorder sections in running_mypy (#13738)
See #13681 "Mapping file paths to modules" belongs next to "Specifying code to be checked". "How imports are found" belongs next to "How mypy handles imports". The "other advice" section was mostly redundant.
1 parent da43c91 commit 9783b46

File tree

1 file changed

+136
-150
lines changed

1 file changed

+136
-150
lines changed

docs/source/running_mypy.rst

Lines changed: 136 additions & 150 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,82 @@ flags, the recommended approach is to use a
103103
:ref:`configuration file <config-file>` instead.
104104

105105

106+
.. _mapping-paths-to-modules:
107+
108+
Mapping file paths to modules
109+
*****************************
110+
111+
One of the main ways you can tell mypy what to type check
112+
is by providing mypy a list of paths. For example::
113+
114+
$ mypy file_1.py foo/file_2.py file_3.pyi some/directory
115+
116+
This section describes how exactly mypy maps the provided paths
117+
to modules to type check.
118+
119+
- Mypy will check all paths provided that correspond to files.
120+
121+
- Mypy will recursively discover and check all files ending in ``.py`` or
122+
``.pyi`` in directory paths provided, after accounting for
123+
:option:`--exclude <mypy --exclude>`.
124+
125+
- For each file to be checked, mypy will attempt to associate the file (e.g.
126+
``project/foo/bar/baz.py``) with a fully qualified module name (e.g.
127+
``foo.bar.baz``). The directory the package is in (``project``) is then
128+
added to mypy's module search paths.
129+
130+
How mypy determines fully qualified module names depends on if the options
131+
:option:`--no-namespace-packages <mypy --no-namespace-packages>` and
132+
:option:`--explicit-package-bases <mypy --explicit-package-bases>` are set.
133+
134+
1. If :option:`--no-namespace-packages <mypy --no-namespace-packages>` is set,
135+
mypy will rely solely upon the presence of ``__init__.py[i]`` files to
136+
determine the fully qualified module name. That is, mypy will crawl up the
137+
directory tree for as long as it continues to find ``__init__.py`` (or
138+
``__init__.pyi``) files.
139+
140+
For example, if your directory tree consists of ``pkg/subpkg/mod.py``, mypy
141+
would require ``pkg/__init__.py`` and ``pkg/subpkg/__init__.py`` to exist in
142+
order correctly associate ``mod.py`` with ``pkg.subpkg.mod``
143+
144+
2. The default case. If :option:`--namespace-packages <mypy
145+
--no-namespace-packages>` is on, but :option:`--explicit-package-bases <mypy
146+
--explicit-package-bases>` is off, mypy will allow for the possibility that
147+
directories without ``__init__.py[i]`` are packages. Specifically, mypy will
148+
look at all parent directories of the file and use the location of the
149+
highest ``__init__.py[i]`` in the directory tree to determine the top-level
150+
package.
151+
152+
For example, say your directory tree consists solely of ``pkg/__init__.py``
153+
and ``pkg/a/b/c/d/mod.py``. When determining ``mod.py``'s fully qualified
154+
module name, mypy will look at ``pkg/__init__.py`` and conclude that the
155+
associated module name is ``pkg.a.b.c.d.mod``.
156+
157+
3. You'll notice that the above case still relies on ``__init__.py``. If
158+
you can't put an ``__init__.py`` in your top-level package, but still wish to
159+
pass paths (as opposed to packages or modules using the ``-p`` or ``-m``
160+
flags), :option:`--explicit-package-bases <mypy --explicit-package-bases>`
161+
provides a solution.
162+
163+
With :option:`--explicit-package-bases <mypy --explicit-package-bases>`, mypy
164+
will locate the nearest parent directory that is a member of the ``MYPYPATH``
165+
environment variable, the :confval:`mypy_path` config or is the current
166+
working directory. Mypy will then use the relative path to determine the
167+
fully qualified module name.
168+
169+
For example, say your directory tree consists solely of
170+
``src/namespace_pkg/mod.py``. If you run the following command, mypy
171+
will correctly associate ``mod.py`` with ``namespace_pkg.mod``::
172+
173+
$ MYPYPATH=src mypy --namespace-packages --explicit-package-bases .
174+
175+
If you pass a file not ending in ``.py[i]``, the module name assumed is
176+
``__main__`` (matching the behavior of the Python interpreter), unless
177+
:option:`--scripts-are-modules <mypy --scripts-are-modules>` is passed.
178+
179+
Passing :option:`-v <mypy -v>` will show you the files and associated module
180+
names that mypy will check.
181+
106182

107183
How mypy handles imports
108184
************************
@@ -326,6 +402,66 @@ on your system in an unconventional way.
326402
In this case, follow the steps above on how to handle
327403
:ref:`missing type hints in third party libraries <missing-type-hints-for-third-party-library>`.
328404

405+
406+
.. _finding-imports:
407+
408+
How imports are found
409+
*********************
410+
411+
When mypy encounters an ``import`` statement or receives module
412+
names from the command line via the :option:`--module <mypy --module>` or :option:`--package <mypy --package>`
413+
flags, mypy tries to find the module on the file system similar
414+
to the way Python finds it. However, there are some differences.
415+
416+
First, mypy has its own search path.
417+
This is computed from the following items:
418+
419+
- The ``MYPYPATH`` environment variable
420+
(a list of directories, colon-separated on UNIX systems, semicolon-separated on Windows).
421+
- The :confval:`mypy_path` config file option.
422+
- The directories containing the sources given on the command line
423+
(see :ref:`Mapping file paths to modules <mapping-paths-to-modules>`).
424+
- The installed packages marked as safe for type checking (see
425+
:ref:`PEP 561 support <installed-packages>`)
426+
- The relevant directories of the
427+
`typeshed <https://github.com/python/typeshed>`_ repo.
428+
429+
.. note::
430+
431+
You cannot point to a stub-only package (:pep:`561`) via the ``MYPYPATH``, it must be
432+
installed (see :ref:`PEP 561 support <installed-packages>`)
433+
434+
Second, mypy searches for stub files in addition to regular Python files
435+
and packages.
436+
The rules for searching for a module ``foo`` are as follows:
437+
438+
- The search looks in each of the directories in the search path
439+
(see above) until a match is found.
440+
- If a package named ``foo`` is found (i.e. a directory
441+
``foo`` containing an ``__init__.py`` or ``__init__.pyi`` file)
442+
that's a match.
443+
- If a stub file named ``foo.pyi`` is found, that's a match.
444+
- If a Python module named ``foo.py`` is found, that's a match.
445+
446+
These matches are tried in order, so that if multiple matches are found
447+
in the same directory on the search path
448+
(e.g. a package and a Python file, or a stub file and a Python file)
449+
the first one in the above list wins.
450+
451+
In particular, if a Python file and a stub file are both present in the
452+
same directory on the search path, only the stub file is used.
453+
(However, if the files are in different directories, the one found
454+
in the earlier directory is used.)
455+
456+
Setting :confval:`mypy_path`/``MYPYPATH`` is mostly useful in the case
457+
where you want to try running mypy against multiple distinct
458+
sets of files that happen to share some common dependencies.
459+
460+
For example, if you have multiple projects that happen to be
461+
using the same set of work-in-progress stubs, it could be
462+
convenient to just have your ``MYPYPATH`` point to a single
463+
directory containing the stubs.
464+
329465
.. _follow-imports:
330466

331467
Following imports
@@ -387,153 +523,3 @@ hard-to-debug errors.
387523
Adjusting import following behaviour is often most useful when restricted to
388524
specific modules. This can be accomplished by setting a per-module
389525
:confval:`follow_imports` config option.
390-
391-
392-
.. _mapping-paths-to-modules:
393-
394-
Mapping file paths to modules
395-
*****************************
396-
397-
One of the main ways you can tell mypy what to type check
398-
is by providing mypy a list of paths. For example::
399-
400-
$ mypy file_1.py foo/file_2.py file_3.pyi some/directory
401-
402-
This section describes how exactly mypy maps the provided paths
403-
to modules to type check.
404-
405-
- Mypy will check all paths provided that correspond to files.
406-
407-
- Mypy will recursively discover and check all files ending in ``.py`` or
408-
``.pyi`` in directory paths provided, after accounting for
409-
:option:`--exclude <mypy --exclude>`.
410-
411-
- For each file to be checked, mypy will attempt to associate the file (e.g.
412-
``project/foo/bar/baz.py``) with a fully qualified module name (e.g.
413-
``foo.bar.baz``). The directory the package is in (``project``) is then
414-
added to mypy's module search paths.
415-
416-
How mypy determines fully qualified module names depends on if the options
417-
:option:`--no-namespace-packages <mypy --no-namespace-packages>` and
418-
:option:`--explicit-package-bases <mypy --explicit-package-bases>` are set.
419-
420-
1. If :option:`--no-namespace-packages <mypy --no-namespace-packages>` is set,
421-
mypy will rely solely upon the presence of ``__init__.py[i]`` files to
422-
determine the fully qualified module name. That is, mypy will crawl up the
423-
directory tree for as long as it continues to find ``__init__.py`` (or
424-
``__init__.pyi``) files.
425-
426-
For example, if your directory tree consists of ``pkg/subpkg/mod.py``, mypy
427-
would require ``pkg/__init__.py`` and ``pkg/subpkg/__init__.py`` to exist in
428-
order correctly associate ``mod.py`` with ``pkg.subpkg.mod``
429-
430-
2. The default case. If :option:`--namespace-packages <mypy
431-
--no-namespace-packages>` is on, but :option:`--explicit-package-bases <mypy
432-
--explicit-package-bases>` is off, mypy will allow for the possibility that
433-
directories without ``__init__.py[i]`` are packages. Specifically, mypy will
434-
look at all parent directories of the file and use the location of the
435-
highest ``__init__.py[i]`` in the directory tree to determine the top-level
436-
package.
437-
438-
For example, say your directory tree consists solely of ``pkg/__init__.py``
439-
and ``pkg/a/b/c/d/mod.py``. When determining ``mod.py``'s fully qualified
440-
module name, mypy will look at ``pkg/__init__.py`` and conclude that the
441-
associated module name is ``pkg.a.b.c.d.mod``.
442-
443-
3. You'll notice that the above case still relies on ``__init__.py``. If
444-
you can't put an ``__init__.py`` in your top-level package, but still wish to
445-
pass paths (as opposed to packages or modules using the ``-p`` or ``-m``
446-
flags), :option:`--explicit-package-bases <mypy --explicit-package-bases>`
447-
provides a solution.
448-
449-
With :option:`--explicit-package-bases <mypy --explicit-package-bases>`, mypy
450-
will locate the nearest parent directory that is a member of the ``MYPYPATH``
451-
environment variable, the :confval:`mypy_path` config or is the current
452-
working directory. Mypy will then use the relative path to determine the
453-
fully qualified module name.
454-
455-
For example, say your directory tree consists solely of
456-
``src/namespace_pkg/mod.py``. If you run the following command, mypy
457-
will correctly associate ``mod.py`` with ``namespace_pkg.mod``::
458-
459-
$ MYPYPATH=src mypy --namespace-packages --explicit-package-bases .
460-
461-
If you pass a file not ending in ``.py[i]``, the module name assumed is
462-
``__main__`` (matching the behavior of the Python interpreter), unless
463-
:option:`--scripts-are-modules <mypy --scripts-are-modules>` is passed.
464-
465-
Passing :option:`-v <mypy -v>` will show you the files and associated module
466-
names that mypy will check.
467-
468-
469-
.. _finding-imports:
470-
471-
How imports are found
472-
*********************
473-
474-
When mypy encounters an ``import`` statement or receives module
475-
names from the command line via the :option:`--module <mypy --module>` or :option:`--package <mypy --package>`
476-
flags, mypy tries to find the module on the file system similar
477-
to the way Python finds it. However, there are some differences.
478-
479-
First, mypy has its own search path.
480-
This is computed from the following items:
481-
482-
- The ``MYPYPATH`` environment variable
483-
(a list of directories, colon-separated on UNIX systems, semicolon-separated on Windows).
484-
- The :confval:`mypy_path` config file option.
485-
- The directories containing the sources given on the command line
486-
(see :ref:`Mapping file paths to modules <mapping-paths-to-modules>`).
487-
- The installed packages marked as safe for type checking (see
488-
:ref:`PEP 561 support <installed-packages>`)
489-
- The relevant directories of the
490-
`typeshed <https://github.com/python/typeshed>`_ repo.
491-
492-
.. note::
493-
494-
You cannot point to a stub-only package (:pep:`561`) via the ``MYPYPATH``, it must be
495-
installed (see :ref:`PEP 561 support <installed-packages>`)
496-
497-
Second, mypy searches for stub files in addition to regular Python files
498-
and packages.
499-
The rules for searching for a module ``foo`` are as follows:
500-
501-
- The search looks in each of the directories in the search path
502-
(see above) until a match is found.
503-
- If a package named ``foo`` is found (i.e. a directory
504-
``foo`` containing an ``__init__.py`` or ``__init__.pyi`` file)
505-
that's a match.
506-
- If a stub file named ``foo.pyi`` is found, that's a match.
507-
- If a Python module named ``foo.py`` is found, that's a match.
508-
509-
These matches are tried in order, so that if multiple matches are found
510-
in the same directory on the search path
511-
(e.g. a package and a Python file, or a stub file and a Python file)
512-
the first one in the above list wins.
513-
514-
In particular, if a Python file and a stub file are both present in the
515-
same directory on the search path, only the stub file is used.
516-
(However, if the files are in different directories, the one found
517-
in the earlier directory is used.)
518-
519-
520-
Other advice and best practices
521-
*******************************
522-
523-
There are multiple ways of telling mypy what files to type check, ranging
524-
from passing in command line arguments to using the :confval:`files` or :confval:`mypy_path`
525-
config file options to setting the
526-
``MYPYPATH`` environment variable.
527-
528-
However, in practice, it is usually sufficient to just use either
529-
command line arguments or the :confval:`files` config file option (the two
530-
are largely interchangeable).
531-
532-
Setting :confval:`mypy_path`/``MYPYPATH`` is mostly useful in the case
533-
where you want to try running mypy against multiple distinct
534-
sets of files that happen to share some common dependencies.
535-
536-
For example, if you have multiple projects that happen to be
537-
using the same set of work-in-progress stubs, it could be
538-
convenient to just have your ``MYPYPATH`` point to a single
539-
directory containing the stubs.

0 commit comments

Comments
 (0)