Improve import time of various stdlib modules #118761
Comments
|
Sure! |
|
I've opened a PR over at the
@layday were you planning on tackling these?
This seems to be solved on main, |
importlib.metadataCo-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
This has been merged and released in version 8.4 of
I've submitted python/importlib_metadata#502 that defers zip import, and python/importlib_metadata#503 which defers |
|
(removing the 3.14 label since features always target the main branch) |
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Importing `pickle` is now roughly 25% faster. Importing the `re` module is no longer needed and thus `re` is no more implicitly exposed as `pickle.re`. --------- Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
|
@picnixz: I suggest to close this issue. |
annotationlib is used quite a few times in typing.py, but I think the usages are just rare enough that this makes sense. The import would get triggered by: - Using get_type_hints(), evaluate_forward_ref(), and similar introspection functions - Using a string annotation anywhere that goes through _type_convert (e.g., "Final['x']" will trigger an annotationlib import in order to access the ForwardRef class). - Creating a TypedDict or NamedTuple (unless it's empty or PEP 563 is on). Lots of programs will want to use typing without any of these, so the tradeoff seems worth it.
annotationlib is used quite a few times in typing.py, but I think the usages are just rare enough that this makes sense. The import would get triggered by: - Using get_type_hints(), evaluate_forward_ref(), and similar introspection functions - Using a string annotation anywhere that goes through _type_convert (e.g., "Final['x']" will trigger an annotationlib import in order to access the ForwardRef class). - Creating a TypedDict or NamedTuple (unless it's empty or PEP 563 is on). Lots of programs will want to use typing without any of these, so the tradeoff seems worth it.
|
For |
Backport upstream patch that fixes lazy importing in the `subprocess` module that caused bugs in the `__del__` finalizer. Ref: python/cpython#118761 (comment)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
annotationlib is used quite a few times in typing.py, but I think the usages are just rare enough that this makes sense. The import would get triggered by: - Using get_type_hints(), evaluate_forward_ref(), and similar introspection functions - Using a string annotation anywhere that goes through _type_convert (e.g., "Final['x']" will trigger an annotationlib import in order to access the ForwardRef class). - Creating a TypedDict or NamedTuple (unless it's empty or PEP 563 is on). Lots of programs will want to use typing without any of these, so the tradeoff seems worth it.
…thon#132103) Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
…ython#132035) Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
…lazy This ensures that if we jump through some hoops to make sure something is imported lazily, we don't regress on importing it. I recently already accidentally made typing import warnings and annotationlib eagerly.
…132614) This ensures that if we jump through some hoops to make sure something is imported lazily, we don't regress on importing it. I recently already accidentally made typing import warnings and annotationlib eagerly. Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
|
#132614 adds a test helper If anyone reading this is interested, it'd be helpful to add test cases using this new helper to other modules that we've previously optimized by deferring imports. |
Feature or enhancement
Proposal:
Following on from #109653, further improvements can be made to import times.
Links to previous discussion of this feature:
https://discuss.python.org/t/deferred-computation-evalution-for-toplevels-imports-and-dataclasses/34173
For example:
importlib.metadatais often used for tasks that need to happen at import, e.g. to enumerate/load entry point plug-ins, so it might be worth seeing if we can cut down its own import time a bit more.importlib.metadataimportszipfileat the top for a function that won't be called in the vast majority of cases. It also importsimportlib.abc, which in turn importsimportlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method inimportlib.metadatawould be harmless. Some other less frequently-used imports which are only accessed once or twice, such asjson, could also be tucked away in their calling functions.Linked PRs
pprint#122725socketby writingsocket.errorTabas a constant and lazy import modules #121424mimetypes#126979pickle#128732reimport inbase64.b16decodefor a more efficient alternative #128736secrets#128738csv#128858tomllib#128907pstatsandzipfileby removing imports totyping#128981sqlite3#129118subprocess#129427warningsinthreading#129428subprocess(GH-129427) #129447subprocess(GH-129427) #129448warnings#129765traceback#129811reinlocale#129860dataclasses#129925cmdmodule #130056subprocess(GH-129427)" #130201subprocess(GH-129427)" (GH-130201) #130204subprocess(GH-129427)" (GH-130201) #130205sqlite3#131796ast#132024annotationlib#132028enum._simple_enumforannotationlib.Format#132031shlex#132036string#132037email.quoprimimeremovingreimport #132046_types#132103The text was updated successfully, but these errors were encountered: