Skip to content

The implementation for suggestions in "ModuleNotFoundError" #137671

@Locked-chess-official

Description

@Locked-chess-official

Feature or enhancement

Previous discussion here. This issue for the implemention of the idea.

Before all, I find a better way to get the list of the module:

import os
import sys
from importlib import machinery

def scan_dir(path):
    """
    Return all of the packages in the path without import
    contains:
      - .py file
      - directory with "__init__.py"
      - the .pyd/so file that has right ABI
    """
    if not os.path.isdir(path):
        return []

    suffixes = machinery.EXTENSION_SUFFIXES
    result = []

    for name in os.listdir(path):
        full_path = os.path.join(path, name)

        # .py file
        if name.endswith(".py") and os.path.isfile(full_path):
            modname = name[:-3]
            if modname.isidentifier():
                result.append(modname)

        # directory with "__init__.py"
        elif os.path.isdir(full_path):
            init_file = os.path.join(full_path, "__init__.py")
            if os.path.isfile(init_file) and name.isidentifier():
                result.append(name)

        # the .pyd/so file that has right ABI
        elif os.path.isfile(full_path):
            for suf in suffixes:
                if name.endswith(suf):
                    modname = name[:-len(suf)]
                    if modname.isidentifier():
                        result.append(modname)
                    break

    return sorted(result)

def find_all_packages():
    return sorted(sum([scan_dir(i) if i and
                isinstance(i, str) and not
                i.endswith("idlelib") else []
                for i in sys.path ], []) + 
                list(sys.builtin_module_names))

# example
if __name__ == "__main__":
    print(find_all_packages())

The new test for test here:

No. number of entries used time(/s) average time(/s) result
1 5 0.064 0.013 success
2 5 0.072 0.014 success
3 5 0.052 0.010 success
4 5 0.056 0.011 success
5 5 0.057 0.011 success
6 6 0.081 0.014 success
7 6 0.062 0.010 success
8 6 0.091 0.015 success
9 6 0.064 0.011 success
10 6 0.064 0.011 success

The speed test here:

tool function arg number average time(/ms)
timeit find_all_packages (no args) 1000 8.567
timeit scan_dir path/to/site-packages 1000 4.845

And the script behaves better in finding ".pyd" modules.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-featureA feature request or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions