Skip to content

Local file existence checking requires URL modules to be loaded in _datasource.py #12536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
EelcoPeacs opened this issue Dec 12, 2018 · 2 comments

Comments

@EelcoPeacs
Copy link
Contributor

While loading a local file with the loadtxt function I received an exception that was related to importing the urlopen module. Since I tried to open a local file I was a bit surprised by this error. Going through the stack-trace I ended up in the _datasource.py file.

Here on line 551, the module is tried to be loaded:

    # We import this here because importing urllib2 is slow and
    # a significant fraction of numpy's total import time.
    if sys.version_info[0] >= 3:
        from urllib.request import urlopen
        from urllib.error import URLError
    else:
        from urllib2 import urlopen
        from urllib2 import URLError

    # Test local path
    if os.path.exists(path):
        return True

I noticed that on line 558 the file is actually being checked for being a local file, which being true in my case should not require the urlopen module to be loaded.

I would like to propose to change the code at line 550 to:

    # Test local path
    if os.path.exists(path):
        return True

    # We import this here because importing urllib2 is slow and
    # a significant fraction of numpy's total import time.
    if sys.version_info[0] >= 3:
        from urllib.request import urlopen
        from urllib.error import URLError
    else:
        from urllib2 import urlopen
        from urllib2 import URLError

This will result in the same behavior but will be quicker when a file is a local file and, in my case prevent the exception.

Reproducing code example:

In my case the exception was cause by the missing _socket object/module which is being imported deep down when importing the urlopen module

Numpy/Python version information:

The proposed change is based on the current code, revision #12388

@mattip
Copy link
Member

mattip commented Dec 23, 2018

Could you make this into a pull request?

charris pushed a commit to charris/numpy that referenced this issue Jan 17, 2019
Check for local file first to avoid need to import urllib modules in
numpy._datasource.py. See numpy#12536
@seberg
Copy link
Member

seberg commented Sep 14, 2021

Seems like the linked PRs closed this.

@seberg seberg closed this as completed Sep 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants