Skip to content

Check joblib memmap alignment fix #22607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

lesteve
Copy link
Member

@lesteve lesteve commented Feb 25, 2022

This is a draft PR which should not be merged. Its goal is to check that joblib/joblib#1254 makes the fix in #21654 not necessary any more.

@lesteve lesteve force-pushed the check-joblib-memmap-alignment branch from 7d41541 to f40cec3 Compare February 25, 2022 04:14
@lesteve lesteve force-pushed the check-joblib-memmap-alignment branch from 254446e to 66c2e36 Compare February 25, 2022 04:38
@lesteve
Copy link
Member Author

lesteve commented Feb 25, 2022

Great there is still the segmentation fault in the Ubuntu_Bionic py37_conda_forge_openblas_ubuntu_1804 build without the fix: https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=38592&view=logs&j=4371db68-60d2-5800-021d-f190a09db133&t=ae1b5267-f8ef-579b-7034-78324f7c3041&l=247

@lesteve lesteve force-pushed the check-joblib-memmap-alignment branch from 7e267d1 to af06a90 Compare February 25, 2022 06:25
@lesteve
Copy link
Member Author

lesteve commented Feb 25, 2022

The good news is that the Ubuntu_Bionic py37_conda_forge_openblas_ubuntu_1804 does not have the segmentation fault.

I don't really understand the failures which seem to be related to setuptools bundling distutils. I don't really see how installing joblib development version can break this and why these builds break and not others ... the error is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/runner/work/1/s/sklearn/utils/_show_versions.py", line 85, in show_versions
    deps_info = _get_deps_info()
  File "/Users/runner/work/1/s/sklearn/utils/_show_versions.py", line 69, in _get_deps_info
    mod = importlib.import_module(modname)
  File "/usr/local/miniconda/envs/testvenv/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/miniconda/envs/testvenv/lib/python3.10/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/usr/local/miniconda/envs/testvenv/lib/python3.10/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/usr/local/miniconda/envs/testvenv/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 72, in do_override
    ensure_local_distutils()
  File "/usr/local/miniconda/envs/testvenv/lib/python3.10/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /usr/local/miniconda/envs/testvenv/lib/python3.10/distutils/core.py

@lesteve
Copy link
Member Author

lesteve commented Feb 25, 2022

The error seems to be a problem in setuptools I can reproduce by importing pip before setuptools, this seems like a side-effect of removing the usage of distutils version classes in joblib master ... I opened #22614 about this.

@lesteve
Copy link
Member Author

lesteve commented Feb 25, 2022

I found a setuptools issue that says that this won't be fixed: pypa/setuptools#3044

@lesteve
Copy link
Member Author

lesteve commented Feb 25, 2022

Summary:

So this PR has reached his goal and can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant