Can't install plyvel with standalone toolchain (0.10.2) #775

dhalperi · 2022-08-02T04:33:43Z

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

No, I'm not sure this ever worked in a previous version. This is the first version of standalone I've tried.

The library does work with a custom interpreter.

Description

plyvel requires leveldb installed on the system. With brew install leveldb, I get the file at:

/usr/local/Cellar/leveldb@1.22/1.22/include/leveldb/db.h

When resolving plyvel, the below stack trace is thrown. It seems that plyvel can't find the C headers.

🔬 Minimal Reproduction

Using macOS Monterey (12.5) on a 2017 Macbook Pro (x64)
Install leveldb from Homebrew
Use the following simplish repo:
bazel build @pip_plyvel//:pkg - will throw the given error
Comment out python_interpreter_target and rerun; will succeed.

Repo:
WORKSPACE:

workspace(name = "demo")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

##########################################################
## Python setup

http_archive(
    name = "rules_python",
    sha256 = "a3a6e99f497be089f81ec082882e40246bfd435f52f4e82f37e89449b04573f6",
    strip_prefix = "rules_python-0.10.2",
    url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.10.2.tar.gz",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python3_9",
    # Available versions are listed in @rules_python//python:versions.bzl.
    python_version = "3.9",
)

load("@python3_9//:defs.bzl", "interpreter")
load("@rules_python//python:pip.bzl", "pip_parse")

pip_parse(
    name = "pip",
    python_interpreter_target = interpreter,
    requirements_lock = "//:requirements.txt",
)

load("@pip//:requirements.bzl", "install_deps")

install_deps()

requirements.txt:

#
# This file is autogenerated by pip-compile with python 3.7
# To update, run:
#
#    pip-compile --allow-unsafe --generate-hashes --output-file=requirements.txt requirements.in
#
plyvel==1.4.0 \
    --hash=sha256:0741be98096db98929ba507c5d99be6b9e009e38fb1361e755e52e69b2a6a496 \
    --hash=sha256:4ea98bea04ebf0f44747bacdfafefc8827787106fbb787f0aedc46482b2dfd53 \
    --hash=sha256:61d6dc2edff07412be0bcfb20cd3a6cb0b2ac6e136e0d53ce91a2d9b10e89ebc \
    --hash=sha256:b4a4e25b75852a379744f14e743f17ae78513d0ba4f354be54199735487c71d7 \
    --hash=sha256:d50eb65421e40d788868943bbaa0e1b62e806a07323a21ce9a4b7bf6bea022ec \
    --hash=sha256:daef3f2076db41fa0a0fcbb3eabf1a36445739e101367356b7ad3c455b77b44d
    # via -r requirements.in

🔥 Exception or Error


ERROR: An error occurred during the fetch of repository 'pip_plyvel':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/pip_repository.bzl", line 445, column 13, in _whl_library_impl
		fail("whl_library %s failed: %s (%s)" % (rctx.attr.name, result.stdout, result.stderr))
Error in fail: whl_library pip_plyvel failed: Collecting plyvel==1.4.0
  Using cached plyvel-1.4.0.tar.gz (151 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: plyvel
  Building wheel for plyvel (setup.py): started
  Building wheel for plyvel (setup.py): finished with status 'error'
  Running setup.py clean for plyvel
Failed to build plyvel
 (  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-x86_64-3.9
      creating build/lib.macosx-10.9-x86_64-3.9/plyvel
      copying plyvel/_version.py -> build/lib.macosx-10.9-x86_64-3.9/plyvel
      copying plyvel/__init__.py -> build/lib.macosx-10.9-x86_64-3.9/plyvel
      running build_ext
      building 'plyvel._plyvel' extension
      creating build/temp.macosx-10.9-x86_64-3.9
      creating build/temp.macosx-10.9-x86_64-3.9/plyvel
      Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
      Please check your Xcode installation
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/include/python3.9 -c plyvel/_plyvel.cpp -o build/temp.macosx-10.9-x86_64-3.9/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++
      plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found
      #include "leveldb/db.h"
               ^~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for plyvel
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 81, in 
    main()
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 57, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/var/folders/zy/32xth96x23vb7bddgpscl7g40000gn/T/tmpf70uqj3w']' returned non-zero exit status 1.
)

🌍 Your Environment

Operating System:

macOS Monterey, x86-64 processor

Output of bazel version:

Bazelisk version: development
Build label: 5.2.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 7 16:11:03 2022 (1654618263)
Build timestamp: 1654618263
Build timestamp as int: 1654618263

Rules_python version:

0.10.2

Anything else relevant?

Is there a way to supply C headers in-project?

The text was updated successfully, but these errors were encountered:

dhalperi · 2022-08-02T04:41:35Z

It seems to me that /usr/local/include is not on the compile path. CC: @hrfuller who may know the most about this?

mvgijssel · 2022-08-29T14:40:34Z

Running into a similar problem trying to install gevent using pip_parse on macOS 12.5.1

      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -mmacosx-version-min=11.0 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fPIC -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/ncursesw -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DLIBEV_EMBED=1 -DEV_CLEANUP_ENABLE=0 -DEV_EMBED_ENABLE=0 -DEV_PERIODIC_ENABLE=0 -DEV_USE_REALTIME=1 -DEV_USE_MONOTONIC=1 -DEV_USE_FLOOR=1 -I/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/include/python3.9 -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent -I/install/include/python3.9 -c build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.c -o build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.o
      clang -bundle -undefined dynamic_lookup -arch arm64 -mmacosx-version-min=11.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -L/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/lib -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.o -L/install/lib -o build/lib.macosx-11.0-arm64-cpython-39/gevent/libev/_corecffi.abi3.so
      ld: warning: directory not found for option '-L/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/lib'
      ld: warning: directory not found for option '-L/install/lib'
      building 'gevent.libuv._corecffi' extension
      creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv
      creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv/src
      creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv/src/unix
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -mmacosx-version-min=11.0 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fPIC -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/ncursesw -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DLIBUV_EMBED=1 -D_LARGEFILE_SOURCE=1 -D_FILE_OFFSET_BITS=64 -D_DARWIN_USE_64_BIT_INODE=1 -D_DARWIN_UNLIMITED_SELECT=1 -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libuv/include -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libuv/src -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent -I/install/include/python3.9 -c build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.c -o build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.o -Wno-unreachable-code -Wno-deprecated-declarations -Wno-incompatible-sysroot -Wno-tautological-compare -Wno-implicit-function-declaration -Wno-unused-value -Wno-macro-redefined
      build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.c:50:14: fatal error: 'pyconfig.h' file not found
      #    include <pyconfig.h>
                   ^~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for gevent
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 83, in <module>
    main()
  File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 59, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/tmpt5hi3lcn']' returned non-zero exit status 1.
)

mvgijssel · 2022-08-29T14:47:44Z

Just came across #809 which might help for me, validating if this is the case and will post back here.

mvgijssel · 2022-08-29T15:28:28Z

Adding the most recent rules_python solved the issue for me 🎉

http_archive(
    name = "rules_python",
    sha256 = "b593d13bb43c94ce94b483c2858e53a9b811f6f10e1e0eedc61073bd90e58d9c",
    strip_prefix = "rules_python-0.12.0",
    url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.12.0.tar.gz",
)

github-actions · 2023-02-25T22:50:24Z

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

dhalperi · 2023-02-26T03:39:53Z

activity

chrislovecnm · 2023-08-07T18:44:19Z

@dhalperi can you test with the latest version of rules_python?

github-actions · 2024-02-03T22:50:25Z

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

github-actions · 2024-03-04T22:50:34Z

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"

mvgijssel mentioned this issue Sep 1, 2022

Move macOS provisioning into docker container vgijssel/setup#97

Open

github-actions bot added the Can Close? Will close in 30 days if there is no new activity label Feb 25, 2023

github-actions bot removed the Can Close? Will close in 30 days if there is no new activity label Feb 26, 2023

github-actions bot added the Can Close? Will close in 30 days if there is no new activity label Feb 3, 2024

github-actions bot closed this as completed Mar 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't install plyvel with standalone toolchain (0.10.2) #775

Can't install plyvel with standalone toolchain (0.10.2) #775

dhalperi commented Aug 2, 2022 •

edited

Loading

dhalperi commented Aug 2, 2022

mvgijssel commented Aug 29, 2022

mvgijssel commented Aug 29, 2022

mvgijssel commented Aug 29, 2022

github-actions bot commented Feb 25, 2023

dhalperi commented Feb 26, 2023

chrislovecnm commented Aug 7, 2023

github-actions bot commented Feb 3, 2024

github-actions bot commented Mar 4, 2024

Can't install plyvel with standalone toolchain (0.10.2) #775

Can't install plyvel with standalone toolchain (0.10.2) #775

Comments

dhalperi commented Aug 2, 2022 • edited Loading

🐞 bug report

Affected Rule

Is this a regression?

Description

🔬 Minimal Reproduction

🔥 Exception or Error

🌍 Your Environment

dhalperi commented Aug 2, 2022

mvgijssel commented Aug 29, 2022

mvgijssel commented Aug 29, 2022

mvgijssel commented Aug 29, 2022

github-actions bot commented Feb 25, 2023

dhalperi commented Feb 26, 2023

chrislovecnm commented Aug 7, 2023

github-actions bot commented Feb 3, 2024

github-actions bot commented Mar 4, 2024

dhalperi commented Aug 2, 2022 •

edited

Loading