Skip to content

Regression (1.4.0-rc0) | Transient python pypi deps not working #2796

Closed
@JeroenSchmidt

Description

@JeroenSchmidt

🐞 bug report

Affected Rule

py_test() & py_library

Is this a regression?

This problem appears to happen after #2629

Description

Transient dependencies of python packages are not being provided to the test environment.

🔬 Minimal Reproduction

# lib/libA/tests/test_fixtures/test_mlflow.py
import mlflow
# lib/libA/BUILD

py_library(
    name = "fixtures",
    srcs = glob(["src/**/*.py"]),
    imports = ["src"],
    visibility = ["//visibility:public"],
    deps = [
        requirement("mlflow-skinny"),
    ],
)

py_test(
    name = "test",
    timeout = "long",
    srcs = glob(["tests/**"]),
    deps = [
        requirement("pytest"),
        requirement("pytest-cov"),
        ":test_fixtures",
    ],
)
# mydeps_without_hashes.txt
# py3.10

mlflow-skinny==2.14.3
pytest-cov==5.0.0
pytest==8.3.1
# .bazelrc
common --java_runtime_version=remotejdk_11
common --java_language_version=11
common --tool_java_runtime_version=remotejdk_11
common --tool_java_language_version=11

build --nolegacy_external_runfiles
build --incompatible_default_to_explicit_init_py

🔥 Exception or Error


==================================== ERRORS ====================================
_ ERROR collecting lib/libA/tests/test_mlflow.py _
ImportError while importing test module '/private/var/tmp/_bazel_USER1/bdf93a2b33b4c731566cb9fa9e8f36ec/sandbox/darwin-sandbox/27/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/lib/test_fixtures/test.runfiles/_main/lib/libA/tests/test_mlflow.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../rules_python++pip+pypi_deps_mydeps_310_pytest/site-packages/_pytest/python.py:493: in importtestmodule
    mod = import_path(
../rules_python++pip+pypi_deps_mydeps_310_pytest/site-packages/_pytest/pathlib.py:582: in import_path
    importlib.import_module(module_name)
../../../../../../../../../../../../execroot/_main/external/rules_python++python+python_3_10_aarch64-apple-darwin/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
:1050: in _gcd_import
    ???
:1027: in _find_and_load
    ???
:1006: in _find_and_load_unlocked
    ???
:688: in _load_unlocked
    ???
../rules_python++pip+pypi_deps_mydeps_310_pytest/site-packages/_pytest/assertion/rewrite.py:174: in exec_module
    exec(co, module.__dict__)
lib/libA/tests/test_mlflow.py:1: in 
    import mlflow
../rules_python++pip+pypi_deps_mydeps_310_mlflow_skinny/site-packages/mlflow/__init__.py:34: in 
    from mlflow import (
../rules_python++pip+pypi_deps_mydeps_310_mlflow_skinny/site-packages/mlflow/artifacts/__init__.py:9: in 
    from mlflow.exceptions import MlflowException
../rules_python++pip+pypi_deps_mydeps_310_mlflow_skinny/site-packages/mlflow/exceptions.py:5: in 
    from mlflow.protos.databricks_pb2 import (
../rules_python++pip+pypi_deps_mydeps_310_mlflow_skinny/site-packages/mlflow/protos/databricks_pb2.py:5: in 
    from google.protobuf.internal import enum_type_wrapper
E   ModuleNotFoundError: No module named 'google'

🌍 Your Environment

Operating System:

  
MACOS 15.3.2 (24D81)
  

Output of bazel version:

  
Bazelisk version: 1.24.0
Starting local Bazel server (8.1.1) and connecting to it...
Build label: 8.1.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Tue Feb 25 18:53:44 2025 (1740509624)
Build timestamp: 1740509624
Build timestamp as int: 1740509624
  

Rules_python version:

  
bazel_dep(name = "rules_python", version = "1.3.0")
git_override(module_name = "rules_python",
    remote = "https://github.com/bazel-contrib/rules_python",
    commit = "2cb920c1e52a85239d6bcc38919fbf143b514dac" # breaks
    # commit = "aa0d16c1463e4e26f6ed633ae83d9785a2ea9dfa" # working
    # commit="84351d4ec14e474bc196c0b8cd70e04fcc9a25ca" # working
)
  

Anything else relevant?
Adding protobuf to the BUILD deps results in another transient missing import error when import airflow is called.

    from opentelemetry.sdk.trace import Event as OTelEvent
E   ModuleNotFoundError: No module named 'opentelemetry'

Lock File Generation

I didn't regenerate the lockfile between testing the changing rules_python versions; but incase it helps I used rules_uv to create the initial lock file with rules_python==1.3

load("@rules_uv//uv:pip.bzl", uv_compile = "pip_compile")

uv_compile(
    name = "mydeps",
    py3_runtime = "@python_3_10//:py3_runtime",
    requirements_in = "mydeps_without_hashes.txt",
    requirements_txt = "mydeps_with_hashes.txt",
)

Metadata

Metadata

Assignees

Labels

type: pippip/pypi integration

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions