I don't think it's a good idea to bypass pyvenv.cfg's home for the prefix calculation by looking at base_executable.

That said, I do think the failure mode and error log are pretty bad. We should instead fail with a message explaining that there is a mismatch in the Python installation used to create the virtual environment, and the one being used to run it. We should also suggest using python -m venv --upgrade to fix the issue.

We should also suggest using python -m venv --upgrade to fix the issue.

Thanks for the suggestion :) But python -m venv --upgrade doesn't work for me because the error occurs before the venv is created.

>>> ./pylink -m venv ./bad_venv --upgrade

Error: Command '['/home/xx/tmp/bad_venv/bin/pylink', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

Very few people use ./bad_venv/bin/python3 to start python (I use this for debugging), so they don’t even see the error message above.

I'm not trying to bypass home, but provide a way to calculate prefix when the home is inaccurate, just like if PREFIX and not prefix: ... does. Not sure if this has potential side effects, because the tests all passed, please let me know if there are.

Don't you have access to the base Python interpreter? The venv upgrade command is meant to be run with that, not with venv interpreter.

If not, I would like to see if we can figure out other way to deal with your use-case before opting for this approach.

I guess this error has nothing to do with the base interpreter, it comes from:

# Lib/venv/__init__.py
...
def _setup_pip(self, context):
    """Installs or upgrades pip in a virtual environment"""
    self._call_new_python(context, '-m', 'ensurepip', '--upgrade',
                          '--default-pip', stderr=subprocess.STDOUT)

The python used here is the python in the venv (/venv/bin/python).

>>> python3 -VV
Python 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]

>>> which python3
/usr/bin/python3

>>> python3 -m venv --upgrade ./bad_venv/
Error: Command '['/home/xx/tmp/bad_venv/bin/python3', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

I think we should treat this as a venv bug, it should check home and update it if necessary.

I found that the doc for venv describes home like this:

python -m venv /path/to/new/virtual/environment

This creates the target directory (including parent directories as needed) and places a pyvenv.cfg file in it with a home key pointing to the Python installation from which the command was run.

It seems that the doc assumes that the location to run python command is always in the Python installation directory (Normally, this is correct).
But now the home does not always point to the Python installation, it may point to any directory containing a python symlink used to create a venv. I'm not sure if this is what the designers of venv intended.

Now we seem to expect home to always point to the real Python installation. Can anyone else help us identify the solution? @FFY00

After looking into this a bit more, I think your proposed solution is reasonable. To fix the root of the issue, though, I opened GH-127895. We can apply your solution to 3.12 and 3.13 as a bugfix, and move away from using home entirely in 3.14.