Add prefix at the end of the args list, as noted above. Also, supplying both root and prefix arguments doesn't really make sense (and may even be rejected by pip at some point in the future, if it isn't already) so it would be better to make these arguments mutually exclusive.

Thinking about it, I'm not sure they should be mutual exclusive.

If root equals $(DESTDIR) and prefix equals $(prefix), I would guess we want to install into $(DESTDIR)/$(prefix). I need to look more closely at this later today.

I don't know enough about the Unix build process to know if using --prefix instead of --root is the right thing to do here. I'm going to mark my review as resolved (and approve the PR) but I'd recommend getting someone who knows how Unix/MacOS installs work to check this part of the change before merging (unless, of course, you are such a person and you're happy it's right 🙂)

I can test it on *nix and macOS, but I know very little about pip/ensurepip :)

OK. Looking at the pip source, --prefix corresponds roughly to the sysconfig scheme posix_prefix. And --root can be used with --prefix, it basically rebases the installation directories on the root directory.

So if you have prefix=/foo/bar and root=/bax/quux, then you'd get purelib pointing to /baz/quux/foo/bar/lib/pythonX.Y/site-packages. With just prefix you'd get /foo/bar/lib/pythonX.Y/site-packages. And with just root you'd get /baz/quux/lib/pythonX.Y/site-packages.

There's all sorts of qualifications and special cases (because this is packaging 🙁) but that's the basic idea.

Given that the original issue was around cross-compilation for Android, I have zero idea how any of this would impact that actual scenario, though...

I would guess you'd use --root for the directory where your target rootfs is being populated, and then you'd use --prefix for the path within the target rootfs.

cc. @gpshead, since you dabble with embedded systems IIRC.

i'm not the right person to answer this either, i've poked #build on discord to see if anyone else is.

I can confirm that using --prefix for the expected location in the destination filesystem and --root(=$DESTDIR) for the current location of the virtual filesystem where the package is being prepared makes total sense. (and is how setup.py works )

Having only one or the other leads to headaches;

TL;DR: We expect --prefix to change the install location of files and the path they use to reference one another while --root should change the location we put files without changing how they refer to one another

With a basic build process and environment as such :

SOURCE_DIR=/tmp/py3.12.3
TARGET_ROOT=/tmp/build.out
TARGET_PREFIX=/opt/python/py3.12.3
cd ${SOURCE_DIR}
configure --prefix=${TARGET_PREFIX} --with-ensurepip=no [...]
make
make DESTDIR=${TARGET_ROOT} install altinstall

We get these results :

${TARGET_ROOT}${TARGET_PREFIX}/bin/python3 -m ensurepip
head -n 1 ${TARGET_ROOT}${TARGET_PREFIX}/bin/pip
#!/tmp/build.out/opt/python/py3.12.3/bin/python3

${TARGET_ROOT}${TARGET_PREFIX}/bin/python -m ensurepip --root ${TARGET_ROOT}
head -n 1 ${TARGET_ROOT}${TARGET_ROOT}${TARGET_PREFIX}/bin/pip
#!/tmp/build.out/tmp/build.out/opt/python/py3.12.3/bin/python3

In both cases the script installed contains the ${TARGET_ROOT} in the shebang when it shouldn't, but specificaly using --root behaves the way I would expect --prefix to work if no prefix had been set in the configure script.

The expected install locations for pip3 should be:

• --prefix : ${PREFIX}/bin/pip3 with shebang #!${PREFIX}/bin/python3
• --root (assuming no prefix) : ${ROOT}/bin/pip3 with shebang #!/bin/python3
• --prefix and --root : ${ROOT}${PREFIX}/bin/pip3 with shebang #!${PREFIX}/bin/python3

The behaviour becomes even weirder when using --with-ensurepip=(install|upgrade) in configure.
It triggers a call to something equivalent to :

LD_LIBRARY_PATH=${SOURCE_DIR} ${SOURCE_DIR}/python -E -m ensurepip [--upgrade] --root=${DESTDIR}

But this call ignores the --root parameter completely and just acts on the host system if an installation using the same prefix exists. The -E parameter makes setting PYTHONHOME irrelevant but omitting it doesn't resolve the issue.

With thing the way they are currently, with-ensurepip=no is the only option making sense in configure. After that, I see two quick and dirty solutions :

• calling ${TARGET_ROOT}${TARGET_PREFIX}/bin/python3 -m ensurepip on the build host, and fixing the shebang in the scripts afterwards
• adding ${TARGET_PREFIX}/bin/python3 -m ensurepip as a post-install action on the destination host

I agree that the proper solution would be to have the makefile call something along the lines of:

I'm not sure the trailing / is necessary.

One issue remains when cross-compiling (eg. AARCH64 binaries on a X86_64 host), the build host (x86_64) must be able to execute the target's binaries (AARCH64) for the actual call to work. Assuming ensurepip is pure python, the call should use the host's python3 binary (x86_64) with --prefix and --root rather than the one being built (AARCH64).

I don't see it as a blocking issue since the person building the package should be able to tell if they can use --with-ensurepip or if they should rather call ensurepip from the build host's python rather than letting the Makefile guess/try.

EDIT:
I forgot to mention that the actual module installation path follows the behaviour shown with the scripts and shebang. I just used them to illustrate the expected and actual behaviours

EDIT2: Added TL;DR an some clarifications

I think we should explain the difference between root and prefix better. Per now, it is not obvious how they differ.

I agree it's not obvious. And that's speaking as a pip maintainer... It's all shrouded in mystery and legacy behaviour dating back to the origins of distutils, I think. And it's very much to do with Unix - on Windows, those options are essentially never used (which is mostly why I don't understand them...)

But I think it's fair to assume that anyone using these options is doing so because they know of the same options in pip, and so they understand what they are doing (or at least their confusion isn't our problem). It's not ideal, but OTOH, this isn't the place for a tutorial on packaging concepts, either.

root vs prefix goes back to the base unix model I think, and may be explained in make documentation or something similarly old.

Something along the lines: the prefix (for installation) can be /usr or /usr/local or /opt/my-program but the root directory (for building) is /tmp/something, so the actual part where files are created is /tmp/something/usr/local/bin/python but strings embedded in the programs/libraries/docs look like /usr/local/bin/python so that they’re good at runtime.

(don’t ask me which of the three values corresponds to DESTDIR)

(edit: better replies at #17634 (comment))

DESTDIR is the make equivalent of specifying --root