Skip to content

Microsoft Store installer !adds /Scripts to user PATH env! #118498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dylanh724 opened this issue May 2, 2024 · 10 comments
Closed

Microsoft Store installer !adds /Scripts to user PATH env! #118498

dylanh724 opened this issue May 2, 2024 · 10 comments
Labels
OS-windows topic-installation type-bug An unexpected behavior, bug, or error

Comments

@dylanh724
Copy link

dylanh724 commented May 2, 2024

Bug report

Bug description:

Related: sphinx-doc/sphinx#12342

Microsoft Store adds python and pip to path for me -- great

...but not /Scripts, where all pip installs go?

Sure, let me find the path/to/Python/.... ⛔ Cannot, since the Microsoft Store apps are in some encrypted dir I cannot even view. I have to deep google search to find the path:

C:\Users\{user}\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts

I finally add this to user data and my pip installs work.

This feels wildly tedious to not add this to user path during installation. This is not a feature request, but a bug -- what's the point of adding pip to path if you can't use the apps?

CPython versions tested on:

3.10

Operating systems tested on:

Windows

@dylanh724 dylanh724 added the type-bug An unexpected behavior, bug, or error label May 2, 2024
@zooba
Copy link
Member

zooba commented May 2, 2024

You can python -m <command> most apps, and we encourage that pattern because managing PATH gets very complicated very quickly. Alternatively, create and activate a venv and you will get its Scripts folder on PATH.

pip will also print out the path you need if it installs commands that aren't already on PATH.

And in any case, Windows doesn't allow modifying PATH as part of their install technology, because they know how bad an idea it is. We don't really have a choice in this regard.

@zooba zooba closed this as not planned Won't fix, can't repro, duplicate, stale May 2, 2024
@dylanh724
Copy link
Author

dylanh724 commented May 2, 2024

You can python -m <command> most apps, and we encourage that pattern because managing PATH gets very complicated very quickly. Alternatively, create and activate a venv and you will get its Scripts folder on PATH.

pip will also print out the path you need if it installs commands that aren't already on PATH.

And in any case, Windows doesn't allow modifying PATH as part of their install technology, because they know how bad an idea it is. We don't really have a choice in this regard.

Perhaps this workaround should be in the Microsoft Store README? I'm fairly confident if installed via choco, the path would be there, which may confuse folks if they're transitioning.

0 pip apps show to run pip that way. For example, I'm terrible at Python and more of a .NET guy. I simply need a pip app then never have to use Python again. However, I did read the Microsoft Store description to ensure there were no gotcha's. This should really be added.

@zooba
Copy link
Member

zooba commented May 3, 2024

Perhaps a better first step would be to define a single Scripts folder that is shared by all Python versions (for non-venv installs, of course). That way there's only one path to add, and we don't have to worry about changing it on every release. I believe this is more like POSIX behaviour (again, for non-venv installs).

It would be a fairly simple change to sysconfig, but would only apply from 3.13 onwards (and only if we decide this week). @FFY00, @pfmoore, @pradyunsg any thoughts on this?

I'd propose making it only affect sysconfig.get_paths("nt_user")["scripts"]. This used to be a single directory shared by all installs, but that was terrible for sysconfig.get_paths("nt_user")["platlib"]. I think it'd be okay for scripts.

We'd still want to recommend python -m ... to make sure you run the one you intend, but at least for common/simple cases, having user Scripts be shared seems okay. (Default installs on Windows will use get_paths("nt")["scripts"] normally - I'm not proposing changing that, so most users won't even notice. Only those who end up installing down the --user path.)

@pfmoore
Copy link
Member

pfmoore commented May 3, 2024

The use case that needs a bit of thinking about is people with multiple Python installs. That's rare, but I don't have a good intuition for how rare (all I can say is that I fall into that category 🙂).

Things could get messy if I can pip install black in two Python environments. What should happen then? The second overwrites the first? Then if I uninstall the first, there's no wrapper even though black is still available in the second installation. Also, packages using the old feature of allowing packages to include scripts will actually get broken, because the scripts are listed in RECORD. Overwriting would mean that the old package's hash would no longer be correct.

@zooba
Copy link
Member

zooba commented May 3, 2024

all I can say is that I fall into that category

Me too! Though I do everything with Store installed Python (python3.x.exe) and venv or isolated installs these days, so this particular case never hits me and I don't have anything on PATH.

Things could get messy if I can pip install black in two Python environments. What should happen then?

[For complete clarity, we're talking about pip3.10 install --user black and then pip3.11 install --user black. Most other variations would be unaffected, except that Store installs always decide to use --user.]

I expect that black.exe and black3.exe refer to the one that was most recently installed (i.e. it overwrites) and black3.#.exe refers to the matched version. This isn't entirely unlike having two entries on PATH (today's scenario), except you'll get your most recent pip install version rather than most recently installed Python.

packages using the old feature of allowing packages to include scripts will actually get broken, because the scripts are listed in RECORD. Overwriting would mean that the old package's hash would no longer be correct.

Does anyone actually check RECORD after installation though? During extraction, sure, but I've not come across anyone validating an install by looking at RECORD (in part because it's trivial to update RECORD when you update one of the installed files).

Besides, if someone does check and the file has been overwritten, shouldn't it show that it's changed? This feels a bit like working as intended.

Then if I uninstall the first, there's no wrapper even though black is still available in the second installation.

Yeah, this is the problem, and it's basically common to any file that pip overwrites when installing. My preferred solution to the general problem1 wouldn't work here. I'm actually facing the same issue with #114881, which is largely why that has stalled, despite being (IMHO) the best way forward for PATH management.2

Presumably reinstalling in the other version would fix it, and that's a narrower set of people to document for ("people with multiple Python installs who install global tools in both and sometimes uninstall them from one but still want to use the tool" vs "people with Python who install global tools"). I don't know that there's anywhere we could make this obvious, unfortunately...

Footnotes

  1. Reference count if the new/old files are identical; abort if different.

  2. Again, excluding Scripts directories, though I've considered adding an entrypoints lookup to generate all the commands.

@pfmoore
Copy link
Member

pfmoore commented May 3, 2024

I expect that black.exe and black3.exe refer to the one that was most recently installed (i.e. it overwrites) and black3.#.exe refers to the matched version.

pip doesn't install versioned entry points, so no - there's only black.exe which will behave as I described. The problem is tat if I uninstall either copy of black, black.exe vanishes. It doesn't revert to the previously installed one. So it's very easy to end up with black installed but no wrapper available, which would be incredibly confusing for the sort of user we're trying to help here (a user who isn't comfortable manually managing PATH).

Does anyone actually check RECORD after installation though?

It makes sense for uninstall to check hashes and refuse to uninstall if they are incorrect. I don't think pip does this, but it could. And I can't speak for uv.

Yeah, this is the problem, and it's basically common to any file that pip overwrites when installing.

I'm pretty sure pip either refuses to overwrite files owned by a different package, or if not, then we have an active issue where we are discussing the possibility.

I don't know how Unix systems (with shared /usr/local/bin) address this. But in general, Unix users are more technically sophisticated than Windows users, so maybe they just don't find all this as confusing in the first place...?

To be honest, I think that "don't install stuff in the system Python" combined with something like pipx, plus a user-friendly environment manager tool, is the right answer here. The separation of concerns between the core and the packaging community makes it hard to advocate for this, though - I don't see the core team officially recommending pipx any time soon, and environment management is in a huge state of flux at the moment, to an extent because the core offering (the bare bones venv module) is so weak.

@zooba
Copy link
Member

zooba commented May 3, 2024

pip doesn't install versioned entry points, so no

Interesting, I guess I assumed it did because python -m pip install pip does. But I see now that pip's entry_points.txt specifies them all (and presumably something fixes up the pip3.10 one to match the current version?)

I'm pretty sure pip either refuses to overwrite files owned by a different package, or if not, then we have an active issue where we are discussing the possibility.

Last I heard there was an active issue, which has been running for a couple of years. I have big teams at work who regularly install on top of each others' files (semi on purpose), so I'm sure I would've heard something if the behaviour had changed.

To be honest, I think that "don't install stuff in the system Python" combined with something like pipx, plus a user-friendly environment manager tool, is the right answer here. The separation of concerns between the core and the packaging community makes it hard to advocate for this, though

Agreed on all counts. Maybe I really just need to get back to my py --activate idea and make that happen...

@pradyunsg
Copy link
Member

Note that there's also pypa/pip#4625 which is viewed as a bug and not a feature, and this proposal would rely on that "bug" as an explicit feature. :)

@dylanh724
Copy link
Author

dylanh724 commented May 4, 2024

Since this gets complicated, I'm mostly just thinking some nice readme documentation, perhaps under troubleshooting or prerequisites? Or a post-install sanity check section (EDIT: I like post-install sanity check section, personally -- right after install section since most attention span drops after install)?

@zooba
Copy link
Member

zooba commented May 7, 2024

this proposal would rely on that "bug" as an explicit feature. :)

Only for packages that directly bundle their own scripts. Nothing inherent to that bug (which is the same one I referred to earlier) prevents us from properly handling collisions between console script entry points.

I like post-install sanity check section, personally -- right after install

You mean like the message that pip printed out when you installed a script and it found that the directory wasn't on PATH?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS-windows topic-installation type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants