Skip to content

GH-113464: Get LLVM from cpython-bin-deps on Windows #133278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

GH-113464: Get LLVM from cpython-bin-deps on Windows #133278

wants to merge 2 commits into from

Conversation

brandtbucher
Copy link
Member

@brandtbucher brandtbucher commented May 2, 2025 •
edited by bedevere-app bot
Loading

This makes LLVM easier to acquire for Windows JIT builds (like the one @zooba is going to do next week). We can now get it from https://github.com/python/cpython-bin-deps/tree/llvm.

I also cleaned up a couple things I tripped over while working on this:

  • Fix the regen target to work better with recent changes to how jit_stencils.h is built (and clean up after itself).
  • Don't touch jit_stencils.h unless it's actually changing, to avoid triggering a rebuild.

@brandtbucher brandtbucher added OS-windows build The build process and cross-build topic-JIT labels May 2, 2025
@brandtbucher brandtbucher requested a review from zooba May 2, 2025 00:27
@brandtbucher brandtbucher self-assigned this May 2, 2025
@brandtbucher brandtbucher requested review from a team and savannahostrowski as code owners May 2, 2025 00:27
@bedevere-app bedevere-app bot mentioned this pull request May 2, 2025
Closed
@zooba
Copy link
Member

zooba commented May 2, 2025

Looks great! Though so far, this is the part that makes me sad (watch the timestamps). I guess it's unavoidable though.

Fri, 02 May 2025 00:38:42 GMT
 Fetching external libraries...
Fri, 02 May 2025 00:38:42 GMT
 Fetching bzip2-1.0.8...
Fri, 02 May 2025 00:38:46 GMT
 Fetching mpdecimal-4.0.0...
Fri, 02 May 2025 00:38:47 GMT
 Fetching sqlite-3.49.1.0...
Fri, 02 May 2025 00:38:48 GMT
 Fetching xz-5.2.5...
Fri, 02 May 2025 00:38:49 GMT
 Fetching zlib-ng-2.2.4...
Fri, 02 May 2025 00:38:50 GMT
 Fetching external binaries...
Fri, 02 May 2025 00:38:50 GMT
 Fetching libffi-3.4.4...
Fri, 02 May 2025 00:38:51 GMT
 Fetching openssl-bin-3.0.16.2...
Fri, 02 May 2025 00:38:53 GMT
 Fetching tcltk-8.6.[15](https://github.com/python/cpython/actions/runs/14786163251/job/41515167379?pr=133278#step:4:16).0...
Fri, 02 May 2025 00:38:55 GMT
 Fetching llvm-19.1.7.0...
Fri, 02 May 2025 00:42:13 GMT
 Finished.

Just to confirm, these lines in the build output mean that it's working, yes? (I think the last one isn't from your script, but it's not an error, so that implies success)

  
  =============================================================
  JIT support for x86_64-pc-windows-msvc is still experimental!
             Please report any issues you encounter.           
  =============================================================
  
  Regenerate cases
  Generated sources are up to date

@zooba
Copy link
Member

zooba commented May 2, 2025

Another possibility is to integrate the LLVM fetch into the script itself, so that we can check whether the files are up to date first and only download LLVM if it's needed (using/ignoring a separate install, as appropriate - I could see this being a case where we'd prefer to ignore it when we don't know which version it is)

@Wulian233
Copy link
Contributor

Do we need to change 18 to 19 here to unify the build?

cpython/PCbuild/readme.txt

Lines 69 to 76 in a6ddd07

You can also use a specific version of clang-cl downloaded from
https://github.com/llvm/llvm-project/releases, e.g.
clang+llvm-18.1.8-x86_64-pc-windows-msvc.tar.xz.
Given you have extracted that to <my-clang-dir>, you can use it like so
build.bat --pgo "/p:PlatformToolset=ClangCL" "/p:LLVMInstallDir=<my-clang-dir>" "/p:LLVMToolsVersion=18"
Setting LLVMToolsVersion to the major version is enough, although you
can be specific and use 18.1.8 in the above example, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zooba zooba Awaiting requested review from zooba

@savannahostrowski savannahostrowski Awaiting requested review from savannahostrowski savannahostrowski is a code owner

Assignees

@brandtbucher brandtbucher

Labels
awaiting core review build The build process and cross-build OS-windows topic-JIT
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@brandtbucher @zooba @Wulian233