Skip to content

Add retries to downloads of Windows dependencies #134262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
emmatyping opened this issue May 19, 2025 · 4 comments
Open

Add retries to downloads of Windows dependencies #134262

emmatyping opened this issue May 19, 2025 · 4 comments
Labels
build The build process and cross-build infra CI, GitHub Actions, buildbots, Dependabot, etc. type-feature A feature request or enhancement

Comments

@emmatyping
Copy link
Member

emmatyping commented May 19, 2025 •
edited by bedevere-app bot
Loading

generate_sbom.py downloads sources to verify their hashes. Sometimes these downloads flake, like in https://github.com/python/cpython/actions/runs/15118338626/job/42494602073?pr=134253

I've made a PR to add retries with exponential backoff:

Linked PRs
@emmatyping emmatyping added the build The build process and cross-build label May 19, 2025
@bedevere-app bedevere-app bot mentioned this issue May 19, 2025
Merged
@picnixz picnixz added type-feature A feature request or enhancement infra CI, GitHub Actions, buildbots, Dependabot, etc. labels May 20, 2025
hugovk pushed a commit that referenced this issue May 21, 2025
@emmatyping @donBarbos
gh-134262: Add retries to generate_sbom.py (#134263)
0c5a8b0 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@hugovk hugovk closed this as completed May 21, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 21, 2025
@emmatyping @donBarbos @miss-islington
pythongh-134262: Add retries to generate_sbom.py (pythonGH-134263)
d99a712 
(cherry picked from commit 0c5a8b0)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@bedevere-app bedevere-app bot mentioned this issue May 21, 2025
Merged
@emmatyping emmatyping reopened this May 23, 2025
@bedevere-app bedevere-app bot mentioned this issue May 23, 2025
Merged
@emmatyping
Copy link
Member Author

Re-opening since the underlying issue isn't resolved yet.

lkollar pushed a commit to lkollar/cpython that referenced this issue May 26, 2025
@emmatyping @donBarbos @lkollar
pythongh-134262: Add retries to generate_sbom.py (python#134263)
2ce7e21 
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
hugovk pushed a commit that referenced this issue May 27, 2025
@emmatyping
gh-134262: increase retries in Tools/build/generate_sbom.py (#134558)
3f9eb55
@emmatyping emmatyping changed the title Add retries to Tools/build/generate_sbom.py Add retries to downloads of Windows dependencies May 28, 2025
@bedevere-app bedevere-app bot mentioned this issue May 28, 2025
Merged
zooba pushed a commit that referenced this issue May 28, 2025
@emmatyping
gh-134262: Add retries to downloads in PCbuild\get_external.py (GH-13…
e9d845b 
…4820)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 28, 2025
@emmatyping @miss-islington
pythongh-134262: Add retries to downloads in PCbuild\get_external.py (p…
d6e4844 
…ythonGH-134820)

(cherry picked from commit e9d845b)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
@bedevere-app bedevere-app bot mentioned this issue May 28, 2025
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 28, 2025
@emmatyping @miss-islington
pythongh-134262: Add retries to downloads in PCbuild\get_external.py (p…
184db42 
…ythonGH-134820)

(cherry picked from commit e9d845b)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
This was referenced May 28, 2025
Merged
Merged
zooba pushed a commit that referenced this issue May 28, 2025
@miss-islington @emmatyping
gh-134262: Add retries to downloads in PCbuild\get_external.py (GH-13…
029ecea 
…4820)

(cherry picked from commit e9d845b)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
zware pushed a commit that referenced this issue May 28, 2025
@emmatyping
gh-134262: Fix off by one errors in download retry functions (GH-134867)
e64395e
zware pushed a commit that referenced this issue May 28, 2025
@miss-islington @emmatyping
[3.14] gh-134262: Add retries to downloads in PCbuild\get_external.py (…
34b4515 
…GH-134865)

(cherry picked from commit e9d845b)

Includes fix for off-by-one error from GH-134867
(cherry-picked from commit e64395e)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
zware pushed a commit that referenced this issue May 28, 2025
@miss-islington @emmatyping @donBarbos
[3.14] gh-134262: Add retries to generate_sbom.py (GH-134460)
7dc4496 
(cherry picked from commit 0c5a8b0)

Includes fix for off-by-one error from GH-134867
(cherry-picked from commit e64395e)

Co-authored-by: Emma Smith <emma@emmatyping.dev>
Co-authored-by: Semyon Moroz <donbarbos@proton.me>
@emmatyping
Copy link
Member Author

I skimmed a couple of pages of PRs and haven't seen any failures due to these connection issues, so hopefully this worked! We can always bump numbers or re-evaluate the solution in a future PR if need be.

@StanFromIreland
Copy link
Contributor

A failure (at one of the lines you recently modified) popped up today: https://github.com/python/cpython/actions/runs/15565936921/job/43829935370?pr=135355

@emmatyping
Copy link
Member Author

Yeah, we should probably catch both URLError and ConnectionError in these retry loops.

@emmatyping emmatyping reopened this Jun 10, 2025
@bedevere-app bedevere-app bot mentioned this issue Jun 10, 2025
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build The build process and cross-build infra CI, GitHub Actions, buildbots, Dependabot, etc. type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hugovk @emmatyping @picnixz @StanFromIreland