Skip to content

gh-127545: Specify minimum PyGC_Head alignment to fix build failure #127546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

fthain
Copy link

@fthain fthain commented Dec 3, 2024

As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.

This alignment requirement is documented but it's not actually encoded.
The code works when python happens to run on a platform that has a large
minimum alignment, but fails otherwise.

Since we know that 2 bits are needed, we also know the minimum alignment
that's needed. Let's make that explicit, so the compiler can then make
sure those 2 bits are available.

This patch fixes a segfault in _bootstrap_python. It also clarifies the
code by making the actual requirement explicit.

This bug was investigated by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600

Although Adrian's patch isn't really correct (because natural alignment
is not needed), he deserves full credit for finding the root cause.

#127545

@ghost
Copy link

ghost commented Dec 3, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Dec 3, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a blurb entry. Could you add one using one of the links the bot gave?

@ZeroIntensity ZeroIntensity added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes labels Dec 3, 2024
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to force push--everything is squashed at the end anyway.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provisionally, this LGTM. (I say "provisionally" because I don't totally understand the issue--there's just nothing wrong with this PR from a triage standpoint.)

@fthain
Copy link
Author

fthain commented Dec 8, 2024

Provisionally, this LGTM. (I say "provisionally" because I don't totally understand the issue--there's just nothing wrong with this PR from a triage standpoint.)

I think the patch can made more understandable, by making use of the existing macro definitions. I will update the branch.

@tomasr8 tomasr8 removed the needs backport to 3.12 only security fixes label Apr 10, 2025
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.14 bugs and security fixes label May 8, 2025
@fthain
Copy link
Author

fthain commented May 12, 2025

I just noticed that @markshannon refactored the same headers 2 months ago. So I have now updated the fix to resolve those merge conflicts. Please review. This bug is still present in 3.14.0beta1.

@stefanor
Copy link
Contributor

We're applying this patch in Debian's python3.14 to get it to build on m68k.

@fthain
Copy link
Author

fthain commented Jun 1, 2025

I see that one of Debian's 3.14.0b2 packages (the one --with-pydebug) fails with an assertion failure that hasn't been addressed in my patch. I'm presently testing a patch to deal with that too, by aligning struct _object.
https://buildd.debian.org/status/fetch.php?pkg=python3.14&arch=m68k&ver=3.14.0%7Eb2-1&stamp=1748435657&raw=0

As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.

This alignment requirement is documented but it's not actually encoded.
The code only works when python happens to run on a platform that has a
sufficiently large minimum alignment for the structs in question.

The same problem arises with PyObject pointers because the least
significant bits get used for PyStackRef tags.

Since we know that 2 bits are needed, we also know the minimum alignment
that's needed. Let's make that explicit, so the compiler can then make
those bits available.

This patch fixes a segfault in _bootstrap_python. In 3.14.0 beta 2
this fixes the "Assertion `!PyStackRef_IsTaggedInt(ref)' failed" when
built with --config-pydebug.

Also, making the requirements explicit improves clarity.

This bug was previously investigated by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600

Although Adrian's patch isn't really correct (because natural alignment
is not needed), he deserves full credit for finding the root cause.
@fthain
Copy link
Author

fthain commented Jun 2, 2025

I pushed this branch a second time after rebasing on 'main' instead of v3.14.0b2... but I'm still not sure which branch I should be using -- 3.14 or main?

@ZeroIntensity
Copy link
Member

main, we'll automatically backport it to 3.14 and 3.13 using the bot. That said, the incorrect rebase triggered a review from every codeowner, so any more activity on this PR will flood inboxes (removing requests for review doesn't unsubscribe).

Would you opening a new PR? Feel free to tag me on it and I'll apply the necessary labels.

@fthain
Copy link
Author

fthain commented Jun 2, 2025

Would you opening a new PR? Feel free to tag me on it and I'll apply the necessary labels.

Sorry for the mess I made when I rebased on 3.14.
I've now opened PR#135016.

@hugovk
Copy link
Member

hugovk commented Jun 2, 2025

It's best to avoid rebasing, just merge main into your branch if needed. Everything is squash merged at the end anyway.

https://devguide.python.org/getting-started/pull-request-lifecycle/#quick-guide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants