Hi, thanks for the report! Can you please try to come up with a reproducer that does not include rust, but only C-API?

I don't think it's a CPython problem.

PyO3 has to replicate the C API at the ABI level, so it's not a surprise when bumping python versions that if structures change we need to make adjustments.

The PyUnicode data access macros are a known problem for PyO3 and to be honest I think the better option might be to just remove our mapping of them. But we can discuss that downstream.

I suspect the "offending" change would be #128196, but yeah--we won't guarantee any stability with the internals, especially in the alpha phase.

Though, I am having some thoughts about whether or not the structure of PyASCIIObject is actually internal, because we do technically expose the structure in the non-limited API.

cc @encukou for the stability rules here.

By my reading of PEP-387, backwards-incompatible changes to PyASCIIObject needs deprecation period (or SC exception). However, that would be C API changes: rewriting that header in Rust is, unfortunately, not covered by any guarantees.
IMO, CPython should still try to keep more stable than promised. But this is a bit of a tight spot regarding backwards compatibility and free-threading support.

I see #128196 preserves C API compatibility but introduces an implementation-defined C feature (short bitfields) to do it. I fear that this will bite us in the future.

By my reading of PEP-387, backwards-incompatible changes to PyASCIIObject needs deprecation period (or SC exception). However, that would be C API changes: rewriting that header in Rust is, unfortunately, not covered by any guarantees.
IMO, CPython should still try to keep more stable than promised. But this is a bit of a tight spot regarding backwards compatibility and free-threading support.

If we have to guarantee object layout stuff, one possible negotiation point would be just change for the free-threaded build because it is kind of an experimental build, so some of the breaking changes will be acceptable. (And we already break ABI compatibility between the default build and the free-threaded build)

cc @colesbury

Reverted to give everyone time to stop relying on these details. But, note that:

• again, struct layout is not guaranteed to stay fixed across CPython versions, unless it's part of the Stable ABI including all members (like Py_buffer for example).
• I plan to deprecate these structs entirely, after adding some missing API.

Thanks for the report!