Skip to content

gh-97588: Move ctypes struct/union layout logic to Python #123352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 95 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
ac1d0a3
Add classes that will implement the struct packing
encukou Jul 25, 2024
b35a97c
Instantiate the layout and take `_align_` from it
encukou Jul 25, 2024
2cc8b40
Get base size and align; etc.
encukou Jul 27, 2024
a9fd94a
Sorta start creating CField objects
encukou Jul 27, 2024
c4d3c66
Turn constructor into initializer
encukou Jul 27, 2024
cbb3081
Use goto for error handling
encukou Aug 8, 2024
1adb23a
Handle Py_DECREF(prop) in the error block
encukou Aug 8, 2024
01ee4f8
Pass bit_size and name to CField
encukou Aug 8, 2024
eb7666c
Use name from prop
encukou Aug 8, 2024
be6113f
Use desc from prop
encukou Aug 8, 2024
a7f05f9
Use bit_size from prop; avoid pair
encukou Aug 8, 2024
ef24a0c
Pass None for undefined bit_size, not -1
encukou Aug 8, 2024
728ec0a
Fix a refleak
encukou Aug 8, 2024
5ef81b3
Move bitsize check to CField initializer
encukou Aug 8, 2024
16c2d77
desc is proto
encukou Aug 8, 2024
14fd92d
Remove the unused TYPEFLAG_HASBITFIELD and TYPEFLAG_HASUNION
encukou Aug 8, 2024
e3337c2
Use -1 for "not is_bitfield"
encukou Aug 8, 2024
f3c678d
Handle _swappedbytes_ in Python
encukou Aug 9, 2024
0c47fa2
Put layout mode in the field
encukou Aug 9, 2024
2eefab1
error handling
encukou Aug 9, 2024
5b1723c
Take out is_bitfield arg
encukou Aug 22, 2024
9507752
Keep _pack_ on the field
encukou Aug 22, 2024
dc859c3
Stick the index in the struct
encukou Aug 22, 2024
beeab0b
Add packstate struct, with pfield_size
encukou Aug 22, 2024
accb589
Put pbitoffs in the state
encukou Aug 22, 2024
71fbfb8
Put the size in the state
encukou Aug 22, 2024
9c6e3b0
Add offset to the state
encukou Aug 22, 2024
a6f8fdc
Put align in the state
encukou Aug 22, 2024
6803815
Move getfunc/setfunc to __new__
encukou Aug 22, 2024
e0fc0af
Start verifying the internal pack state
encukou Aug 23, 2024
a6814b7
Put offset & align in for the simple cases
encukou Aug 23, 2024
3771721
Remove unnecessary condition
encukou Aug 23, 2024
ca4d40f
Move bitsize calculation to the algorithm
encukou Aug 23, 2024
3e3eec8
finish
encukou Aug 23, 2024
3853c38
Copy the GCC/SysV algorithm to (ugly) Python code
encukou Aug 23, 2024
a8ad714
Copy the MS algorithm to (ugly) Python code
encukou Aug 23, 2024
5be7edf
Remove the C code
encukou Aug 23, 2024
7218287
Move assertion & big-endian adjustment
encukou Aug 23, 2024
addf44c
Move assertions out
encukou Aug 23, 2024
d92831a
Remove PyCField_InitFromDesc
encukou Aug 23, 2024
466d82f
Remove field_size, bitofs, offset from the pack state
encukou Aug 23, 2024
1af5db3
Remove pack from C code
encukou Aug 23, 2024
df3143d
Make _layout_ return an object with fields
encukou Aug 23, 2024
a22c557
Avoid self.* assignments
encukou Aug 23, 2024
47150d2
Pass size and align from Python
encukou Aug 23, 2024
c244045
Remove some more of the packstate
encukou Aug 23, 2024
903c0fa
Remove align from the state
encukou Aug 23, 2024
551a6ed
Compute padding in Python
encukou Aug 23, 2024
0bb7797
Move union_size to Python
encukou Aug 23, 2024
9bc45c5
Remove the C state
encukou Aug 23, 2024
7a76a8e
Set up for checking format
encukou Aug 23, 2024
cdd028c
Compute the format spec in Python
encukou Aug 23, 2024
4de515f
Fix refcounting/error handling
encukou Aug 23, 2024
adbe019
Remove the format calculations from C
encukou Aug 23, 2024
c1644b6
Return the aligned size from Python
encukou Aug 23, 2024
4c29379
layout_fields is a tuple
encukou Aug 23, 2024
c2e54a8
Borrow from the layout_fields tuple
encukou Aug 23, 2024
61a33d3
Use stuff from prop, not pair
encukou Aug 23, 2024
f3f3622
Remove use of pair
encukou Aug 23, 2024
34edfd8
Only pass fields to Python, do nothing else with it
encukou Aug 23, 2024
833edf4
Don't pass forced_align
encukou Aug 23, 2024
779fa81
Remove scaffolding members from CFieldObject
encukou Aug 23, 2024
7b3926e
Unify the bit_size variable
encukou Aug 23, 2024
5f351e7
Start simplifying the Python code
encukou Aug 23, 2024
d2f0da3
Remove state_align
encukou Aug 23, 2024
4b346d4
Make it a bit clearer yet
encukou Aug 23, 2024
95d9c54
More cleanup
encukou Aug 23, 2024
af5c52b
Simplify C call using borrowed references
encukou Aug 23, 2024
51a4042
Use _Py_ID for the literals
encukou Aug 23, 2024
5a805cd
Use floordiv, not int(truediv), for non-negative numbers
encukou Aug 26, 2024
12fd84a
Use a data class + function
encukou Aug 26, 2024
e4ad839
Warning comment
encukou Aug 26, 2024
c1ba0e8
Remove test scaffolding
encukou Aug 26, 2024
25ceb4b
CField is instantiable now
encukou Aug 26, 2024
54640c3
PEP-8
encukou Aug 26, 2024
e62bb02
Move a comment over to its code
encukou Aug 26, 2024
e0b0de0
Remove unused function
encukou Aug 26, 2024
51b7568
PEP-7, typo, align comments
encukou Aug 26, 2024
60c5557
Merge in the main branch
encukou Aug 26, 2024
1092d68
Regenerate global objects
encukou Aug 27, 2024
6820d23
Avoid BytesWarning
encukou Aug 27, 2024
12311ad
Move an assertion after a check
encukou Aug 27, 2024
4921407
Remove unused field, bit_size
encukou Aug 27, 2024
ff4bcf5
Apply suggestions from code review
encukou Aug 28, 2024
6b9a8b7
Move bit_size checks to assertions; they're checked in Python
encukou Aug 28, 2024
991bab3
Move `i` declaration to for loops
encukou Aug 28, 2024
298c822
Remove a clearly unneeded assert
encukou Aug 28, 2024
3088e5d
Add rudimentary test for bitfields in Union
encukou Aug 28, 2024
511e25d
Improve error message for _field_ element unpacking
encukou Aug 28, 2024
0689e59
Use correct length modifier
encukou Aug 28, 2024
a7c3724
Avoid a case where MSC and GCC's __attribute__((ms_struct)) disagree
encukou Aug 28, 2024
bd53c28
Adjust Windows assertions
encukou Aug 28, 2024
b387d30
Apply suggestions from code review
encukou Aug 30, 2024
9cf18cd
Remove unused last_field
encukou Aug 28, 2024
96a5c0d
Remove the bitfield-in-union test, for now
encukou Aug 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 6 additions & 4 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,6 @@ struct _Py_global_strings {
STRUCT_FOR_ID(_abc_impl)
STRUCT_FOR_ID(_abstract_)
STRUCT_FOR_ID(_active)
STRUCT_FOR_ID(_align_)
STRUCT_FOR_ID(_anonymous_)
STRUCT_FOR_ID(_argtypes_)
STRUCT_FOR_ID(_as_parameter_)
Expand All @@ -248,21 +247,18 @@ struct _Py_global_strings {
STRUCT_FOR_ID(_initializing)
STRUCT_FOR_ID(_io)
STRUCT_FOR_ID(_is_text_encoding)
STRUCT_FOR_ID(_layout_)
STRUCT_FOR_ID(_length_)
STRUCT_FOR_ID(_limbo)
STRUCT_FOR_ID(_lock_unlock_module)
STRUCT_FOR_ID(_loop)
STRUCT_FOR_ID(_needs_com_addref_)
STRUCT_FOR_ID(_only_immortal)
STRUCT_FOR_ID(_pack_)
STRUCT_FOR_ID(_restype_)
STRUCT_FOR_ID(_showwarnmsg)
STRUCT_FOR_ID(_shutdown)
STRUCT_FOR_ID(_slotnames)
STRUCT_FOR_ID(_strptime)
STRUCT_FOR_ID(_strptime_datetime)
STRUCT_FOR_ID(_swappedbytes_)
STRUCT_FOR_ID(_type_)
STRUCT_FOR_ID(_uninitialized_submodules)
STRUCT_FOR_ID(_warn_unawaited_coroutine)
Expand All @@ -276,6 +272,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(after_in_parent)
STRUCT_FOR_ID(aggregate_class)
STRUCT_FOR_ID(alias)
STRUCT_FOR_ID(align)
STRUCT_FOR_ID(allow_code)
STRUCT_FOR_ID(append)
STRUCT_FOR_ID(arg)
Expand All @@ -295,6 +292,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(before)
STRUCT_FOR_ID(big)
STRUCT_FOR_ID(binary_form)
STRUCT_FOR_ID(bit_size)
STRUCT_FOR_ID(block)
STRUCT_FOR_ID(bound)
STRUCT_FOR_ID(buffer)
Expand Down Expand Up @@ -423,6 +421,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(fd2)
STRUCT_FOR_ID(fdel)
STRUCT_FOR_ID(fget)
STRUCT_FOR_ID(fields)
STRUCT_FOR_ID(file)
STRUCT_FOR_ID(file_actions)
STRUCT_FOR_ID(filename)
Expand All @@ -439,6 +438,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(fold)
STRUCT_FOR_ID(follow_symlinks)
STRUCT_FOR_ID(format)
STRUCT_FOR_ID(format_spec)
STRUCT_FOR_ID(from_param)
STRUCT_FOR_ID(fromlist)
STRUCT_FOR_ID(fromtimestamp)
Expand Down Expand Up @@ -475,6 +475,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(importlib)
STRUCT_FOR_ID(in_fd)
STRUCT_FOR_ID(incoming)
STRUCT_FOR_ID(index)
STRUCT_FOR_ID(indexgroup)
STRUCT_FOR_ID(inf)
STRUCT_FOR_ID(infer_variance)
Expand All @@ -495,6 +496,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(intersection)
STRUCT_FOR_ID(interval)
STRUCT_FOR_ID(is_running)
STRUCT_FOR_ID(is_struct)
STRUCT_FOR_ID(isatty)
STRUCT_FOR_ID(isinstance)
STRUCT_FOR_ID(isoformat)
Expand Down
10 changes: 6 additions & 4 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 24 additions & 16 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading