Skip to content

gh-112015: Implement ctypes.memoryview_at() #112018

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jan 3, 2025

Conversation

rianhunter
Copy link
Contributor

@rianhunter rianhunter commented Nov 13, 2023

ctypes currently has no way to easily create a buffer object from a pointer and a dynamic size. It is possible to create memoryview objects of array objects that refer to existing memory (e.g.
memoryview((c_byte * size).from_address(address))) but this is inefficient and the resulting memoryview object doesn't allow editing via python (invalid format errors are thrown, this is due to how ctypes objects implements the buffer protocol).

ctypes.memoryview_at() fills that gap in the API. This is similar to ffi.buffer() in the cffi module.

Fixes #112015


📚 Documentation preview 📚: https://cpython-previews--112018.org.readthedocs.build/

ctypes currently has no way to easily create a buffer object
from a pointer and a dynamic size. It is possible to create
memoryview objects of array objects (e.g.
memoryview((c_ubyte * 10)())) but this is excessively slow when
implementing a callback function in Python that is passed a
dynamic void * and a size_t.

`ctypes.buffer_at()` fills that gap in the API. This is similar
to `ffi.buffer()` in the cffi module.
@rianhunter rianhunter changed the title Implement ctypes.buffer_at() gh-112015: Implement ctypes.buffer_at() Nov 13, 2023
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests and What's New entry.

And I think that it is worth to support large buffers.

rianhunter and others added 6 commits January 18, 2024 14:14
Documentation improvements

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
It's a more descriptive and precise name.
It's more common that the user will want a memoryview object that can
be mutated. An immutable object is more rare, so make the default case
return a mutable object.
@rianhunter
Copy link
Contributor Author

When merging, just flatten all the commits down to a single commit.

@rianhunter rianhunter changed the title gh-112015: Implement ctypes.buffer_at() gh-112015: Implement ctypes.memoryview_at() Jan 19, 2024
@rianhunter
Copy link
Contributor Author

Abandoned

@rianhunter rianhunter closed this Sep 19, 2024
@encukou
Copy link
Member

encukou commented Sep 26, 2024

I'm sorry but I no longer have time to work on this patch.

No worries. Thank you for your work so far!
I hope you won't mind if I take over and finish it.

@encukou encukou reopened this Sep 26, 2024
@rianhunter
Copy link
Contributor Author

No problem, feel free

@encukou
Copy link
Member

encukou commented Nov 29, 2024

OK. I went back to the string_at mechanism of calling the C code via ctypes, as the conversion to c_void_p is much more flexible.

@encukou
Copy link
Member

encukou commented Dec 16, 2024

Does this look interesting to you, @ZeroIntensity or @picnixz?

@picnixz picnixz self-requested a review December 16, 2024 17:25
@ZeroIntensity
Copy link
Member

I'll review sometime soon (today or in the next few days).

encukou and others added 3 commits December 19, 2024 14:33
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this!

@encukou encukou merged commit b4f799b into python:main Jan 3, 2025
45 checks passed
WolframAlph pushed a commit to WolframAlph/cpython that referenced this pull request Jan 4, 2025
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide ctypes.memoryview_at()
5 participants