Skip to content

Allow setting the memory alignment of raw memory from Python #112448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
bertsch-bf opened this issue Nov 27, 2023 · 7 comments
Open

Allow setting the memory alignment of raw memory from Python #112448

bertsch-bf opened this issue Nov 27, 2023 · 7 comments
Labels
topic-ctypes type-feature A feature request or enhancement

Comments

@bertsch-bf
Copy link

bertsch-bf commented Nov 27, 2023 •
edited by github-actions bot
Loading

Feature or enhancement

Proposal:

Introduction

Memory buffers used by native libraries often come with alignment requirements, such as page alignment. If an API dealing with such buffers is exposed into Python user code, the Python side is now responsible for providing a suitably aligned memory buffer. However, there is no way of properly specifying the alignment of any object supporting the buffer protocol, neither high-level ones like bytes and bytearray, nor lower-level ones like ctypes arrays and mmap.

Over 10 years ago, a StackOverflow question has been asked for this exact problem, and one answer provides the most compact known workaround available to many Python versions. An additional problem it needs to consider here is that accessing the memory address of a Python buffer is nontrivial.

size = getBufferSizeFromSomewhere()
alignment = getBufferAlignmentFromSomewhere()

import ctypes
# Maximum amount of extra memory required - we can't know exactly before allocating!
requiredOversize = size + alignment - 1
oversizeBuffer = bytearray(requiredOversize)
# Get the address of the bytearray's backing buffer
oversizeCType = ctypes.c_char * requiredOversize
oversizeMemory = oversizeCType.from_buffer(oversizeBuffer)
oversizeAddress = ctypes.addressof(oversizeMemory)
# Calculate required offset into oversized buffer to reach proper alignment
offsetToAligned = alignment - oversizeAddress % alignment
# Create a raw C buffer with offset to not copy the byte array in the process
bufferCType = ctypes.c_char * (requiredOversize - offsetToAligned)
correctRawCMemory = bufferCType.from_buffer(oversizeMemory, offsetToAligned)

# Now use correctRawCMemory...

The C API gained PyMem_AlignedAlloc() in 3.7 and numpy also had a feature request accepted for aligned arrays.

Proposal

Note that I would like to find a better place for an aligned memory API, I just couldn't find one so far. See "Alternatives" below.

On all platforms supported by the mmap module, add a new parameter align to the mmap.mmap function.

# Windows
mmap.mmap(fileno, length, tagname=None, access=ACCESS_DEFAULT, align=1[, offset])
# Unix
mmap.mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT, align=1[, offset])

If the mapping is a named file mapping (positive file descriptor number), the file's contents are mapped into memory starting from an address that is a multiple of align. If the mapping is an anonymous mapping (file descriptor -1), memory is allocated at an address that is a multiple of align. The default of 1 guarantees backwards compatibility, since it allows mapping to any address as before. The offset parameter only affects the start offset into the source data and has no effect on the starting address of the mapping.

Alternatives

Add memory alignment control to ctypes instead of mmap

The only place where I can see this would fit is ctypes._CData.from_buffer_copy. Since ctypes does not provide a direct allocation API, the buffer needs to be allocated twice: once by the buffer provider (e.g. bytearray), and once for copying by ctypes. It also seems like a less convenient and obvious API than the other proposals

Add memory alignment control to bytearray

It is debatable whether adding such a low-level control feature to a high-level API like bytearray is a good design. However, this would make the feature available on Emscripten and WASI.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response
@bertsch-bf bertsch-bf added the type-feature A feature request or enhancement label Nov 27, 2023
@bertsch-bf bertsch-bf changed the title Allow setting the memory alignment of raw memory Allow setting the memory alignment of raw memory from Python Nov 27, 2023
@ronaldoussoren
Copy link
Contributor

Note that mmap already returns page aligned data.

@monkeyman192
Copy link
Contributor

This seems closely related to my proposal here also: #112433

@bertsch-bf
Copy link
Author

Note that mmap already returns page aligned data.

@ronaldoussoren Good to know, but since this is not documented as an API guarantee and not even POSIX guarantees this ("If MAP_FIXED is specified, the implementation may require that addr is a multiple of the page size."), users can't rely on it.

This seems closely related to my proposal here also: #112433

@monkeyman192 I have seen this proposal and consider it useful on its own terms. However, in my opinion it solves a slightly different problem and either solution is going to look ugly for the opposite problem:

  • I need to work with the buffer API, not direct FFI. (We have a binding layer that handles the translation from and to the buffer API, and the usage should be as close to normal high-level Python code as feasible.) All the extra structure definition work seems like a lot of boilerplate for this.
  • Using ad-hoc classes in deeply nested code to handle the dynamic alignment seems severely un-pythonic. Your proposal solves statically-known alignment requirements.

@ronaldoussoren
Copy link
Contributor

Note that mmap already returns page aligned data.

@ronaldoussoren Good to know, but since this is not documented as an API guarantee and not even POSIX guarantees this ("If MAP_FIXED is specified, the implementation may require that addr is a multiple of the page size."), users can't rely on it.

I've yet to see an operating system where mmap(2) does not return a page aligned pointer. This might not be guaranteed by a standard, but is pretty much a side effect of how virtual memory works on modern processors.

@bertsch-bf
Copy link
Author

Note that mmap already returns page aligned data.

@ronaldoussoren Good to know, but since this is not documented as an API guarantee and not even POSIX guarantees this ("If MAP_FIXED is specified, the implementation may require that addr is a multiple of the page size."), users can't rely on it.

I've yet to see an operating system where mmap(2) does not return a page aligned pointer. This might not be guaranteed by a standard, but is pretty much a side effect of how virtual memory works on modern processors.

We've yet to see whether a limited implementation of mmap will appear on Wasm platforms, where many assumptions of native platforms go out the window :^). Either way, this proposal is for any alignment, including relatively small alignments like 4 or 8 bytes for storing 32-bit or 64-bit words, or larger alignments when very large buffer sizes and specific driver requirements are at play.

@ronaldoussoren
Copy link
Contributor

One reason for not adding an align option to mmap.mmap would be that the underlying API (mmap(2)) doesn't support this other than by using MAP_FIXED.

That said, there might be other ways to get similar behaviour such as using an alignment argument to bytearray and/or array.array. That likely requires some plumbing and/or accepting higher memory usage because AFAIK our current allocator doesn't allow specifying an alignment (I couldn't find PyMem_AllignedAlloc in the source tree).

@bertsch-bf
Copy link
Author

That said, there might be other ways to get similar behaviour such as using an alignment argument to bytearray and/or array.array.

I'm happy to find a better place to fit this API, I just felt it was too low-level for bytearray.

That likely requires some plumbing and/or accepting higher memory usage because AFAIK our current allocator doesn't allow specifying an alignment (I couldn't find PyMem_AllignedAlloc in the source tree).

My mistake, I thought the feature was merged but looking into the migrated GitHub issue and linked pull requests it turns out that it was indeed rejected. Maybe this is enough reason to revive the C API? It should also come in handy for #112433, since creating an aligned structure must create it at a suitably aligned memory address, thereby requiring an aligned allocation API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic-ctypes type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ronaldoussoren @serhiy-storchaka @monkeyman192 @bertsch-bf