Skip to content

gh-112087: Store memory allocation information into _PyListArray #116529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Mar 9, 2024
Merged

gh-112087: Store memory allocation information into _PyListArray #116529

merged 9 commits into from
Mar 9, 2024

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Mar 9, 2024 •
edited by bedevere-app bot
Loading

@corona10 corona10 requested a review from colesbury March 9, 2024 02:11
@bedevere-app bedevere-app bot mentioned this pull request Mar 9, 2024
Closed
4 tasks
corona10
corona10 commented Mar 9, 2024
View reviewed changes
Objects/listobject.c Outdated
PyObject **old_items = self->ob_item;
if (self->ob_item) {
if (allocated < new_allocated) {
memcpy(&array->ob_item, self->ob_item, allocated * sizeof(PyObject*));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching shrink case, @colesbury

@corona10 corona10 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 9, 2024
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @corona10 for commit b98f853 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 9, 2024
@corona10 corona10 linked an issue Mar 9, 2024 that may be closed by this pull request
Make list objects thread-safe in --disable-gil builds #112087
Closed
4 tasks
@corona10 corona10 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 9, 2024
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @corona10 for commit d1ac2d4 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 9, 2024
@corona10 corona10 added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @corona10 for commit d1ac2d4 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
corona10
corona10 commented Mar 9, 2024
View reviewed changes
Misc/NEWS.d/next/Core and Builtins/2024-03-09-11-10-53.gh-issue-112087.nbI0Pw.rst
@@ -0,0 +1 @@
:class:`list` is now compatible with the implementation of :pep:`703`.
Copy link
Member Author

@corona10 corona10 Mar 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should be the final PR for #112087.
I think that we can close the issue, and after GIL is disabled, we can track relevant issues from a separate issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing I noticed here is that list.sort should have a critical section

@corona10
Copy link
Member Author

corona10 commented Mar 9, 2024

PPC related failure: Not related to this PR.

/tmp/ccfdq0MP.s: Fatal error: can't write 821 bytes to section .gnu.lto_ucs4lib_rfind.672.76bf242dbd85e46 of Objects/unicodeobject.o: 'No space left on device'
/tmp/ccfdq0MP.s: Fatal error: Objects/unicodeobject.o: No such file or directory
make: *** [Makefile:3000: Objects/unicodeobject.o] Error 1

@corona10 corona10 added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @corona10 for commit 3b7d24e 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
@corona10 corona10 added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @corona10 for commit eb277c4 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 9, 2024
corona10
corona10 commented Mar 9, 2024
View reviewed changes
colesbury
colesbury reviewed Mar 9, 2024
View reviewed changes
@corona10 corona10 requested a review from colesbury March 9, 2024 20:42
colesbury
colesbury reviewed Mar 9, 2024
View reviewed changes
Objects/listobject.c Outdated
#ifdef Py_GIL_DISABLED
typedef struct {
Py_ssize_t allocated;
PyObject *ob_item;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be a flexible array member like:

Suggested change
PyObject *ob_item;
PyObject *ob_item[];

Like dk_indices in PyDictKeysObject

cpython/Include/internal/pycore_dict.h

Line 167 in 1e68c4b

char dk_indices[]; /* char is required to avoid strict aliasing. */

Misc/NEWS.d/next/Core and Builtins/2024-03-09-11-10-53.gh-issue-112087.nbI0Pw.rst
@@ -0,0 +1 @@
:class:`list` is now compatible with the implementation of :pep:`703`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing I noticed here is that list.sort should have a critical section

@corona10 corona10 removed a link to an issue Mar 9, 2024
Make list objects thread-safe in --disable-gil builds #112087
Closed
4 tasks
@corona10 corona10 enabled auto-merge (squash) March 9, 2024 23:33
@corona10
Copy link
Member Author

corona10 commented Mar 9, 2024

One more thing I noticed here is that list.sort should have a critical section

I will submit the PR right a way.

@corona10 corona10 merged commit 17d31bf into python:main Mar 9, 2024
@corona10 corona10 deleted the gh-112087-allocation branch March 9, 2024 23:53
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
@corona10 @adorilson
pythongh-112087: Store memory allocation information into _PyListArray (
7f096bc 
pythongh-116529)
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@corona10 @diegorusso
pythongh-112087: Store memory allocation information into _PyListArray (
6429975 
pythongh-116529)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@colesbury colesbury colesbury approved these changes

Assignees
No one assigned
Labels
topic-free-threading
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@corona10 @bedevere-bot @colesbury