Skip to content

gh-129107: make bytearrayiter free-threading safe #130096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Feb 19, 2025

Conversation

tom-pytel
Copy link
Contributor

@tom-pytel tom-pytel commented Feb 13, 2025

This is the bytearray iterator made safe under free-threading. The behavior is copied from the list iterator, it is not synchronized but it won't crash.

@tom-pytel
Copy link
Contributor Author

This PR is split off of #129108. Ping @kumaraditya303

@tom-pytel
Copy link
Contributor Author

tom-pytel commented Feb 13, 2025

EDIT: Removed, unnecessary critical sections in bytearrayiter_next and bytearrayiter_reduce. Really no point having them in this PR, will add them back in bytearray PR after this one is merged.

@kumaraditya303 kumaraditya303 self-requested a review February 14, 2025 16:41
@tom-pytel
Copy link
Contributor Author

@kumaraditya303 I thought you would do the iterator first and backport it so removed an unnecessary critical section here if bytearray not safe itself. If you don't intend to backport this then I will just add it back here, but otherwise it doesn't make sense (though harmless) unless you backport bytearray as well (which I am not suggesting). For reference, the critical section is:

static PyObject *
bytearrayiter_next(PyObject *self)
{
    bytesiterobject *it = _bytesiterobject_CAST(self);
    int val;

    assert(it != NULL);
    Py_ssize_t index = FT_ATOMIC_LOAD_SSIZE_RELAXED(it->it_index);
    if (index < 0) {
        return NULL;
    }
    PyByteArrayObject *seq = it->it_seq;
    assert(PyByteArray_Check(seq));

    Py_BEGIN_CRITICAL_SECTION(seq);
    if (index < PyByteArray_GET_SIZE(seq)) {
        val = (unsigned char)PyByteArray_AS_STRING(seq)[index];
    }
    else {
        val = -1;
    }
    Py_END_CRITICAL_SECTION();

    if (val == -1) {
        FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, -1);
#ifndef Py_GIL_DISABLED
        it->it_seq = NULL;
        Py_DECREF(seq);
#endif
        return NULL;
    }
    FT_ATOMIC_STORE_SSIZE_RELAXED(it->it_index, index + 1);
    return _PyLong_FromUnsignedChar((unsigned char)val);
}

Though can also just use bytearray_getitem here, but I don't like calling an unnecessary function where can just read a byte instead.

@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Feb 15, 2025

I don't intend to backport the thread safety of bytearray as change is large and there isn't much point in backporting the iterator while bytearray isn't thread safe.

@colesbury Do you think we should backport the thread safety of bytearray?

Yes, it is not safe to read data without acquiring critical section of the seq, you need to add back the critical section, the change to use atomic makes sense.

@colesbury
Copy link
Contributor

I probably wouldn't backport the changes.

…e-129107._olg-L.rst

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Copy link
Contributor

@kumaraditya303 kumaraditya303 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I would like a review from @colesbury as well.

@kumaraditya303 kumaraditya303 merged commit 1b6bef8 into python:main Feb 19, 2025
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants