Skip to content

add arrayiter.__reduce__ #3868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 18, 2022
Merged

add arrayiter.__reduce__ #3868

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0f24b18
Add arrayiter __reduce__
maong0927 Jul 11, 2022
ab284c8
update test_iterator_pickle
maong0927 Jul 11, 2022
821f3e7
fix position value to struct value
maong0927 Jul 12, 2022
1a526af
Fix: Modify iter next to be implemented only internal
Yaminyam Jul 16, 2022
50aebef
Update stdlib/src/array.rs
Yaminyam Jul 16, 2022
2417d3e
Feat: impl `PyArrayIter.setstate`
Yaminyam Jul 16, 2022
cdd937c
Refactor: convert indent from tab to space
Yaminyam Jul 16, 2022
2bf61aa
Refactor: delete unnecessary clones
Yaminyam Jul 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions Lib/test/test_array.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,8 +284,6 @@ def test_pickle_for_empty_array(self):
self.assertEqual(a.x, b.x)
self.assertEqual(type(a), type(b))

# TODO: RUSTPYTHON
@unittest.expectedFailure
def test_iterator_pickle(self):
orig = array.array(self.typecode, self.example)
data = list(orig)
Expand Down
53 changes: 32 additions & 21 deletions stdlib/src/array.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ mod array {
common::{
atomic::{self, AtomicUsize},
lock::{
PyMappedRwLockReadGuard, PyMappedRwLockWriteGuard, PyRwLock, PyRwLockReadGuard,
PyRwLockWriteGuard,
PyMappedRwLockReadGuard, PyMappedRwLockWriteGuard, PyMutex, PyRwLock,
PyRwLockReadGuard, PyRwLockWriteGuard,
},
str::wchar_t,
},
vm::{
builtins::{
PyByteArray, PyBytes, PyBytesRef, PyDictRef, PyFloat, PyInt, PyIntRef, PyList,
PyListRef, PyStr, PyStrRef, PyTupleRef, PyTypeRef,
PositionIterInternal, PyByteArray, PyBytes, PyBytesRef, PyDictRef, PyFloat, PyInt,
PyIntRef, PyList, PyListRef, PyStr, PyStrRef, PyTupleRef, PyTypeRef,
},
class_or_notimplemented,
convert::{ToPyObject, ToPyResult, TryFromObject},
Expand Down Expand Up @@ -1258,9 +1258,8 @@ mod array {

impl Iterable for PyArray {
fn iter(zelf: PyRef<Self>, vm: &VirtualMachine) -> PyResult {
Ok(PyArrayIterator {
position: AtomicUsize::new(0),
array: zelf,
Ok(PyArrayIter {
internal: PyMutex::new(PositionIterInternal::new(zelf, 0)),
}
.into_pyobject(vm))
}
Expand All @@ -1278,24 +1277,36 @@ mod array {
#[pyattr]
#[pyclass(name = "arrayiterator")]
#[derive(Debug, PyPayload)]
pub struct PyArrayIterator {
position: AtomicUsize,
array: PyArrayRef,
pub struct PyArrayIter {
internal: PyMutex<PositionIterInternal<PyArrayRef>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you really need both position and internal? Though I didn't look in details, those fields look like duplication.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because internal contains position because it is PositionIterInternal as its name suggests?

Copy link
Member

@youknowone youknowone Jul 13, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the PositionIterInternal type. it contains position field. By this change, the code have duplicated position fields but most of code uses position and new code uses internal. I don't think it can correctly work

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With comparison with other types' iterator, we really don't need position and array field because PyMutex<PositionIterInternal<PyArrayRef>> will do their roles. Because of __next__ and __setstate__, we need to modify position and array fields. It leads to the need for PyMutex.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we replace position and array fields with PyMutex<PositionIterInternal<PyArrayRef>>, we can implement other methods as usual we did (__next__ and __reduce__)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does arrayiterator require closed field? I thought it only need position.
closed is useful when it include another iterator in it.

}

#[pyimpl(with(IterNext))]
impl PyArrayIterator {}
#[pyimpl(with(IterNext), flags(HAS_DICT))]
impl PyArrayIter {
#[pymethod(magic)]
fn setstate(&self, state: PyObjectRef, vm: &VirtualMachine) -> PyResult<()> {
self.internal
.lock()
.set_state(state, |obj, pos| pos.min(obj.len()), vm)
}

impl IterNextIterable for PyArrayIterator {}
impl IterNext for PyArrayIterator {
#[pymethod(magic)]
fn reduce(&self, vm: &VirtualMachine) -> PyTupleRef {
self.internal
.lock()
.builtins_iter_reduce(|x| x.clone().into(), vm)
}
}

impl IterNextIterable for PyArrayIter {}
impl IterNext for PyArrayIter {
fn next(zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<PyIterReturn> {
let pos = zelf.position.fetch_add(1, atomic::Ordering::SeqCst);
let r = if let Some(item) = zelf.array.read().get(pos, vm) {
PyIterReturn::Return(item?)
} else {
PyIterReturn::StopIteration(None)
};
Ok(r)
zelf.internal.lock().next(|array, pos| {
Ok(match array.read().get(pos, vm) {
Some(item) => PyIterReturn::Return(item?),
None => PyIterReturn::StopIteration(None),
})
})
}
}

Expand Down