Skip to content

Chain reduce #4232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 21, 2022
Merged

Chain reduce #4232

merged 13 commits into from
Oct 21, 2022

Conversation

dannasman
Copy link
Contributor

@dannasman dannasman commented Oct 18, 2022

Hello!

I tried to implement the reduce method for PyItertoolsChain (issue #3611) but got stuck. I used the CPython implementation as a reference. The implementation seems to work just fine with simple arrays etc. but fails the test_chain_reducible test. Any suggestions on why?

The error message:

ERROR: test_chain_reducible (test.test_itertools.TestBasicOps)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dan/Projects/RustPython/pylib/Lib/test/test_itertools.py", line 187, in test_chain_reducible
    self.assertEqual(list(oper(it)), list('abcdef'))
  File "/home/dan/Projects/RustPython/pylib/Lib/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/dan/Projects/RustPython/pylib/Lib/copy.py", line 279, in _reconstruct
    y.__dict__.update(state)
AttributeError: 'chain' object has no attribute '__dict__'

----------------------------------------------------------------------

@dannasman dannasman marked this pull request as ready for review October 18, 2022 11:31
@dannasman dannasman marked this pull request as draft October 18, 2022 11:32
@youknowone
Copy link
Member

Search for HAS_DICT in the same file and add it to Chain's flags annotation.

@dannasman
Copy link
Contributor Author

I added HAS_DICT and now the problem seems to be that when a deepcopy operation is performed on a chain object it appears empty. I was thinking that it might iterate through the values somewhere where it shouldn't and at the same time "clears" them.

How it works with Python 3:

>>> import copy
>>> from itertools import chain
>>> oper = copy.deepcopy
>>> it = chain('abc', 'def')
>>> res = oper(it)
>>> list(res)
['a', 'b', 'c', 'd', 'e', 'f']
>>> 

How it works with this implementation:

>>>>> import copy
>>>>> from itertools import chain
>>>>> oper = copy.deepcopy
>>>>> it = chain('abc', 'def')
>>>>> res = oper(it)
>>>>> list(res)
[]
>>>>> 

@dannasman dannasman marked this pull request as ready for review October 19, 2022 13:28
@dannasman
Copy link
Contributor Author

dannasman commented Oct 19, 2022

The solution was to implement setstate for chain. This will also solve #3779 I think.

@fanninpm
Copy link
Contributor

@Snowapril you may be interested in this

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I left a few suggestions about style and locks.

Comment on lines 68 to 76
match source {
Some(source) => match active {
Some(active) => {
Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone(), (source, active))))
}
None => Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone(), (source,)))),
},
None => Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone()))),
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match source {
Some(source) => match active {
Some(active) => {
Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone(), (source, active))))
}
None => Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone(), (source,)))),
},
None => Ok(vm.new_tuple((cls, vm.ctx.empty_tuple.clone()))),
}
let empty_tuple = vm.ctx.empty_tuple.clone();
let reduced = match source {
Some(source) => match active {
Some(active) => vm.new_tuple((cls, empty_tuple, (source, active))),
None => vm.new_tuple((cls, empty_tuple, (source,))),
},
None => vm.new_tuple((cls, empty_tuple)),
};
Ok(reduced)

Comment on lines 103 to 104
*zelf.source.write() = source.to_owned().try_into_value(vm)?;
*zelf.active.write() = active.to_owned().try_into_value(vm)?;
Copy link
Member

@youknowone youknowone Oct 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid unpleasure broken synchronization, locking both and writing will be better.

Suggested change
*zelf.source.write() = source.to_owned().try_into_value(vm)?;
*zelf.active.write() = active.to_owned().try_into_value(vm)?;
let mut source_lock = zelf.source.write();
let mut active_lock = zelf.active.write();
*source_lock = source.to_owned().try_into_value(vm)?;
*active_lock = active.to_owned().try_into_value(vm)?;

@dannasman dannasman requested a review from youknowone October 21, 2022 05:51
@youknowone youknowone merged commit 78586f0 into RustPython:main Oct 21, 2022
@youknowone
Copy link
Member

Thank you for contributing!

@dannasman dannasman deleted the chain_reduce branch October 21, 2022 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants