Skip to content

[Bug]: ValueError upon deepcopy of a Figure object #21554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tovrstra opened this issue Nov 6, 2021 · 11 comments · Fixed by #21570 or #21597
Closed

[Bug]: ValueError upon deepcopy of a Figure object #21554

tovrstra opened this issue Nov 6, 2021 · 11 comments · Fixed by #21570 or #21597
Milestone

Comments

@tovrstra
Copy link

tovrstra commented Nov 6, 2021

Bug summary

A deepcopy of a Figure object results in ValueError: 'Spines' object does not contain a '__deepcopy__' spine.

This issue surfaced in a bug report of schemdraw: https://bitbucket.org/cdelker/schemdraw/issues/56/copydeepcopy-drawing-fails Nevertheless, the current issue is unrelated to schemdraw. I'm just mentioning this to illustrate a relevant use case leading to this problem.

Code for reproduction

import matplotlib.pyplot as plt
import copy
fig, ax = plt.subplots()
copy.deepcopy(fig)

Disclaimer: this is taken from a message of cdelker on https://bitbucket.org/cdelker/schemdraw/issues/56/copydeepcopy-drawing-fails

Actual outcome

Traceback (most recent call last):
  File "/usr/lib64/python3.9/site-packages/matplotlib/spines.py", line 551, in __getattr__
    return self._dict[name]
KeyError: '__deepcopy__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib64/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib64/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib64/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib64/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.9/copy.py", line 205, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/usr/lib64/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib64/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib64/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib64/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib64/python3.9/copy.py", line 151, in deepcopy
    copier = getattr(x, "__deepcopy__", None)
  File "/usr/lib64/python3.9/site-packages/matplotlib/spines.py", line 553, in __getattr__
    raise ValueError(
ValueError: 'Spines' object does not contain a '__deepcopy__' spine

Expected outcome

Either a deepcopy of the figure or a meaningful error message explaining that this operation is not supported.

Operating system

Fedora Linux

Matplotlib Version

3.4.3

Matplotlib Backend

Qt5Agg

Python version

3.9.7

Jupyter version

6.1.6, (not installed)

Other libraries

None

Installation

Linux package manager (Debian/Fedora/etc.)

Conda channel

No response

@anntzer anntzer added this to the v3.5.0 milestone Nov 6, 2021
@anntzer
Copy link
Contributor

anntzer commented Nov 6, 2021

My (unverified) guess is that this is a regression due to #17107 and https://bugs.python.org/issue19364), hence milestoning as 3.5.

@dstansby
Copy link
Member

dstansby commented Nov 6, 2021

In 3.3.4 (ie. before #17107) this still errors, but with a different message:

NotImplementedError: TransformNode instances can not be copied. Consider using frozen() instead.

@tovrstra do you know if this has ever worked in Matplotlib?

@tovrstra
Copy link
Author

tovrstra commented Nov 6, 2021

I only ran into this recently. (beginning of October this year) I'm not sure what would have happened with earlier versions of matplotlib.

I'm also wondering if the operation should be supported. Would the deepcopy result in two different Figure instances with the same figure number? I'm not sure if that would make sense.

@anntzer
Copy link
Contributor

anntzer commented Nov 6, 2021

Good catch @dstansby; I guess this may have never worked, sorry for the wrong alert.

@anntzer anntzer removed this from the v3.5.0 milestone Nov 6, 2021
QuLogic added a commit to QuLogic/matplotlib that referenced this issue Nov 9, 2021
If the name does not exist, then `__getattr__` should raise
`AttributeError`, not `ValueError`.

Fixes matplotlib#21554
@timhoffm
Copy link
Member

timhoffm commented Nov 9, 2021

In 3.3.4 (ie. before #17107) this still errors, but with a different message:

NotImplementedError: TransformNode instances can not be copied. Consider using frozen() instead.

I had 3.1.3 around and there it's the same error.

Should we explicitly forbid deepcopying a figure by overriding Figure.__deepcopy__. It seems to never have worked and being explicit about it doesn't hurt.

@timhoffm
Copy link
Member

timhoffm commented Nov 9, 2021

I don't think #21570 does close this. It issues a more correct error, but deepcopy(figure) should not fail in some random internal place. Let's make it fail expicitly with NotImplementedError: Figure does not support deepcopy.

@timhoffm timhoffm reopened this Nov 9, 2021
@tovrstra
Copy link
Author

tovrstra commented Nov 9, 2021

@timhoffm That would indeed clarify the error indeed.

I'm trying to understand if this would be the right behavior. For example, the following does work, which is almost the same as a deepcopy:

import pickle
import matplotlib.pyplot as plt
fig1, ax = plt.subplots()
print(fig1.number)
fig2 = pickle.loads(pickle.dumps(fig1))
print(fig2.number)

It generates the following output:

1
2

One may wonder, if pickle works, why deepcopy does not. (I don't know, but that doesn't mean there cannot be a good reason, obviously. I'm just saying I'm a bit confused.)

@QuLogic
Copy link
Member

QuLogic commented Nov 9, 2021

I don't understand what you mean here. With the attached PR, the code in the OP works on both Python 3.7 and 3.10.

@greglucas
Copy link
Contributor

Oops, I think that was my bad in the comment on your PR! This indeed allows deepcopy to work without error, but it does not seem to "work" as expected, so I would say the NotImplementedError on deepcopy would still make sense here.

Here, I get two figures, but the second one is missing some spines and the axes plot data.

import matplotlib.pyplot as plt
import copy
fig, ax = plt.subplots()

ax.plot([0, 1], [2, 3])

fig2 = copy.deepcopy(fig)
plt.show()

@QuLogic
Copy link
Member

QuLogic commented Nov 10, 2021

Ah, I see yes.

It's strange though that pickling seems to work since the copy docs say 'Classes can use the same interfaces to control copying that they use to control pickling.'

@tacaswell
Copy link
Member

I suspect that at a conceptual level, you are very correct that if pickle/unpickle works than deepcopy should also work (and vis versa), but at the object protocol level, Python gives us different magic methods for the two cases (which makes sense, pickle is a serialization protocol for sending data structures to a different process over the wire, deepcopy is for making a copy within the same process. If you can easily imagine that there are more efficient ways of doing the in-process copy than fully serializing for the wire). I think the current issue is exposing Matplotlib gets the pickling code path correct (and we test it) but does not get the deepcopy codepath right (yet). Chasing through getting deepcopy to be correct (the semantics are a bit subtle) will take an indeterminate time, where has intercepting the call on Figure an raising NotImplemented will be very easy ;) .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants