I haven't looked at the Grouper much in detail, but would it be possible to use a weakkey dictionary here? Or a weakset for the values?

I didn't check in detail, but weakref.py doesn't seem to make any specific allowance for pickling, so probably something like this PR would remain needed. I guess you could open a separate issue to track the usefulness of switching the implementation to weakkeydicts and weaksets, though.

I agree you still need to go back and forth between weak and strong during pickling, but I'm a bit confused why you're doing it in Figure's getstate, rather than adding getstate/setstate to the Grouper class itself?

Here was a quick attempt at hacking it in with the WeakKeyDictionary, but now that I think about it, we might actually need the weakset as well here (I overlooked that there was a cycle from key: list(key) in the key-value pairs)
main...greglucas:matplotlib:grouper-pickle

But I think that shows you could probably move your current implementation into getstate/setstate and it would be a little easier to follow all in one place (and allow other Grouper's to take advantage of it).

Ah, yes. I think I was slightly confused because I also wrote the pickling of shared axes (via _AxesBase._shared_axes) and that one needs specific support (both because the Grouper is (effectively) a global living at the class level and thus doesn't get picked up by pickling, and also because even if it did you wouldn't want to pickle the entire Grouper (and thus all currently existing shared axes) but only the connected components of the axes currently being pickled). But here _align_label_groups is per-figure, so I guess just adding pickling support directly to Grouper is OK (and doesn't harm the above case.

Do you want to take over the issue and push your patch instead? :) [to be clear, the PR here is still correct AFAICT, but @greglucas' solution is more general]

It will probably be a bit before I could get around to adding tests and thinking more about the proper weak-reference counting and refactoring that area properly, so I say we push forward here for now and leave that as a follow-up PR. Are you willing to move your weak/strong transitions over into Grouper's getstate/setstate instead of the classmethod and figure updates? I think this is good to go then.

Sure, done.