-
-
Notifications
You must be signed in to change notification settings - Fork 31.8k
Pickle ignores custom getstate methods on TextIOWrapper in Python 3.12 #122559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Seems like it was added in this PR: https://github.com/python/cpython/pull/101948/files#diff-f6d1380274f6859688c821fced58fb1c829d8acad44ae252ba988e220e1080b3R3240 |
Interesting! While it makes sense that the original IOBase can not be pickled, I feel like it should be possible for subclasses overriding getstate |
So I can verify that overriding reduce and reduce_ex worked for me. I would leave the issue open for now as I do not think getstate and setstate not working is nice - especially since reduce and reduce_ex of IOBase are buried in the C code. Please feel free to correct me if you think this has been resolved. |
Pickling file objects was explicitly forbidden in bpo-10180 by adding The right way is to remove |
It turned out that this issue is a bit more complex. There are more differences between implementations. Classes are not tested separately, existing tests do not cover all cases. I am working on this problem. |
…dule about pickling In the C implementation, remove __reduce__ and __reduce_ex__ methods that always raise TypeError and restore __getstate__ methods that always raise TypeErrori. This restores fine details of the pre-3.12 behavior and unifies both implementations.
I added tests as a guard against such type of regression, but in general there are no guaranties about pickleability (especially with protocols 0 and 1). In meantime, the workaround is to reset class EncodedFile(TextIOWrapper):
...
__reduce__ = object.__reduce__
__reduce_ex__ = object.__reduce_ex__ |
…bout pickling (GH-122628) In the C implementation, remove __reduce__ and __reduce_ex__ methods that always raise TypeError and restore __getstate__ methods that always raise TypeErrori. This restores fine details of the pre-3.12 behavior and unifies both implementations.
…e io module about pickling (pythonGH-122628) In the C implementation, remove __reduce__ and __reduce_ex__ methods that always raise TypeError and restore __getstate__ methods that always raise TypeErrori. This restores fine details of the pre-3.12 behavior and unifies both implementations. (cherry picked from commit e9253eb) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
…odule about pickling (GH-122628) (GH-133381) In the C implementation, remove __reduce__ and __reduce_ex__ methods that always raise TypeError and restore __getstate__ methods that always raise TypeErrori. This restores fine details of the pre-3.12 behavior and unifies both implementations. (cherry picked from commit e9253eb)
* origin/main: (111 commits) pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385) pythongh-131178: Add tests for `ast` command-line interface (python#133329) Regenerate pcbuild.sln in Visual Studio 2022 (python#133394) pythongh-133042: disable HACL* HMAC on Emscripten (python#133064) pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387) pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946) pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388) pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830) pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368) pythongh-132457: make staticmethod and classmethod generic (python#132460) pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812) pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490) pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517) pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628) pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358) bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226) pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287) pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364) pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027) pythonGH-91048: Add utils for printing the call stack for asyncio tasks (python#133284) ...
Bug report
Bug description:
So I am not entirely sure whether this is unintended behaviour, but it is definitely a noticeable change between 3.11 and 3.12 that is rather unintuitive
This works in Python 3.11 and 3.10, but fails in 3.12 with
CPython versions tested on:
3.10, 3.11, 3.12
Operating systems tested on:
macOS
Linked PRs
The text was updated successfully, but these errors were encountered: