Skip to content

Update Python libraries and tests from Python 3.13 #5529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
youknowone opened this issue Feb 20, 2025 · 21 comments
Open

Update Python libraries and tests from Python 3.13 #5529

youknowone opened this issue Feb 20, 2025 · 21 comments
Labels
A-stdlib C-compat A discrepancy between RustPython and CPython C-enhancement New feature or request cpython CPython source code related good first issue Good for newcomers

Comments

@youknowone
Copy link
Member

youknowone commented Feb 20, 2025

Though it will be sometimes blocked by rust side issues, don't worry!
Please just create PR about the broken test and and create issues related to the new bugs.

If you are new to updating cpython libraries, this guide will be useful:
https://github.com/RustPython/RustPython/wiki/How-to-update-test-files

When updating module Lib/X, please don't forget to also update Lib/test/test_X

A few libraries doesn't need to be updated because CPython also kept it same as previous version. In that case, please leave a comment to let us mark it as done. Thank you!

Tips

  • To get specific version of CPython source code, e.g. try git checkout v3.13.1 from CPython repository.
  • Please check Update libraries and tests from CPython 3.10.6 #4064 for example. Not the entire changes but the each commits. One of the middle size example is 17e12de . Sometimes it can be really simple like 9571a68
  • Please create one PR for one library+test pair whenever possible. Once you stuck by dependencies, go to one of the dependency first and finish it before restarting the stuck one unless they become simpler to be updated at the same time. Normally adding more libraries make passing tests (a lot) harder.
  • To run tests, try cargo run --features ssl -- -m test -v <test_name>. e.g. If you changed test_unicode.py, cargo run --features ssl -- -m test -v test_unicode
    • If it runs too slow, try to add --release like cargo run --release --features ssl -- -m test -v test_unicode

List of libraries

List of no-update expected libraries

  • __hello__.py unchanged
  • __phello__
  • antigravity.py not changed
  • this.py

List of tests without python libraries

  • test/test___all__.py
  • test/test__locale.py
  • test/test__opcode.py
  • test/test_abstract_numbers.py
  • test/test_array.py
  • test/test_asdl_parser.py
  • test/test_asyncgen.py
  • test/test_atexit.py
  • test/test_audioop.py removed in 3.13
  • test/test_audit.py Added test_audit from 3.13.2 #5637
  • test/test_augassign.py unchanged
  • test/test_baseexception.py
  • test/test_bigaddrspace.py
  • test/test_bigmem.py unchanged
  • test/test_binascii.py unchanged
  • test/test_binop.py
  • test/test_bool.py unchanged
  • test/test_buffer.py
  • test/test_bufio.py
  • test/test_builtin.py
  • test/test_bytes.py
  • test/test_c_locale_coercion.py
  • test/test_call.py
  • test/test_charmapcodec.py
  • test/test_check_c_globals.py
  • test/test_class.py
  • test/test_clinic.py
  • test/test_cmath.py
  • test/test_cmd_line.py
  • test/test_cmd_line_script.py
  • test/test_codeccallbacks.py
  • test/test_compare.py
  • test/test_compile.py
  • test/test_complex.py
  • test/test_contains.py
  • test/test_context.py
  • test/test_contextlib_async.py
  • test/test_coroutines.py
  • test/test_crashers.py Update test_crashers from Cpython v3.11.2 #4807 CPython specific
  • test/test_csv.py
  • test/test_decorators.py
  • test/test_defaultdict.py
  • test/test_deque.py
  • test/test_descr.py
  • test/test_descrtut.py
  • test/test_devpoll.py
  • test/test_dict.py
  • test/test_dict_version.py Update test_dict_version.py from Cpython v3.11.2 #4809 Not implemented in RustPython
  • test/test_dictcomps.py
  • test/test_dictviews.py
  • test/test_docxmlrpc.py
  • test/test_dtrace.py
  • test/test_dynamic.py
  • test/test_dynamicclassattribute.py
  • test/test_eintr.py
  • test/test_embed.py
  • test/test_enumerate.py
  • test/test_eof.py
  • test/test_epoll.py
  • test/test_errno.py
  • test/test_except_star.py
  • test/test_exception_group.py
  • test/test_exception_hierarchy.py
  • test/test_exception_variations.py
  • test/test_exceptions.py
  • test/test_extcall.py
  • test/test_faulthandler.py
  • test/test_fcntl.py
  • test/test_file.py
  • test/test_file_eintr.py
  • test/test_fileio.py
  • test/test_fileutils.py
  • test/test_finalization.py
  • test/test_float.py
  • test/test_flufl.py Add test_flufl.py from Cpython v3.11.2 #4824 CPython specific
  • test/test_fork1.py
  • test/test_format.py
  • test/test_frame.py
  • test/test_frozen.py
  • test/test_fstring.py
  • test/test_funcattrs.py
  • test/test_future.py
  • test/test_future3.py
  • test/test_future4.py
  • test/test_future5.py
  • test/test_gdb.py
  • test/test_generator_stop.py
  • test/test_generators.py
  • test/test_genericalias.py
  • test/test_genericclass.py
  • test/test_genexps.py
  • test/test_getpath.py
  • test/test_global.py
  • test/test_grammar.py
  • test/test_grp.py
  • test/test_hash.py
  • test/test_hashlib.py
  • test/test_index.py
  • test/test_int.py
  • test/test_int_literal.py ()
  • test/test_interpreters.py
  • test/test_ioctl.py
  • test/test_isinstance.py
  • test/test_iter.py
  • test/test_iterlen.py
  • test/test_itertools.py
  • test/test_keywordonlyarg.py
  • test/test_kqueue.py
  • test/test_largefile.py
  • test/test_launcher.py
  • test/test_list.py
  • test/test_listcomps.py
  • test/test_lltrace.py
  • test/test_long.py
  • test/test_longexp.py
  • test/test_marshal.py
  • test/test_math.py
  • test/test_memoryio.py
  • test/test_memoryview.py
  • test/test_metaclass.py
  • test/test_minidom.py
  • test/test_mmap.py
  • test/test_module.py
  • test/test_multibytecodec.py
  • test/test_named_expressions.py
  • test/test_nis.py
  • test/test_numeric_tower.py
  • test/test_opcache.py
  • test/test_openpty.py unchanged
  • test/test_ordered_dict.py
  • test/test_ossaudiodev.py
  • test/test_osx_env.py
  • test/test_patma.py
  • test/test_peepholer.py
  • test/test_pep646_syntax.py
  • test/test_pkg.py
  • test/test_poll.py
  • test/test_popen.py
  • test/test_positional_only_arg.py
  • test/test_posix.py
  • test/test_pow.py
  • test/test_print.py
  • test/test_property.py
  • test/test_pulldom.py
  • test/test_pwd.py
  • test/test_pyexpat.py
  • test/test_raise.py
  • test/test_range.py
  • test/test_readline.py
  • test/test_regrtest.py
  • test/test_repl.py
  • test/test_resource.py
  • test/test_richcmp.py
  • test/test_robotparser.py
  • test/test_sax.py
  • test/test_scope.py
  • test/test_script_helper.py
  • test/test_select.py
  • test/test_set.py
  • test/test_setcomps.py
  • test/test_slice.py
  • test/test_smtpnet.py
  • test/test_sort.py
  • test/test_source_encoding.py
  • test/test_spwd.py
  • test/test_stable_abi_ctypes.py
  • test/test_startfile.py
  • test/test_strftime.py
  • test/test_string.py
  • test/test_string_literals.py
  • test/test_strptime.py
  • test/test_strtod.py
  • test/test_structseq.py
  • See also __match_args__ for structseq #4063
  • test/test_subclassinit.py
  • test/test_sundry.py
  • test/test_super.py
  • test/test_support.py
  • test/test_syntax.py
  • test.test_sys
  • test/test_sys_setprofile.py
  • test/test_sys_settrace.py
  • test/test_syslog.py
  • test/test_tcl.py
  • test/test_thread.py
  • test/test_threadedtempfile.py
  • test/test_threadsignals.py
  • test/test_time.py
  • test/test_timeout.py
  • test/test_tix.py
  • test/test_tuple.py
  • test/test_type_annotations.py
  • test/test_type_cache.py
  • test/test_type_comments.py
  • test/test_typechecks.py
  • test/test_types.py
  • test/test_ucn.py
  • test/test_unary.py
  • test/test_unicode.py
  • test/test_unicode_file.py
  • test/test_unicode_file_functions.py
  • test/test_unicode_identifiers.py
  • test/test_unicodedata.py
  • test/test_univnewlines.py
  • test/test_unpack.py
  • test/test_unpack_ex.py
  • test/test_unparse.py
  • test/test_userdict.py
  • test/test_userlist.py
  • test/test_userstring.py
  • test/test_utf8_mode.py
  • test/test_utf8source.py
  • test/test_wait3.py
  • test/test_wait4.py
  • test/test_weakset.py
  • test/test_winconsoleio.py
  • test/test_winreg.py Rewrite of winreg module and bump to 3.13.2 #5594
  • test/test_winsound.py
  • test/test_with.py
  • test/test_yield_from.py
  • test/test_zlib.py
  • test/test_asyncio
  • test/test_capi
  • test/test_import
  • test/test_tools

Not yet added libraries

These libraris are not added yet. Pure python one will be possible while others are not.

@youknowone youknowone pinned this issue Feb 20, 2025
@youknowone youknowone added C-enhancement New feature or request good first issue Good for newcomers C-compat A discrepancy between RustPython and CPython A-stdlib cpython CPython source code related labels Feb 20, 2025
@arihant2math
Copy link
Collaborator

antigravity does not need updating.

@arihant2math
Copy link
Collaborator

__hello__.py is also not updated

@arihant2math
Copy link
Collaborator

Same with test_openpty

@hbina
Copy link
Contributor

hbina commented Mar 8, 2025

Python 3.13 added the following test

with self.assertWarns(DeprecationWarning):
            # We need to put the bool in a variable, because the constant
            # ~False is evaluated at compile time due to constant folding;
            # consequently the DeprecationWarning would be issued during
            # module loading and not during test execution.
            false = False
            self.assertEqual(~false, -1)

AFAICT, this is deceptively a big change to how its implemented now. It seems to me that bools are just ints in RustPython? The call to ~false here delegates to PyInt::invert.

Edit: Oh wait...the test is disabled in RustPython.

Edit2: Is this an acceptable fix?
14b9178d759ca2d6545972c49b2e0572d05b9098
Or its not worth it to support something that would be deprecated?

@hbina
Copy link
Contributor

hbina commented Mar 8, 2025

test_unary is okay

hbina085@akarin-thinkpad ~/g/RustPython (main)> cp ../cpython/Lib/test/test_unary.py Lib/test/test_unary.py                                                                                                           (base) 
hbina085@akarin-thinkpad ~/g/RustPython (main)> cargo run --release --quiet -- -m test test_unary -v --quiet                                                                                                          (base) 
Run tests sequentially
test_unary passed

== Tests result: SUCCESS ==

Total duration: 10 ms
Tests result: SUCCESS

@arihant2math
Copy link
Collaborator

__hello__.py is unchanged and _osx_support.py has been updated. Also cgi.py has been removed.

@hbina
Copy link
Contributor

hbina commented Mar 23, 2025

test_bool.py doesn't change.

@youknowone
Copy link
Member Author

@hbina @arihant2math Thank you! all the comments are applied

@youknowone
Copy link
Member Author

Is this an acceptable fix? Or its not worth it to support something that would be deprecated?

@hbina Sure for both! We actually are good for either way. We can ignore it until being fully deprecated and removed later to work less, but welcome to merge it if you already made a patch.

@hbina
Copy link
Contributor

hbina commented Mar 30, 2025

test_audit.py

This basically noop because RustPython does not have sys.audit yet?
I have created PR here
#5637

test_audioop

I don't think test/test_audioop.py exists?
Fails because audioop module is not implemented.

test_augassign

Unchanged

test_bigaddrspace

Unchanged

test_bigmem

Unchanged

test_binascii

Added a new test that is failing, it has to do with the memoryview/buffer layer which I am not familiar enough so I am just adding expectedFailure now.

def test_c_contiguity(self):
        m = memoryview(bytearray(b'noncontig'))
        noncontig_writable = m[::-2]
        with self.assertRaises(BufferError):
            binascii.b2a_hex(noncontig_writable)

Edit:

Sorry...I checked out the wrong version of CPython. Retrying everyting.

@arihant2math
Copy link
Collaborator

@arihant2math
Copy link
Collaborator

Additionally, these prs haven't been merged yet:

@Shakil582
Copy link

Finishing up hmac, only need to get the all the unit tests working.

@arihant2math
Copy link
Collaborator

arihant2math commented Apr 1, 2025

Here, I compiled through some obvious ones with a script:
I manually checked the tests for these.
antigravity.py bisect.py contextvars.py copyreg.py graphlib.py heapq.py keyword.py numbers.py struct.py this.py

This isn't all of them, I'll get my script to output things in checklist form soon.

@arihant2math
Copy link
Collaborator

I have created a script to update this, which keeps the notes and the update prs in:

out.md

@youknowone
Copy link
Member Author

@arihant2math there is a previous work about it #4870

@arihant2math
Copy link
Collaborator

This script just generates the issue rather than attempt any updates.

I might consider writing a different script that does auto-upgrading later (or using that script).

@youknowone
Copy link
Member Author

youknowone commented Apr 19, 2025

@arihant2math will it be helpful if you can edit this issue?

@arihant2math
Copy link
Collaborator

@youknowone yes, definitely.

@arihant2math
Copy link
Collaborator

arihant2math commented Apr 20, 2025

Seems like triage access doesn't allow for the editing of other peoples issues.
I wonder if github actions could update an issue, and we setup the script to do that.

@arihant2math
Copy link
Collaborator

From #5731:

  • pprint
  • colorsys
  • queue
  • linecache
  • sched
  • heapq
  • graphlib

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-stdlib C-compat A discrepancy between RustPython and CPython C-enhancement New feature or request cpython CPython source code related good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants