Skip to content

gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode #112124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 16, 2023

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Nov 15, 2023

Reenable test_call.test_super_deep() on the s390x architecture.

@vstinner
Copy link
Member Author

@hroncok: I'm working on a fix for test_call.test_super_deep().

Once GitHub Action jobs validate this PR, I plan to test it on WASI on s390x buildbots.

@vstinner
Copy link
Member Author

!buildbot s390x

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 5e343d7 🤖

The command will test the builders whose names match following regular expression: s390x

The builders matched are:

  • s390x RHEL7 PR
  • s390x RHEL7 LTO + PGO PR
  • s390x Fedora Clang Installed PR
  • s390x Fedora Rawhide LTO PR
  • s390x Fedora Refleaks PR
  • s390x RHEL8 LTO PR
  • s390x SLES PR
  • s390x Fedora Rawhide Clang PR
  • s390x Fedora Rawhide PR
  • s390x Debian PR
  • s390x Fedora Rawhide LTO + PGO PR
  • s390x RHEL7 LTO PR
  • s390x RHEL8 Refleaks PR
  • s390x Fedora Rawhide Clang Installed PR
  • s390x Fedora LTO PR
  • s390x Fedora LTO + PGO PR
  • s390x Fedora Rawhide Refleaks PR
  • s390x RHEL7 Refleaks PR
  • s390x RHEL8 PR
  • s390x Fedora PR
  • s390x Fedora Clang PR
  • s390x RHEL8 LTO + PGO PR

@vstinner
Copy link
Member Author

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 5e343d7 🤖

The command will test the builders whose names match following regular expression: wasm32

The builders matched are:

  • wasm32-emscripten browser (dynamic linking, no tests) PR
  • wasm32 WASI PR
  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-wasi PR

@brettcannon
Copy link
Member

Once GitHub Action jobs validate this PR, I plan to test it on WASI on s390x buildbots.

FYI all the WASI buildbots are non-pydebug builds, so this won't validate it fixes that scenario (I have been doing that manually so that once things work I can switch the WASI buildbot over to doing pydebug builds). I had to get Py_C_RECURSION_LIMIT down to 85 to get test_call to pass.

@vstinner
Copy link
Member Author

FYI all the WASI buildbots are non-pydebug builds, so this won't validate it fixes that scenario (I have been doing that manually so that once things work I can switch the WASI buildbot over to doing pydebug builds).

I wrote this PR to fix the test_call crash on Linux when Python is built with gcc -O0 in debug mode. I reuse your issue for that.

This change should not impact WASI, since it sets Py_C_RECURSION_LIMIT to 500 in debug mode. Before, Py_C_RECURSION_LIMIT was always set to 500 on WASI. Same this constant value is not changed by my PR on WASI.

I suppose that a following change is needed to adjust limits for WASI built in debug mode.

@vstinner
Copy link
Member Author

Mmmh. Builders have a long queue of jobs but do nothing:

Screenshot 2023-11-16 at 05-23-50 Python builder s390x RHEL7 PR

Screenshot 2023-11-16 at 05-23-59 Python builder wasm32 WASI PR

I restarted to the buildbot server. It seems like it was blocked. By the way, when I stopped it, it said that it was "already shutting down" which remains me a Twisted/Buildbot bug. Sometimes, when I tried to stop the server, it moves to a broken state where it still runs but does nothing, and it doesn't stop.

2023-11-16 04:21:02+0000 [-] Ignoring SIGTERM, master is already shutting down.

@vstinner
Copy link
Member Author

Not good: test_call fails on s390x RHEL7 LTO PR on s390x RHEL7 LTO PR buildbots :-(

There are also many buildbots which fail for unrelated reasons, like Git clone failure, disk full, known build failure, known test failures, etc. Not sure why buildbot

@vstinner
Copy link
Member Author

vstinner commented Nov 16, 2023

Not good: test_call fails on s390x RHEL7 LTO PR on s390x RHEL7 LTO PR buildbots :-(

Oh, s390x RHEL7 LTO + PGO PR builds Python in release mode ("release LTO+PGO") and s390x RHEL7 LTO PR is also built in release mode ("release LTO").

This change only affects Python built in debug mode if the Py_DEBUG macro is defined.

* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
  test_trace_unpack_long_sequence() now adjust the maximum recursion
  depth depending on the the remaining C recursion.
@vstinner
Copy link
Member Author

I don't have the bandwidth to investigate the s390x failure right now, so I just kept the s390x skip for now sadly.

@vstinner vstinner enabled auto-merge (squash) November 16, 2023 13:39
@vstinner vstinner merged commit bd89bca into python:main Nov 16, 2023
@vstinner vstinner deleted the test_super_deep branch November 16, 2023 13:52
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
…#112124)

* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
  test_trace_unpack_long_sequence() now adjust the maximum recursion
  depth depending on the the remaining C recursion.
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
…#112124)

* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
  test_trace_unpack_long_sequence() now adjust the maximum recursion
  depth depending on the the remaining C recursion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants