Skip to content

Failure marker #5695

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 24, 2025
Merged

Failure marker #5695

merged 6 commits into from
Apr 24, 2025

Conversation

arihant2math
Copy link
Collaborator

@arihant2math arihant2math commented Apr 14, 2025

Working script to auto-mark rustpython failures.
Has a few limitations but should work in 90% of the cases.
Ideally I'd fix the class/subclass situation as well as proper ast parsing.

@youknowone
Copy link
Member

How do you run this code and what does it output?

@arihant2math
Copy link
Collaborator Author

arihant2math commented Apr 14, 2025

Currently I need to patch it a little ... but the idea is that the workflow is the following:

  1. Make some changes to Lib/ and Lib/test/
  2. Find which tests are failing
  3. Run python ./scripts/fix_test.py --test test_venv --path ./Lib/test/test_venv.py or equivalent for the test from the project root.
  4. Actually fix the test.

This works best for platform-independent tests.

The limitations are:

  1. Can't handle unexpected successes (I could fix this, but this is rare enough, I think it's fine
  2. Can't handle entire test file crashes (this is another thing the user has to handle themselves)
  3. Can't handle subclasses and nested classes etc. (I'm working on a fix for this though)

@arihant2math
Copy link
Collaborator Author

Having a working --junit-xml would help with finding failing tests, instead of parsing stdout.
Because some tests like printing out to stdout, ruining the parser.

#4072 is the blocking issue.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the comment to the header of the script as documentation

@arihant2math
Copy link
Collaborator Author

Done.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@youknowone youknowone merged commit c97f4d1 into RustPython:main Apr 24, 2025
11 checks passed
@arihant2math arihant2math deleted the failure-marker branch April 24, 2025 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants