Skip to content

gh-96710: Make the test timing more lenient for the int/str DoS regression test. #96717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 9, 2022
Merged

gh-96710: Make the test timing more lenient for the int/str DoS regression test. #96717

merged 1 commit into from
Sep 9, 2022

Conversation

gpshead
Copy link
Member

@gpshead gpshead commented Sep 9, 2022 •
edited by bedevere-bot
Loading

The regression would still absolutely FAIL if we regressed, and even a flaky PASS, isn't harmful as it'd fail most of the time across our NN CI system test runs.

Windows has a low resolution timer and CI systems are prone to odd timing so this just gives more leeway to avoid occasional flakiness.

If this comes up again we can revisit what is needed here.

@gpshead
Make the test timing more lenient for the int/str DoS regression test.
1af8df2 
A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
@gpshead gpshead added tests Tests in the Lib/test dir OS-windows skip news needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Sep 9, 2022
@gpshead gpshead merged commit 11e3548 into python:main Sep 9, 2022
@miss-islington
Copy link
Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

@gpshead gpshead deleted the reduce-flakiness branch September 9, 2022 19:51
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 9, 2022
@gpshead @miss-islington
pythongh-96710: Make the test timing more lenient for the int/str DoS…
64f460d 
… regression test. (pythonGH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label Sep 9, 2022
@bedevere-bot
Copy link

GH-96725 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Sep 9, 2022
@bedevere-bot
Copy link

GH-96726 is a backport of this pull request to the 3.10 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 9, 2022
@gpshead @miss-islington
pythongh-96710: Make the test timing more lenient for the int/str DoS…
5fba3b1 
… regression test. (pythonGH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
miss-islington added a commit that referenced this pull request Sep 9, 2022
@miss-islington @gpshead
gh-96710: Make the test timing more lenient for the int/str DoS regre…
5a17200 
…ssion test. (GH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
miss-islington added a commit that referenced this pull request Sep 9, 2022
@miss-islington @gpshead
gh-96710: Make the test timing more lenient for the int/str DoS regre…
fecda02 
…ssion test. (GH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
@miss-islington
Copy link
Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-98195 is a backport of this pull request to the 3.7 branch.

@bedevere-bot
Copy link

GH-98196 is a backport of this pull request to the 3.9 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 11, 2022
@gpshead @miss-islington
pythongh-96710: Make the test timing more lenient for the int/str DoS…
9a7b032 
… regression test. (pythonGH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 11, 2022
@gpshead @miss-islington
pythongh-96710: Make the test timing more lenient for the int/str DoS…
92d51b9 
… regression test. (pythonGH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-bot bedevere-bot removed the needs backport to 3.9 only security fixes label Oct 11, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 11, 2022
@gpshead @miss-islington
pythongh-96710: Make the test timing more lenient for the int/str DoS…
256b24b 
… regression test. (pythonGH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-bot
Copy link

GH-98197 is a backport of this pull request to the 3.8 branch.

@ambv
Copy link
Contributor

ambv commented Oct 11, 2022

Bedevere failed to comment here but GH-98195 is the 3.7 backport and GH-98196 is the 3.9 one.

ambv pushed a commit that referenced this pull request Oct 11, 2022
@miss-islington @gpshead
[3.8] gh-96710: Make the test timing more lenient for the int/str DoS…
a44cc0a 
… regression test. (GH-96717) (#98197)

gh-96710: Make the test timing more lenient for the int/str DoS regression test. (GH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
ambv pushed a commit that referenced this pull request Oct 11, 2022
@miss-islington @gpshead
[3.9] gh-96710: Make the test timing more lenient for the int/str DoS…
157a8b8 
… regression test. (GH-96717) (#98196)

gh-96710: Make the test timing more lenient for the int/str DoS regression test. (GH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
ambv pushed a commit that referenced this pull request Oct 11, 2022
@miss-islington @gpshead
[3.7] gh-96710: Make the test timing more lenient for the int/str DoS…
e7fe111 
… regression test. (GH-96717) (#98195)

gh-96710: Make the test timing more lenient for the int/str DoS regression test. (GH-96717)

A regression would still absolutely fail and even a flaky pass isn't
harmful as it'd fail most of the time across our N system test runs.

Windows has a low resolution timer and CI systems are prone to odd
timing so this just gives more leeway to avoid flakiness.
(cherry picked from commit 11e3548)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
OS-windows skip news tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gpshead @miss-islington @bedevere-bot @ambv