Update test durations automatically based on latest durations from master #12748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

k-a-il merged 6 commits into master from ci/update-test-durations-automatically

Jun 17, 2025

Contributor

k-a-il commented Jun 12, 2025 •

edited

Loading

Motivation

While switching from CircleCI to GitHub Actions, I saw that the test durations file hadn’t been updated in about a year. Since then, we’ve added a lot of new tests. This pull request adds a workflow that downloads test durations from the latest scheduled run on the master branch, combines them, and creates a PR with the updated test durations file. New workflow is scheduled to run every month.

Changes

Added a workflow to download and merge the test durations file, and create a pull request with the updates

Testing

Tested in a forked repostiory.

PR created automatically: https://github.com/k-a-il/localstack/pull/5
Merged test durations uploaded as artifact

k-a-il added 3 commits

June 6, 2025 17:00


          CI: add wokflow to automatically update test durations every month

0e5ce8d


          CI: added detection of latest run id of main pipeline

fe03e6b


          CI: test durations rename token

b22b9a9

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results - Preflight, Unit

21 621 tests +8 19 964 ✅ +6 6m 11s ⏱️ -49s
1 suites ±0 1 657 💤 +2
1 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (MA/MR) - Preflight, Unit

21 621 tests +8 19 964 ✅ +6 6m 14s ⏱️ -7s
1 suites ±0 1 657 💤 +2
1 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 8s ⏱️ -1s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (amd64, MA/MR) - Acceptance

7 tests ±0 5 ✅ ±0 3m 5s ⏱️ -14s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (MA/MR) - Alternative Providers

987 tests ±0 589 ✅ ±0 31m 35s ⏱️ + 1m 34s
4 suites ±0 398 💤 ±0
4 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results - Alternative Providers

987 tests ±0 589 ✅ ±0 30m 37s ⏱️ +15s
4 suites ±0 398 💤 ±0
4 files ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (amd64) - Integration, Bootstrap

5 files ±0 5 suites ±0 2h 21m 30s ⏱️ + 1m 9s
5 231 tests +1 4 302 ✅ +1 929 💤 ±0 0 ❌ ±0
5 237 runs +1 4 302 ✅ +1 935 💤 ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

github-actions bot commented Jun 12, 2025 •

edited

Loading

Test Results (amd64, MA/MR) - Integration, Bootstrap

5 files ±0 5 suites ±0 2h 23m 14s ⏱️ +48s
5 231 tests +1 4 302 ✅ +1 929 💤 ±0 0 ❌ ±0
5 237 runs +1 4 302 ✅ +1 935 💤 ±0 0 ❌ ±0

Results for commit c4975e8. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.


          CI: GH PAT token, fix special chars in url

22e9809

k-a-il self-assigned this

k-a-il added the semver: patch label


          CI: update PR title and body

75f3790

k-a-il marked this pull request as ready for review

June 13, 2025 06:55

k-a-il requested review from alexrashed, dfangl, dominikschubert and silv-io as code owners

June 13, 2025 06:55

alexrashed approved these changes

View reviewed changes

Member

alexrashed left a comment

Nice! This is really cool! The update-test-durations.yml might be something we want to consider generalizing and moving to our meta repository in the future to use it in other repositories as well 🤩

silv-io approved these changes

View reviewed changes

Member

silv-io left a comment •

edited

Loading

LGTM! Just some minor non-blocking suggestions :)

.github/workflows/update-test-durations.yml Outdated Show resolved Hide resolved

.github/workflows/update-test-durations.yml Show resolved Hide resolved

.github/workflows/update-test-durations.yml Outdated Show resolved Hide resolved


          CI: fixed comments from PR

c4975e8

k-a-il merged commit 21c4d5d into master

52 checks passed

k-a-il deleted the ci/update-test-durations-automatically branch

June 17, 2025 08:02

alexrashed mentioned this pull request

[Testing] Update test durations #12815

Merged

alexrashed mentioned this pull request

decrease frequency of test duration update workflow #13016

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels