Skip to content

chore: use a smaller runner for e2e test #15034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 12, 2024
Merged

chore: use a smaller runner for e2e test #15034

merged 3 commits into from
Oct 12, 2024

Conversation

matifali
Copy link
Member

@matifali matifali commented Oct 10, 2024

We run our e2e-tests on a 16-core machine with --max-workers=1
Using a standard runner with 2 cores, the machine runs the tests in the same amount of time while reducing the cost 8 times.

@matifali matifali changed the title chore: use a smaller runner fo e2e test chore: use a smaller runner for e2e test Oct 10, 2024
@matifali matifali marked this pull request as ready for review October 10, 2024 12:10
@matifali matifali requested review from ammario and mtojek October 10, 2024 12:11
@matifali
Copy link
Member Author

Ok. The e2e test is failing. Could it be because #14979 hasn't landed yet?

Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--max-workers=1
probably to reduce flakiness

Yes, at one point we had trouble running multiple tests in parallel because the machine was overloaded and started timing out. I think we could experiment to see if the situation has improved?

EDIT:

BTW if we want to increase parallelization level, we need to remember that this may affect API rate limit 👍

@matifali
Copy link
Member Author

Thank you for the context @mtojek. We can investigate parallelism separately, my intention for this change now is to reduce CI compute cost.

Also could you or someone from the team have a look why test-e2e is failing now while test-e2e-enterprise is passing.

@matifali
Copy link
Member Author

Using a 4 core 16GB runner solves the issue. This is interesting for me and I would like to kno why.

@matifali matifali enabled auto-merge (squash) October 12, 2024 11:20
@matifali matifali disabled auto-merge October 12, 2024 11:20
@matifali matifali enabled auto-merge (squash) October 12, 2024 11:21
@matifali matifali merged commit 20b3f8f into main Oct 12, 2024
29 checks passed
@matifali matifali deleted the atif/e2e-runner branch October 12, 2024 11:23
@github-actions github-actions bot locked and limited conversation to collaborators Oct 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants