Skip to content

fix: schedule autobuild directly on TestExecutorAutostopTemplateDisabled #10453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 1, 2023

Conversation

spikecurtis
Copy link
Contributor

@spikecurtis spikecurtis commented Nov 1, 2023

Fixes flake seen here: https://github.com/coder/coder/actions/runs/6716682414/job/18253279654

The test used a cron schedule to compute autobuild ticks, with ticks every hour on the hour. The default TTL was set to an hour. Usually, the next tick is less than one hour in the future, unless the test runs at :00 past the hour, which it did in my flake'd
run. But, given that this is an autostop test, the cron schedule is irrelevant (such schedules are used for auto_start_). So, I've removed it from the test and compute the build ticks directly.

Also, the test originally had the workspace TTL set to longer than the default template TTL, and then tested that no build happened when the tick was prior to both. This seems odd to me, as we want to demonstrate the the executor disregards the workspace TTL.
So, I changed the test to set the workspace TTL shorter, and then send in a tick between the two, verify that we don't autostop, then a tick after the template TTL and verify that we do.

Copy link
Contributor Author

Current dependencies on/for this PR:

This stack of pull requests is managed by Graphite.

Copy link
Member

@mafredri mafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, nice fix! Do you think we'll have similar issues with other tests using cron?

Copy link
Contributor Author

Makes sense to me, nice fix! Do you think we'll have similar issues with other tests using cron?

I don't think so --- the problem here was mismatch between cron and TTL. For autostart it really does use cron.

However, I do think that these tests are needlessly fragile in that they aren't controlling the times closely. Ideally the workspaces would be created at a fixed time, and the build ticks would also be fixed by the test. Right now, the workspaces are created at whatever time the test runs, and then we compute tick times relative to the creation time. That means that to the extent we test edge cases, we do it stochastically, rather than figuring out the edge cases and writing tests for them.

@spikecurtis spikecurtis merged commit 95ce697 into main Nov 1, 2023
@spikecurtis spikecurtis deleted the spike/flake-test-executor branch November 1, 2023 11:16
@github-actions github-actions bot locked and limited conversation to collaborators Nov 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants