Skip to content

Conversation

ethanndickson
Copy link
Member

@ethanndickson ethanndickson commented Sep 3, 2025

Closes coder/internal#961
Likely the same deal as in #19599, the body of require.Eventually now fires immediately, when it used to fire after 250ms (the interval). Presumably, the deployment stats become ready before the vs code session count gets incremented. This was never an issue with the 250ms delay, as this flake has only cropped up after the testify version bump.

We'll fix the issue by making it possible to wait for a full metrics cache refresh, i.e. removing require.Eventually in this test altogether.

Copy link
Member Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ethanndickson ethanndickson marked this pull request as ready for review September 3, 2025 05:49
@spikecurtis
Copy link
Contributor

Can we stop using Eventually entirely? Like, make this test actually deterministic?

@ethanndickson
Copy link
Member Author

ethanndickson commented Sep 3, 2025

Can we stop using Eventually entirely? Like, make this test actually deterministic?

@spikecurtis I've gone with an approach similar to that used by dbpurge_test, though I had to change the way the ticking works slightly. This seems preferable to modifying the API of the metrics cache.

If this approach looks good to you, I'll update the other tests to also no longer use require.Eventually.

EDIT: An unrelated test is failing because it calls clock.Set with a time after the next metricscache tick time (which quartz does not allow).
If we want to go with this solution, we may need to stop piping the clock from coderdtest.Options through to metricscache, and just always give it a real clock when created as part of a coderdtest.
Alternatively, those tests that need to set the clock to a time in the future could just choose really long tick intervals.
WDYT?

Copy link
Contributor

In general, clock.Set is discouraged other than start of test before you pass it to anything.

Also, Quartz mocks are not really designed for integration tests where you have a whole lot of things calling into the clock with different intervals.

@ethanndickson ethanndickson force-pushed the ethan/deployment-stats-flake branch from 1c19d4f to 3093efd Compare September 4, 2025 06:47
Copy link
Contributor

@spikecurtis spikecurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion inline about only triggering a single tick and using Advance rather than AdvanceNext, but I don't need to review again. Nice work.

@ethanndickson
Copy link
Member Author

~/coder [main] » go test github.com/coder/coder/v2/coderd/metricscache -count=1
ok      github.com/coder/coder/v2/coderd/metricscache   5.099s

~/coder [ethan/deployment-stats-flake] » go test github.com/coder/coder/v2/coderd/metricscache -count=1
ok      github.com/coder/coder/v2/coderd/metricscache   0.118s

🎉

@ethanndickson ethanndickson merged commit dae1903 into main Sep 8, 2025
32 checks passed
@ethanndickson ethanndickson deleted the ethan/deployment-stats-flake branch September 8, 2025 02:07
@github-actions github-actions bot locked and limited conversation to collaborators Sep 8, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flake: TestCache_DeploymentStats
2 participants