Skip to content

chore: change mock clock to allow Advance() within timer/tick functions #13500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 10, 2024

Conversation

spikecurtis
Copy link
Contributor

@spikecurtis spikecurtis commented Jun 7, 2024

I've modified the Mock.Advance() function to be more simple and allow more flexibility in testing.

tl;dr - Advance() now only allows you to advance up to the next timer/ticker event and no further, and immediately advances the clock, rather than the clock advancing async as ticks and timers are processed.

Why?

Prior to this change, Advance() moves the clock forward, possibly triggering multiple timers/ticks. You had to Wait() for the result before you could be sure the clock had moved as far as you asked. And, crucially, the internal implementation didn't allow you to make another call to Advance() until the first one finished.

But, a test case that I want to support is being able to have time progress during a timer/ticker functional callback. The added UT to enterprise/tailnet is an example of this. heartbeats makes a call to Now() to record the time of a heartbeat, then later calls Until() to calculate when next to check for expiry. In a real system these calls can happen with different "current" times, and there was a bug that I noticed that if a coordinator is just about to expire, but not quite, by the time we get around to calculated when next to check, it could already be in the past, and we would have ignored it. So, we need to Trap() the call to Until and advance the clock.

With the prior implementation, this fails. I initially went down a rabbit hole of having a central goroutine manage advancing the clock, and allowing Advance() calls to come in while others have not fully resolved, but things get messy and complicated very quickly. In particular, how does the thing managing the clock know the order to process advances when one is happening during another?

That way lies madness, probably, and completely destroys the goal of writing tests where the time is predictable after every mock call.

The proposed solution

So instead, we restrict Advance() to be able to, in a single call, move the clock up to the next event and no further. Then, it can immediately set the new time, even if you asynchronously wait for timers/ticker functions to return.

For cases where you don't know or want to compute how long to the next event, I've added AdvanceNext() which returns the amount advanced for asserting or use in further calculations.

This gets us back to a place where UTs are in the imperative style and the clock moves predictably, monotonically forward as the UT is executed.

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @spikecurtis and the rest of your teammates on Graphite Graphite

@spikecurtis spikecurtis requested review from johnstcn and mafredri June 7, 2024 07:51
@spikecurtis spikecurtis marked this pull request as ready for review June 7, 2024 07:53
select {
case <-t.c:
default:
}
if d == 0 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before, when we had Advance() handle multiple sets of timers/ticks events, we just handled this case of a zero-duration timer reset in the advance loop. But now, since Advance() can only trigger one "flight" of events, we handle the zero-duration case inline.

One slight disadvantage is that there isn't a way to wait for such events to complete.

Copy link
Member

@johnstcn johnstcn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a great usability improvement.
However, I anticipate casual usage will simply ignore the returned duration of AdvanceWait():

_, w := mClock.AdvanceNext(); w.MustWait(ctx)

Do you foresee this being an anti-pattern?

@spikecurtis
Copy link
Contributor Author

I think this is a great usability improvement. However, I anticipate casual usage will simply ignore the returned duration of AdvanceWait():

_, w := mClock.AdvanceNext(); w.MustWait(ctx)

Do you foresee this being an anti-pattern?

I think this is fine for relatively simple cases, where it's clear what the next event is. For example, if you're testing something with a single ticker, you can assert the duration of the ticker once and then use AdvanceNext() without checking the return value.

Where you can get yourself in trouble is when there are multiple timers and/or tickers, and you're not carefully controlling when they are set, or not paying attention, and so the test is written assuming the next event is one thing, but it's actually a different thing.

@spikecurtis spikecurtis merged commit 8326a3a into main Jun 10, 2024
34 checks passed
@spikecurtis spikecurtis deleted the spike/clock-testing-tailnet branch June 10, 2024 11:27
@github-actions github-actions bot locked and limited conversation to collaborators Jun 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants