What about for cases where the test can't continue if a call fails?

What about for cases where the test can't continue if a call fails?

I'd expect we'd need a separate channel to signal t.FailNow from the main goroutine, plus the relevant scaffolding.
However, I would expect the worst case here is that you end up with a NPE or an AIOBE, which should still cause the test to fail.

For reference, here's what failed in the first lint run (not sure why it didn't show up in the runner output):

provisioner/terraform/parse_test.go:36:3: ruleguard: Do not call functions that may call t.FailNow in a goroutine, as this can cause data races (see testing.go:834) (gocritic)
                require.NoError(t, err)
                ^
provisioner/terraform/provision_test.go:56:3: ruleguard: Do not call functions that may call t.FailNow in a goroutine, as this can cause data races (see testing.go:834) (gocritic)
                require.NoError(t, err)
                ^
coderd/database/pubsub_test.go:51:4: ruleguard: Do not call functions that may call t.FailNow in a goroutine, as this can cause data races (see testing.go:834) (gocritic)
                        require.NoError(t, err)
                        ^
make: *** [Makefile:56: lint] Error 1

Edit: these are fixed now, only show up on Linux

I imagine we might get some weird logs from the test not stopping at the require in some cases, but using assert makes more sense in a lot of those go routines 👍

Is this the cause of weird test output occasionally? (can't find a sample, but I feel like you understand what I mean)

Is this the cause of weird test output occasionally? (can't find a sample, but I feel like you understand what I mean)

I think it's a contributing factor. At this stage I think it's basically a case of eliminating all of the possible sources of flakes to find that really annoying one, like that old phone you dropped down a sofa somewhere and has an alarm set.

This looks great!

Idea (for another PR): Considering we have this widespread pattern of go func()ing in tests where the goroutine may outlive the test, perhaps we could consider some kind of test helper that synchronizes the goroutine/cancellation to the test via t.Cleanup.

Usage could look like this (ctx will be cancelled at the end of test and test exit delayed until func exits):

testhelper.Go(t, func(ctx context.Context) {
	err := cmd.ExecuteContext(ctx)
	(require|assert).NoError(err)
})

Package name is obv. bad.. not sure what a good name would be. testutil, ctest, testy... 😄

Yeah this is deffo a good t.Helper candidate