Skip to content

fix: Close tailnet if agent is closed during creation #5375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 12, 2022
Merged

fix: Close tailnet if agent is closed during creation #5375

merged 2 commits into from
Dec 12, 2022

Conversation

mafredri
Copy link
Member

I noticed we're sometimes getting stuck when calling (*agent).Close(), considering the stack trace it looks like this is happening when the agent is shut down whilst bringing up the network. This refactor catches that edge case.

https://github.com/coder/coder/actions/runs/3659961959/jobs/6186551249

Stack trace (panic: test timed out after 20m0s) 
panic: test timed out after 20m0s

goroutine 8432 [running]:
testing.(*M).startAlarm.func1()
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:2036 +0xbb
created by time.goFunc
	/opt/hostedtoolcache/go/1.19.3/x64/src/time/sleep.go:176 +0x48

goroutine 1 [chan receive, 20 minutes]:
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1412 +0x8a6
testing.tRunner(0xc0005aa9c0, 0xc00083fa18)
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1452 +0x255
testing.runTests(0xc0007e8000?, {0x47316c0, 0x2, 0x2}, {0xc00083fbcc?, 0x4767220?, 0x4764b60?})
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1844 +0x7ed
testing.(*M).Run(0xc0007e8000)
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1726 +0xa85
go.uber.org/goleak.VerifyTestMain({0x3b08c80, 0xc0007e8000}, {0x0, 0x0, 0x0})
	/home/runner/go/pkg/mod/go.uber.org/goleak@v1.2.0/testmain.go:53 +0x71
github.com/coder/coder/agent_test.TestMain(...)
	/home/runner/work/coder/coder/agent/agent_test.go:52
main.main()
	_testmain.go:1123 +0x42d

goroutine 53 [chan receive, 19 minutes]:
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1412 +0x8a6
testing.tRunner(0xc0005aab60, 0x24fda10)
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1452 +0x255
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1493 +0x75e

goroutine 1960 [semacquire, 19 minutes]:
sync.runtime_Semacquire(0xc0020f41f0?)
	/opt/hostedtoolcache/go/1.19.3/x64/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc0020f41e8)
	/opt/hostedtoolcache/go/1.19.3/x64/src/sync/waitgroup.go:139 +0xa6
github.com/coder/coder/agent.(*agent).Close(0xc0020f4120)
	/home/runner/work/coder/coder/agent/agent.go:993 +0x1dc
github.com/coder/coder/agent_test.setupAgent.func2()
	/home/runner/work/coder/coder/agent/agent_test.go:799 +0x42
testing.(*common).Cleanup.func1()
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1041 +0x194
testing.(*common).runCleanup(0xc00140a820, 0x0)
	/opt/hostedtoolcache/go/1.19.3/x64/src/testing/testing.go:1210 +0x144
testing.tRunner.func2()
	/opt/hostedtoolcache/go/1.19.3/x64/src/net/udpsock_posix.go:52 +0x296
net.(*UDPConn).readFromUDP(0xc001f8c070, {0xc000eca000, 0x10000, 0x10000}, 0x0?)
	/opt/hostedtoolcache/go/1.19.3/x64/src/net/udpsock.go:149 +0x8d
net.(*UDPConn).ReadFrom(0xc001f8c070?, {0xc000eca000, 0x10000, 0x10000})
	/opt/hostedtoolcache/go/1.19.3/x64/src/net/udpsock.go:158 +0xa6
tailscale.com/net/stun/stuntest.runSTUN({0x3b27240, 0xc00140a820}, {0x3b1f1d0, 0xc001f8c070}, 0xc000e88078, 0xc002596180)
	/home/runner/go/pkg/mod/github.com/coder/tailscale@v1.1.1-0.20221117204504-2d6503f027c3/net/stun/stuntest/stuntest.go:60 +0x122
created by tailscale.com/net/stun/stuntest.ServeWithPacketListener
	/home/runner/go/pkg/mod/github.com/coder/tailscale@v1.1.1-0.20221117204504-2d6503f027c3/net/stun/stuntest/stuntest.go:48 +0x405

goroutine 5550 [chan receive, 19 minutes]:
github.com/coder/coder/agent_test.(*client).AgentReportStats.func2()
	/home/runner/work/coder/coder/agent/agent_test.go:911 +0x53
github.com/coder/coder/agent_test.closeFunc.Close(0xc001d10120)
	/home/runner/work/coder/coder/agent/agent_test.go:768 +0x2f
github.com/coder/coder/agent.(*agent).init.func9()
	/home/runner/work/coder/coder/agent/agent.go:574 +0x88
github.com/coder/coder/agent.(*agent).trackConnGoroutine.func1()
	/home/runner/work/coder/coder/agent/agent.go:243 +0x93
created by github.com/coder/coder/agent.(*agent).trackConnGoroutine
	/home/runner/work/coder/coder/agent/agent.go:241 +0x2d6

goroutine 5549 [semacquire, 19 minutes]:
sync.runtime_SemacquireMutex(0x7f1f549ccf00?, 0x0?, 0xc000d68a50?)
	/opt/hostedtoolcache/go/1.19.3/x64/src/runtime/sema.go:77 +0x25
sync.(*Mutex).lockSlow(0xc0020f4200)
	/opt/hostedtoolcache/go/1.19.3/x64/src/sync/mutex.go:171 +0x225
sync.(*Mutex).Lock(0xc0020f4200)
	/opt/hostedtoolcache/go/1.19.3/x64/src/sync/mutex.go:90 +0x65
github.com/coder/coder/agent.(*agent).init.func8()
	/home/runner/work/coder/coder/agent/agent.go:560 +0xac
github.com/coder/coder/agent_test.(*client).AgentReportStats.func1()
	/home/runner/work/coder/coder/agent/agent_test.go:900 +0x234
created by github.com/coder/coder/agent_test.(*client).AgentReportStats
	/home/runner/work/coder/coder/agent/agent_test.go:888 +0x1a8

@mafredri mafredri self-assigned this Dec 12, 2022
@mafredri mafredri requested a review from a team December 12, 2022 09:11
@mafredri mafredri enabled auto-merge (squash) December 12, 2022 11:16
@mafredri mafredri merged commit 88bb901 into main Dec 12, 2022
@mafredri mafredri deleted the mafredri/fix-agent-closure-during-tailnet-creation branch December 12, 2022 11:26
@github-actions github-actions bot locked and limited conversation to collaborators Dec 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mtojek mtojek mtojek approved these changes

Assignees

@mafredri mafredri

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mafredri @mtojek