Skip to content

flake: Data race in coderd/notifications.(*Manager) (loop/Stop assign/read of m.notifier) #544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mafredri opened this issue Mar 27, 2025 · 0 comments · Fixed by coder/coder#17702
Closed

flake: Data race in coderd/notifications.(*Manager) (loop/Stop assign/read of m.notifier) #544

mafredri opened this issue Mar 27, 2025 · 0 comments · Fixed by coder/coder#17702
Assignees
@defelmnq
Labels
flake

Comments

@mafredri
Copy link
Member

mafredri commented Mar 27, 2025 •
edited
Loading 

==================
WARNING: DATA RACE
Read at 0x00c00f272a68 by goroutine 8190:
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop.func1()
      /home/coder/coder/coderd/notifications/manager.go:371 +0x1d1
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:78 +0xe1
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:69 +0x44
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop()
      /home/coder/coder/coderd/notifications/manager.go:359 +0xb4
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop-fm()
      <autogenerated>:1 +0x47
  github.com/coder/coder/v2/cli.shutdownWithTimeout()
      /home/coder/coder/cli/server.go:1367 +0x83
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:1150 +0xddbd
  github.com/coder/coder/v2/coderd/database/dbmem.(*FakeQuerier).InTx()
      /home/coder/coder/coderd/database/dbmem/dbmem.go:434 +0x365
  github.com/coder/coder/v2/coderd/database/dbmetrics.metricsStore.InTx()
      /home/coder/coder/coderd/database/dbmetrics/dbmetrics.go:83 +0x17a
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/coderd/database/dbauthz.(*querier).InTx()
      /home/coder/coder/coderd/database/dbauthz/dbauthz.go:694 +0xf9
  github.com/coder/coder/v2/coderd/cryptokeys.(*rotator).rotateKeys()
      /home/coder/coder/coderd/cryptokeys/rotate.go:97 +0x195
  github.com/coder/coder/v2/coderd/cryptokeys.StartRotator()
      /home/coder/coder/coderd/cryptokeys/rotate.go:72 +0x590
  github.com/coder/coder/v2/coderd.New()
      /home/coder/coder/coderd/coderd.go:531 +0x451b
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func1()
      /home/coder/coder/cli/server.go:294 +0x26
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:894 +0x9584
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:754 +0x7832
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func6.1()
      /home/coder/coder/cli/root.go:319 +0x141
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func7.PrintDeprecatedOptions.2.1()
      /home/coder/coder/cli/root.go:1361 +0x216
  github.com/coder/coder/v2/cli.(*RootCmd).Server.RequireNArgs.RequireRangeArgs.func6.1()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:753 +0x604
  github.com/coder/coder/v2/cli.(*RootCmd).Server.WriteConfigMW.func5.1()
      /home/coder/coder/cli/server.go:1316 +0x354
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:499 +0x2cab
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:419 +0x16b8
  github.com/coder/serpent.(*Invocation).Run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:594 +0x344
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter.func1()
      /home/coder/coder/cli/clitest/clitest.go:253 +0x144

Previous write at 0x00c00f272a68 by goroutine 19015:
  github.com/coder/coder/v2/coderd/notifications.(*Manager).loop()
      /home/coder/coder/coderd/notifications/manager.go:178 +0x7a4
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run.func1.1()
      /home/coder/coder/coderd/notifications/manager.go:146 +0x6a

Goroutine 8190 (running) created at:
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter()
      /home/coder/coder/cli/clitest/clitest.go:250 +0x3d9
  github.com/coder/coder/v2/cli/clitest.StartWithAssert()
      /home/coder/coder/cli/clitest/clitest.go:153 +0x58
  github.com/coder/coder/v2/cli/clitest.Start()
      /home/coder/coder/cli/clitest/clitest.go:144 +0x20e
  github.com/coder/coder/v2/cli_test.TestServer.func9()
      /home/coder/coder/cli/server_test.go:508 +0x1fa
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1792 +0x225
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1851 +0x44

Goroutine 19015 (running) created at:
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run.func1()
      /home/coder/coder/coderd/notifications/manager.go:145 +0xe4
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:78 +0xe1
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:69 +0x44
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run()
      /home/coder/coder/coderd/notifications/manager.go:143 +0x199
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:963 +0xaff7
  github.com/coder/coder/v2/coderd/database/dbmem.(*FakeQuerier).InTx()
      /home/coder/coder/coderd/database/dbmem/dbmem.go:434 +0x365
  github.com/coder/coder/v2/coderd/database/dbmetrics.metricsStore.InTx()
      /home/coder/coder/coderd/database/dbmetrics/dbmetrics.go:83 +0x17a
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/coderd/database/dbauthz.(*querier).InTx()
      /home/coder/coder/coderd/database/dbauthz/dbauthz.go:694 +0xf9
  github.com/coder/coder/v2/coderd/cryptokeys.(*rotator).rotateKeys()
      /home/coder/coder/coderd/cryptokeys/rotate.go:97 +0x195
  github.com/coder/coder/v2/coderd/cryptokeys.StartRotator()
      /home/coder/coder/coderd/cryptokeys/rotate.go:72 +0x590
  github.com/coder/coder/v2/coderd.New()
      /home/coder/coder/coderd/coderd.go:531 +0x451b
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func1()
      /home/coder/coder/cli/server.go:294 +0x26
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:894 +0x9584
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:754 +0x7832
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func6.1()
      /home/coder/coder/cli/root.go:319 +0x141
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func7.PrintDeprecatedOptions.2.1()
      /home/coder/coder/cli/root.go:1361 +0x216
  github.com/coder/coder/v2/cli.(*RootCmd).Server.RequireNArgs.RequireRangeArgs.func6.1()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:753 +0x604
  github.com/coder/coder/v2/cli.(*RootCmd).Server.WriteConfigMW.func5.1()
      /home/coder/coder/cli/server.go:1316 +0x354
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:499 +0x2cab
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:419 +0x16b8
  github.com/coder/serpent.(*Invocation).Run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:594 +0x344
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter.func1()
      /home/coder/coder/cli/clitest/clitest.go:253 +0x144
==================

==================
WARNING: DATA RACE
Read at 0x00c0181fe4f8 by goroutine 8190:
  sync/atomic.LoadInt32()
      /usr/local/go/src/runtime/race_amd64.s:202 +0xb
  sync/atomic.LoadUint32()
      <autogenerated>:1 +0x10
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:67 +0x30
  github.com/coder/coder/v2/coderd/notifications.(*notifier).stop()
      /home/coder/coder/coderd/notifications/notifier.go:373 +0x25e
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop.func1()
      /home/coder/coder/coderd/notifications/manager.go:376 +0x1e4
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:78 +0xe1
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:69 +0x44
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop()
      /home/coder/coder/coderd/notifications/manager.go:359 +0xb4
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Stop-fm()
      <autogenerated>:1 +0x47
  github.com/coder/coder/v2/cli.shutdownWithTimeout()
      /home/coder/coder/cli/server.go:1367 +0x83
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:1150 +0xddbd
  github.com/coder/coder/v2/coderd/database/dbmem.(*FakeQuerier).InTx()
      /home/coder/coder/coderd/database/dbmem/dbmem.go:434 +0x365
  github.com/coder/coder/v2/coderd/database/dbmetrics.metricsStore.InTx()
      /home/coder/coder/coderd/database/dbmetrics/dbmetrics.go:83 +0x17a
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/coderd/database/dbauthz.(*querier).InTx()
      /home/coder/coder/coderd/database/dbauthz/dbauthz.go:694 +0xf9
  github.com/coder/coder/v2/coderd/cryptokeys.(*rotator).rotateKeys()
      /home/coder/coder/coderd/cryptokeys/rotate.go:97 +0x195
  github.com/coder/coder/v2/coderd/cryptokeys.StartRotator()
      /home/coder/coder/coderd/cryptokeys/rotate.go:72 +0x590
  github.com/coder/coder/v2/coderd.New()
      /home/coder/coder/coderd/coderd.go:531 +0x451b
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func1()
      /home/coder/coder/cli/server.go:294 +0x26
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:894 +0x9584
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:754 +0x7832
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func6.1()
      /home/coder/coder/cli/root.go:319 +0x141
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func7.PrintDeprecatedOptions.2.1()
      /home/coder/coder/cli/root.go:1361 +0x216
  github.com/coder/coder/v2/cli.(*RootCmd).Server.RequireNArgs.RequireRangeArgs.func6.1()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:753 +0x604
  github.com/coder/coder/v2/cli.(*RootCmd).Server.WriteConfigMW.func5.1()
      /home/coder/coder/cli/server.go:1316 +0x354
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:499 +0x2cab
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:419 +0x16b8
  github.com/coder/serpent.(*Invocation).Run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:594 +0x344
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter.func1()
      /home/coder/coder/cli/clitest/clitest.go:253 +0x144

Previous write at 0x00c0181fe4f8 by goroutine 19015:
  github.com/coder/coder/v2/coderd/notifications.newNotifier()
      /home/coder/coder/coderd/notifications/notifier.go:74 +0x4aa
  github.com/coder/coder/v2/coderd/notifications.(*Manager).loop()
      /home/coder/coder/coderd/notifications/manager.go:178 +0x78f
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run.func1.1()
      /home/coder/coder/coderd/notifications/manager.go:146 +0x6a

Goroutine 8190 (running) created at:
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter()
      /home/coder/coder/cli/clitest/clitest.go:250 +0x3d9
  github.com/coder/coder/v2/cli/clitest.StartWithAssert()
      /home/coder/coder/cli/clitest/clitest.go:153 +0x58
  github.com/coder/coder/v2/cli/clitest.Start()
      /home/coder/coder/cli/clitest/clitest.go:144 +0x20e
  github.com/coder/coder/v2/cli_test.TestServer.func9()
      /home/coder/coder/cli/server_test.go:508 +0x1fa
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:1792 +0x225
  testing.(*T).Run.gowrap1()
      /usr/local/go/src/testing/testing.go:1851 +0x44

Goroutine 19015 (running) created at:
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run.func1()
      /home/coder/coder/coderd/notifications/manager.go:145 +0xe4
  sync.(*Once).doSlow()
      /usr/local/go/src/sync/once.go:78 +0xe1
  sync.(*Once).Do()
      /usr/local/go/src/sync/once.go:69 +0x44
  github.com/coder/coder/v2/coderd/notifications.(*Manager).Run()
      /home/coder/coder/coderd/notifications/manager.go:143 +0x199
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:963 +0xaff7
  github.com/coder/coder/v2/coderd/database/dbmem.(*FakeQuerier).InTx()
      /home/coder/coder/coderd/database/dbmem/dbmem.go:434 +0x365
  github.com/coder/coder/v2/coderd/database/dbmetrics.metricsStore.InTx()
      /home/coder/coder/coderd/database/dbmetrics/dbmetrics.go:83 +0x17a
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/coderd/database/dbauthz.(*querier).InTx()
      /home/coder/coder/coderd/database/dbauthz/dbauthz.go:694 +0xf9
  github.com/coder/coder/v2/coderd/cryptokeys.(*rotator).rotateKeys()
      /home/coder/coder/coderd/cryptokeys/rotate.go:97 +0x195
  github.com/coder/coder/v2/coderd/cryptokeys.StartRotator()
      /home/coder/coder/coderd/cryptokeys/rotate.go:72 +0x590
  github.com/coder/coder/v2/coderd.New()
      /home/coder/coder/coderd/coderd.go:531 +0x451b
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func1()
      /home/coder/coder/cli/server.go:294 +0x26
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:894 +0x9584
  github.com/coder/coder/v2/coderd/database/dbmetrics.(*metricsStore).InTx()
      <autogenerated>:1 +0xc8
  github.com/coder/coder/v2/cli.(*RootCmd).Server.func2()
      /home/coder/coder/cli/server.go:754 +0x7832
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func6.1()
      /home/coder/coder/cli/root.go:319 +0x141
  github.com/coder/coder/v2/cli.(*RootCmd).Command.func7.PrintDeprecatedOptions.2.1()
      /home/coder/coder/cli/root.go:1361 +0x216
  github.com/coder/coder/v2/cli.(*RootCmd).Server.RequireNArgs.RequireRangeArgs.func6.1()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:753 +0x604
  github.com/coder/coder/v2/cli.(*RootCmd).Server.WriteConfigMW.func5.1()
      /home/coder/coder/cli/server.go:1316 +0x354
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:499 +0x2cab
  github.com/coder/serpent.(*Invocation).run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:419 +0x16b8
  github.com/coder/serpent.(*Invocation).Run()
      /home/coder/.local/go/pkg/mod/github.com/coder/serpent@v0.10.0/command.go:594 +0x344
  github.com/coder/coder/v2/cli/clitest.StartWithWaiter.func1()
      /home/coder/coder/cli/clitest/clitest.go:253 +0x144
==================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@defelmnq defelmnq

Labels
flake
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: resolve flake test on manager coder/coder
3 participants
@mafredri @mtojek @defelmnq