Skip to content

fix: avoid race between replicas on start #12344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Feb 28, 2024
Merged

fix: avoid race between replicas on start #12344

merged 3 commits into from
Feb 28, 2024

Conversation

deansheather
Copy link
Member

DERP mesh key setup would do a SELECT and then an INSERT on failure, without a lock. During some testing with multiple replicas, I managed to cause a replica to crash due to them initializing simultaneously.

Started HTTP listener at http://0.0.0.0:7080

View the Web UI: http://192.168.208.3:7080
2024-02-28 11:25:00.457 [info]  pubsub: pubsub dialing postgres  network=tcp  address=database:5432  timeout_ms=0
2024-02-28 11:25:00.457 [info]  pubsub: pubsub postgres TCP connection established  network=tcp  address=database:5432  timeout_ms=0  elapsed_ms=0
2024-02-28 11:25:00.462 [info]  pubsub: pubsub connected to postgres
2024-02-28 11:25:00.462 [info]  pubsub: pubsub has started
2024-02-28 11:25:02.307 [info]  pubsub: pubsub is closing
2024-02-28 11:25:02.307 [info]  pubsub: pubsub listen stopped receiving notify
Encountered an error running "coder server"
create coder API: insert mesh key: pq: duplicate key value violates unique constraint "site_configs_key_key"

johnstcn
johnstcn reviewed Feb 28, 2024
View reviewed changes
coderd/database/dbmem/dbmem.go
Comment on lines +1764 to +1766
if q.derpMeshKey == "" {
return "", sql.ErrNoRows
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 #justdbmemthings

@deansheather deansheather enabled auto-merge (squash) February 28, 2024 16:13
@deansheather deansheather merged commit bedd2c5 into main Feb 28, 2024
@deansheather deansheather deleted the dean/use-lock-during-enterprise-start branch February 28, 2024 16:14
@github-actions github-actions bot locked and limited conversation to collaborators Feb 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@johnstcn johnstcn

Assignees

@deansheather deansheather

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@deansheather @johnstcn