Skip to content

fix: fix tailnet remoteCoordination to wait for server #14666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

spikecurtis
Copy link
Contributor

@spikecurtis spikecurtis commented Sep 13, 2024

Fixes #12560

When gracefully disconnecting from the coordinator, we would send the Disconnect message and then close the dRPC stream. However, closing the dRPC stream can cause the server not to process the Disconnect message, since we use the stream context in a select while sending it to the coordinator.

This is a product bug uncovered by the flake, and probably results in us failing graceful disconnect some minority of the time.

Instead, the remoteCoordination (and inMemoryCoordination for consistency) should send the Disconnect message and then wait for the coordinator to hang up (on some graceful disconnect timer, in the form of a context).

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @spikecurtis and the rest of your teammates on Graphite Graphite

@spikecurtis spikecurtis requested a review from sreya September 13, 2024 10:06
@spikecurtis spikecurtis marked this pull request as ready for review September 13, 2024 10:07
@spikecurtis spikecurtis force-pushed the spike/tac-disconnects-flake branch from 2b9d09d to 540d7eb Compare September 13, 2024 10:11
Copy link
Collaborator

@sreya sreya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 for the comments

@spikecurtis spikecurtis merged commit 2df9a3e into main Sep 16, 2024
27 checks passed
@spikecurtis spikecurtis deleted the spike/tac-disconnects-flake branch September 16, 2024 05:24
@github-actions github-actions bot locked and limited conversation to collaborators Sep 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test Flake: TestTailnetAPIConnector_Disconnects
2 participants