Skip to content

docs: reword latency explanation for HA #8860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 15, 2023
Merged

docs: reword latency explanation for HA #8860

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions docs/admin/high-availability.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,27 @@ endpoint. [GCP](https://cloud.google.com/sql/docs/postgres/high-availability), [
and other cloud vendors offer fully-managed HA Postgres services that pair
nicely with Coder.

For Coder to operate correctly, every node must be within 10ms of each other
and Postgres. We make a best-effort attempt to warn the user when inter-Coder
latency is too high, but if requests start dropping, this is one metric to investigate.
For Coder to operate correctly, Coderd instances should have low-latency connections
to each other so that they can effectively relay traffic between users and workspaces no
matter which Coderd instance users or workspaces connect to. We make a best-effort attempt
to warn the user when inter-Coderd latency is too high, but if requests start dropping, this
Copy link
Member

@bpmct bpmct Aug 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we consider "too high" for the health check? We might as well document this threshold as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I'm not sure what this is referring to. @kylecarbs do you know?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bpmct this text is not being changed by this PR, just whitespace. I understand the desire to get better documentation around the health check, but it's scope creep for this PR. Can we agree on whether or not the changes this PR is making are appropriate so it can get merged?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I misunderstood it as a text change.

is one metric to investigate.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: note specific metrics, link to https://coder.com/docs/v2/latest/admin/prometheus

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we expose this over Prometheus... @kylecarbs ?


We also recommend that you deploy all Coderd instances such that they have low-latency
connections to Postgres. Coderd often makes several database round-trips while processing
a single API request, so prioritizing low-latency between Coderd and Postgres is more important
than low-latency between users and Coderd.

Note that this latency requirement applies _only_ to Coder services. Coder will
operate correctly even with few seconds of latency on
workspace <-> Coder and user <-> Coder connections.
operate correctly even with few seconds of latency on workspace <-> Coder and user <-> Coder
connections.

## Setup

Coder automatically enters HA mode when multiple instances simultaneously connect
to the same Postgres endpoint.

HA brings one configuration variable to set in each Coder
HA brings one configuration variable to set in each Coderd
node: `CODER_DERP_SERVER_RELAY_URL`. The HA nodes use these URLs to communicate
with each other. Inter-node communication is only required while using the
embedded relay (default). If you're using [custom relays](../networking/index.md#custom-relays), Coder ignores `CODER_DERP_SERVER_RELAY_URL` since Postgres is the sole rendezvous for the Coder nodes.
Expand Down