Skip to content

Commit 4c03b3a

Browse files
ethanndicksonethanndickson
committed
docs: add networking troubleshooting page
1 parent 4eac2ac commit 4c03b3a

File tree

5 files changed

+120
-37
lines changed

5 files changed

+120
-37
lines changed

‎cli/cliui/agent.go

+4-1
Original file line numberDiff line numberDiff line change
@@ -309,7 +309,7 @@ func PeerDiagnostics(w io.Writer, d tailnet.PeerDiagnostics) {
309309
_, _ = fmt.Fprint(w, "✘ not connected to DERP\n")
310310
}
311311
if d.SentNode {
312-
_, _ = fmt.Fprint(w, "✔ sent local data to Coder networking coodinator\n")
312+
_, _ = fmt.Fprint(w, "✔ sent local data to Coder networking coordinator\n")
313313
} else {
314314
_, _ = fmt.Fprint(w, "✘ have not sent local data to Coder networking coordinator\n")
315315
}
@@ -381,6 +381,9 @@ func (d ConnDiags) Write(w io.Writer) {
381381
_, _ = fmt.Fprintf(w, " - %s\n\n", msg)
382382
}
383383
}
384+
if !d.PingP2P || d.Verbose || len(client) > 0 || len(agent) > 0 {
385+
_, _ = fmt.Fprintln(w, "For more information, see https://coder.com/docs/networking/troubleshooting")
386+
}
384387
}
385388

386389
func (d ConnDiags) splitDiagnostics() (general, client, agent []string) {

‎cli/cliui/agent_test.go

+1-1
Original file line numberDiff line numberDiff line change
@@ -533,7 +533,7 @@ func TestPeerDiagnostics(t *testing.T) {
533533
LastWireguardHandshake: time.Time{},
534534
},
535535
want: []*regexp.Regexp{
536-
regexp.MustCompile(`^✔ sent local data to Coder networking coodinator$`),
536+
regexp.MustCompile(`^✔ sent local data to Coder networking coordinator$`),
537537
},
538538
},
539539
{

‎docs/manifest.json

+5
Original file line numberDiff line numberDiff line change
@@ -346,6 +346,11 @@
346346
"title": "STUN and NAT",
347347
"description": "Learn how Coder establishes direct connections",
348348
"path": "./networking/stun.md"
349+
},
350+
{
351+
"title": "Troubleshooting",
352+
"description": "Troubleshoot networking issues in Coder",
353+
"path": "./networking/troubleshooting.md"
349354
}
350355
]
351356
},

‎docs/networking/index.md

+1-35
Original file line numberDiff line numberDiff line change
@@ -169,41 +169,7 @@ with security policies. In these cases, pass the `--browser-only` flag to
169169
With browser-only connections, developers can only connect to their workspaces
170170
via the web terminal and [web IDEs](../ides/web-ides.md).
171171

172-
## Troubleshooting
173-
174-
The `coder ping -v <workspace>` will ping a workspace and return debug logs for
175-
the connection. We recommend running this command and inspecting the output when
176-
debugging SSH connections to a workspace. For example:
177-
178-
```console
179-
$ coder ping -v my-workspace
180-
181-
2023-06-21 17:50:22.412 [debu] wgengine: ping(fd7a:115c:a1e0:49d6:b259:b7ac:b1b2:48f4): sending disco ping to [cFYPo] ...
182-
pong from my-workspace proxied via DERP(Denver) in 90ms
183-
2023-06-21 17:50:22.503 [debu] wgengine: magicsock: closing connection to derp-13 (conn-close), age 5s
184-
2023-06-21 17:50:22.503 [debu] wgengine: magicsock: 0 active derp conns
185-
2023-06-21 17:50:22.504 [debu] wgengine: wg: [v2] Routine: receive incoming v6 - stopped
186-
2023-06-21 17:50:22.504 [debu] wgengine: wg: [v2] Device closed
187-
```
188-
189-
The `coder speedtest <workspace>` command measures user <-> workspace
190-
throughput. E.g.:
191-
192-
```
193-
$ coder speedtest dev
194-
29ms via coder
195-
Starting a 5s download test...
196-
INTERVAL TRANSFER BANDWIDTH
197-
0.00-1.00 sec 630.7840 MBits 630.7404 Mbits/sec
198-
1.00-2.00 sec 913.9200 MBits 913.8106 Mbits/sec
199-
2.00-3.00 sec 943.1040 MBits 943.0399 Mbits/sec
200-
3.00-4.00 sec 933.3760 MBits 933.2143 Mbits/sec
201-
4.00-5.00 sec 848.8960 MBits 848.7019 Mbits/sec
202-
5.00-5.02 sec 13.5680 MBits 828.8189 Mbits/sec
203-
----------------------------------------------------
204-
0.00-5.02 sec 4283.6480 MBits 853.8217 Mbits/sec
205-
```
206-
207172
## Up next
208173

209174
- Learn about [Port Forwarding](./port-forwarding.md)
175+
- Troubleshoot [Networking Issues](./troubleshooting.md)

‎docs/networking/troubleshooting.md

+109
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Troubleshooting
2+
3+
The `coder ping <workspace>` command not only pings a workspace, but also prints
4+
diagnostics on the state of the connection. Appending the verbose flag
5+
(`-v/--verbose`) to the command will also print client debug logs. These
6+
diagnostics are created by inspecting both the client and agent network
7+
configurations, and provide insights into why a direct connection may be
8+
impeded, or why the quality of one might be degraded.
9+
10+
```console
11+
$ coder ping dev
12+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
13+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
14+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
15+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 210ms
16+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
17+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
18+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 210ms
19+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
20+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
21+
pong from dev proxied via DERP(Council Bluffs, Iowa) in 209ms
22+
✔ preferred DERP region: 10015 (Australia Fly.io (Sydney))
23+
✔ sent local data to Coder networking coordinator
24+
✔ received remote agent data from Coder networking coordinator
25+
preferred DERP region: 999 (Council Bluffs, Iowa)
26+
endpoints: 204.16.241.141:46433, 172.17.0.1:46433, 172.20.0.6:46433
27+
✔ Wireguard handshake 11s ago
28+
29+
❗ You are connected via a DERP relay, not directly (p2p)
30+
Possible client-side issues with direct connection:
31+
- Network interface utun0 has MTU 1280, (less than 1378), which may degrade the quality of direct connections
32+
33+
Possible agent-side issues with direct connection:
34+
- Agent is potentially behind a hard NAT, as multiple endpoints were retrieved from different STUN servers
35+
- Agent IP address is within an AWS range (AWS uses hard NAT)
36+
```
37+
38+
## Common Problems
39+
40+
### Deployment-wide Direct Connection Blocking
41+
42+
Direct connections can be disabled at the deployment level by setting the
43+
`CODER_BLOCK_DIRECT` environment variable or the `coder server`
44+
`--block-direct-connections` flag. This forces all connections to be relayed via
45+
DERP, preventing any and all direct connections to workspace agents. If set,
46+
this will be reflected in the output of `coder ping`.
47+
48+
### Firewall blocking UDP
49+
50+
Some corporate firewalls block UDP traffic. Direct connections, and
51+
communicating with STUN servers to establish them, require UDP. `coder ping`
52+
will indicate if either the Coder agent or client is unable to communicate with
53+
any known STUN servers over UDP.
54+
55+
If this is the case, you may need to add exceptions to the firewall to allow UDP
56+
for Coder workspaces, clients, and STUN servers.
57+
58+
### Hard NAT
59+
60+
`coder ping` will indicate if it's possible the client or agent is behind a hard
61+
NAT. Hard NATs may prevent direct connections from being established,
62+
particularly if both sides are behind one. Direct connections may also be
63+
impeded if one side is behind a hard NAT and the other is running a firewall
64+
that blocks ingress traffic from unknown 5-tuples (Protocol, Source IP, Source
65+
Port, Destination IP, Destination Port). Learn more about
66+
[STUN and NAT](./stun.md).
67+
68+
### VPNs
69+
70+
If a VPN is the default route for all IP traffic, it may interfere with the
71+
ability for clients and agents to form direct connections. This happens if the
72+
NAT does not permit traffic to be
73+
['hairpinned'](./stun.md#3-direct-connections-with-vpn-and-nat-hairpinning) from
74+
the public IP address of the NAT (determined via STUN) to the internal IP
75+
address of the agent.
76+
77+
If this is the case, you may need to add exceptions to the VPN for Coder, modify
78+
the NAT configuration, or deploy an internal STUN server.
79+
80+
### Low MTU
81+
82+
If a network interface on the side of either the client or agent has an MTU
83+
smaller than 1378, any direct connections form may have degraded
84+
quality/performance, as IP packets are fragmented. `coder ping` will indicate if
85+
this is the case by inspecting client and agent interfaces.
86+
87+
If another interface cannot be used, and the MTU cannot be changed, you may have
88+
to disable direct connections, and relaying all traffic via DERP, which will not
89+
be affected by the low MTU.
90+
91+
## Throughput
92+
93+
The `coder speedtest <workspace>` command measures user <-> workspace
94+
throughput:
95+
96+
```console
97+
$ coder speedtest dev
98+
29ms via coder
99+
Starting a 5s download test...
100+
INTERVAL TRANSFER BANDWIDTH
101+
0.00-1.00 sec 630.7840 MBits 630.7404 Mbits/sec
102+
1.00-2.00 sec 913.9200 MBits 913.8106 Mbits/sec
103+
2.00-3.00 sec 943.1040 MBits 943.0399 Mbits/sec
104+
3.00-4.00 sec 933.3760 MBits 933.2143 Mbits/sec
105+
4.00-5.00 sec 848.8960 MBits 848.7019 Mbits/sec
106+
5.00-5.02 sec 13.5680 MBits 828.8189 Mbits/sec
107+
----------------------------------------------------
108+
0.00-5.02 sec 4283.6480 MBits 853.8217 Mbits/sec
109+
```

0 commit comments

Comments
 (0)