Skip to content

Commit 5631239

Browse files
ethanndicksonethanndickson
committed
port #14548 
1 parent 422e10b commit 5631239

File tree

3 files changed

+129
-35
lines changed

3 files changed

+129
-35
lines changed

‎docs/admin/networking/README.md

-35
Original file line numberDiff line numberDiff line change
@@ -171,41 +171,6 @@ with security policies. In these cases, pass the `--browser-only` flag to
171171
With browser-only connections, developers can only connect to their workspaces
172172
via the web terminal and [web IDEs](../../user-guides/workspace-access/web-ides.md).
173173

174-
## Troubleshooting
175-
176-
The `coder ping -v <workspace>` will ping a workspace and return debug logs for
177-
the connection. We recommend running this command and inspecting the output when
178-
debugging SSH connections to a workspace. For example:
179-
180-
```console
181-
$ coder ping -v my-workspace
182-
183-
2023-06-21 17:50:22.412 [debu] wgengine: ping(fd7a:115c:a1e0:49d6:b259:b7ac:b1b2:48f4): sending disco ping to [cFYPo] ...
184-
pong from my-workspace proxied via DERP(Denver) in 90ms
185-
2023-06-21 17:50:22.503 [debu] wgengine: magicsock: closing connection to derp-13 (conn-close), age 5s
186-
2023-06-21 17:50:22.503 [debu] wgengine: magicsock: 0 active derp conns
187-
2023-06-21 17:50:22.504 [debu] wgengine: wg: [v2] Routine: receive incoming v6 - stopped
188-
2023-06-21 17:50:22.504 [debu] wgengine: wg: [v2] Device closed
189-
```
190-
191-
The `coder speedtest <workspace>` command measures user <-> workspace
192-
throughput. E.g.:
193-
194-
```
195-
$ coder speedtest dev
196-
29ms via coder
197-
Starting a 5s download test...
198-
INTERVAL TRANSFER BANDWIDTH
199-
0.00-1.00 sec 630.7840 MBits 630.7404 Mbits/sec
200-
1.00-2.00 sec 913.9200 MBits 913.8106 Mbits/sec
201-
2.00-3.00 sec 943.1040 MBits 943.0399 Mbits/sec
202-
3.00-4.00 sec 933.3760 MBits 933.2143 Mbits/sec
203-
4.00-5.00 sec 848.8960 MBits 848.7019 Mbits/sec
204-
5.00-5.02 sec 13.5680 MBits 828.8189 Mbits/sec
205-
----------------------------------------------------
206-
0.00-5.02 sec 4283.6480 MBits 853.8217 Mbits/sec
207-
```
208-
209174
## Up next
210175

211176
- Learn about [Port Forwarding](./port-forwarding.md)

‎docs/admin/networking/troubleshooting.md

+124
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Troubleshooting
2+
3+
`coder ping <workspace>` will ping the workspace agent and print diagnostics on
4+
the state of the connection. These diagnostics are created by inspecting both
5+
the client and agent network configurations, and provide insights into why a
6+
direct connection may be impeded, or why the quality of one might be degraded.
7+
8+
The `-v/--verbose` flag can be appended to the command to print client debug
9+
logs.
10+
11+
```console
12+
$ coder ping dev
13+
pong from workspace proxied via DERP(Council Bluffs, Iowa) in 42ms
14+
pong from workspace proxied via DERP(Council Bluffs, Iowa) in 41ms
15+
pong from workspace proxied via DERP(Council Bluffs, Iowa) in 39ms
16+
✔ preferred DERP region: 999 (Council Bluffs, Iowa)
17+
✔ sent local data to Coder networking coordinator
18+
✔ received remote agent data from Coder networking coordinator
19+
preferred DERP region: 999 (Council Bluffs, Iowa)
20+
endpoints: x.x.x.x:46433, x.x.x.x:46433, x.x.x.x:46433
21+
✔ Wireguard handshake 11s ago
22+
23+
❗ You are connected via a DERP relay, not directly (p2p)
24+
Possible client-side issues with direct connection:
25+
- Network interface utun0 has MTU 1280, (less than 1378), which may degrade the quality of direct connections
26+
27+
Possible agent-side issues with direct connection:
28+
- Agent is potentially behind a hard NAT, as multiple endpoints were retrieved from different STUN servers
29+
- Agent IP address is within an AWS range (AWS uses hard NAT)
30+
```
31+
32+
## Common Problems with Direct Connections
33+
34+
### Disabled Deployment-wide
35+
36+
Direct connections can be disabled at the deployment level by setting the
37+
`CODER_BLOCK_DIRECT` environment variable or the `--block-direct-connections`
38+
flag on the server. When set, this will be reflected in the output of
39+
`coder ping`.
40+
41+
### UDP Blocked
42+
43+
Some corporate firewalls block UDP traffic. Direct connections require UDP
44+
traffic to be allowed between the client and agent, as well as between the
45+
client/agent and STUN servers in most cases. `coder ping` will indicate if
46+
either the Coder agent or client had issues sending or receiving UDP packets to
47+
STUN servers.
48+
49+
If this is the case, you may need to add exceptions to the firewall to allow UDP
50+
for Coder workspaces, clients, and STUN servers.
51+
52+
### Endpoint-Dependent NAT (Hard NAT)
53+
54+
Hard NATs prevent public endpoints gathered from STUN servers from being used by
55+
the peer to establish a direct connection. Typically, if only one side of the
56+
connection is behind a hard NAT, direct connections can still be established
57+
easily. However, if both sides are behind hard NATs, direct connections may take
58+
longer to establish or may not be possible at all.
59+
60+
`coder ping` will indicate if it's possible the client or agent is behind a hard
61+
NAT.
62+
63+
Learn more about [STUN and NAT](./stun.md).
64+
65+
### No STUN Servers
66+
67+
If there are no STUN servers available within a deployment's DERP MAP, direct
68+
connections may not be possible. Notable exceptions are if the client and agent
69+
are on the same network, or if either is able to use UPnP instead of STUN to
70+
resolve the public IP of the other. `coder ping` will indicate if no STUN
71+
servers were found.
72+
73+
### Endpoint Firewalls
74+
75+
Direct connections may also be impeded if one side is behind a hard NAT and the
76+
other is running a firewall that blocks ingress traffic from unknown 5-tuples
77+
(Protocol, Source IP, Source Port, Destination IP, Destination Port).
78+
79+
If this is suspected, you may need to add an exception for Coder to the
80+
firewall, or reconfigure the hard NAT.
81+
82+
### VPNs
83+
84+
If a VPN is the default route for all IP traffic, it may interfere with the
85+
ability for clients and agents to form direct connections. This happens if the
86+
NAT does not permit traffic to be
87+
['hairpinned'](./stun.md#3-direct-connections-with-vpn-and-nat-hairpinning) from
88+
the public IP address of the NAT (determined via STUN) to the internal IP
89+
address of the agent.
90+
91+
If this is the case, you may need to add exceptions to the VPN for Coder, modify
92+
the NAT configuration, or deploy an internal STUN server.
93+
94+
### Low MTU
95+
96+
If a network interface on the side of either the client or agent has an MTU
97+
smaller than 1378, any direct connections form may have degraded quality or
98+
performance, as IP packets are fragmented. `coder ping` will indicate if this is
99+
the case by inspecting network interfaces on both the client and the workspace
100+
agent.
101+
102+
If another interface cannot be used, and the MTU cannot be changed, you may need
103+
to disable direct connections, and relay all traffic via DERP instead, which
104+
will not be affected by the low MTU.
105+
106+
## Throughput
107+
108+
The `coder speedtest <workspace>` command measures the throughput between the
109+
client and the workspace agent.
110+
111+
```console
112+
$ coder speedtest workspace
113+
29ms via coder
114+
Starting a 5s download test...
115+
INTERVAL TRANSFER BANDWIDTH
116+
0.00-1.00 sec 630.7840 MBits 630.7404 Mbits/sec
117+
1.00-2.00 sec 913.9200 MBits 913.8106 Mbits/sec
118+
2.00-3.00 sec 943.1040 MBits 943.0399 Mbits/sec
119+
3.00-4.00 sec 933.3760 MBits 933.2143 Mbits/sec
120+
4.00-5.00 sec 848.8960 MBits 848.7019 Mbits/sec
121+
5.00-5.02 sec 13.5680 MBits 828.8189 Mbits/sec
122+
----------------------------------------------------
123+
0.00-5.02 sec 4283.6480 MBits 853.8217 Mbits/sec
124+
```

‎docs/manifest.json

+5
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,11 @@
392392
"title": "STUN and NAT",
393393
"description": "Learn how to forward ports in Coder",
394394
"path": "./admin/networking/stun.md"
395+
},
396+
{
397+
"title": "Troubleshooting",
398+
"description": "Troubleshoot networking issues in Coder",
399+
"path": "./networking/troubleshooting.md"
395400
}
396401
]
397402
},

0 commit comments

Comments
 (0)