Skip to content

Commit 53ba361

Browse files
ethanndicksonethanndickson
authored
feat(cli): use coder connect in coder ssh --stdio, if available (#17572)
Closes coder/vscode-coder#447 Closes coder/jetbrains-coder#543 Closes coder/coder-jetbrains-toolbox#21 This PR adds Coder Connect support to `coder ssh --stdio`. When connecting to a workspace, if `--force-new-tunnel` is not passed, the CLI will first do a DNS lookup for `<agent>.<workspace>.<owner>.<hostname-suffix>`. If an IP address is returned, and it's within the Coder service prefix, the CLI will not create a new tailnet connection to the workspace, and instead dial the SSH server running on port 22 on the workspace directly over TCP. This allows IDE extensions to use the Coder Connect tunnel, without requiring any modifications to the extensions themselves. Additionally, `using_coder_connect` is added to the `sshNetworkStats` file, which the VS Code extension (and maybe Jetbrains?) will be able to read, and indicate to the user that they are using Coder Connect. One advantage of this approach is that running `coder ssh --stdio` on an offline workspace with Coder Connect enabled will have the CLI wait for the workspace to build, the agent to connect (and optionally, for the startup scripts to finish), before finally connecting using the Coder Connect tunnel. As a result, `coder ssh --stdio` has the overhead of looking up the workspace and agent, and checking if they are running. On my device, this meant `coder ssh --stdio <workspace>` was approximately a second slower than just connecting to the workspace directly using `ssh <workspace>.coder` (I would assume anyone serious about their Coder Connect usage would know to just do the latter anyway). To ensure this doesn't come at a significant performance cost, I've also benchmarked this PR. <details> <summary>Benchmark</summary> ## Methodology All tests were completed on `dev.coder.com`, where a Linux workspace running in AWS `us-west1` was created. The machine running Coder Desktop (the 'client') was a Windows VM running in the same AWS region and VPC as the workspace. To test the performance of specifically the SSH connection, a port was forwarded between the client and workspace using: ``` ssh -p 22 -L7001:localhost:7001 <host> ``` where `host` was either an alias for an SSH ProxyCommand that called `coder ssh`, or a Coder Connect hostname. For latency, [`tcping`](https://www.elifulkerson.com/projects/tcping.php) was used against the forwarded port: ``` tcping -n 100 localhost 7001 ``` For throughput, [`iperf3`](https://iperf.fr/iperf-download.php) was used: ``` iperf3 -c localhost -p 7001 ``` where an `iperf3` server was running on the workspace on port 7001. ## Test Cases ### Testcase 1: `coder ssh` `ProxyCommand` that bicopies from Coder Connect This case tests the implementation in this PR, such that we can write a config like: ``` Host codercliconnect ProxyCommand /path/to/coder ssh --stdio workspace ``` With Coder Connect enabled, `ssh -p 22 -L7001:localhost:7001 codercliconnect` will use the Coder Connect tunnel. The results were as follows: **Throughput, 10 tests, back to back:** - Average throughput across all tests: 788.20 Mbits/sec - Minimum average throughput: 731 Mbits/sec - Maximum average throughput: 871 Mbits/sec - Standard Deviation: 38.88 Mbits/sec **Latency, 100 RTTs:** - Average: 0.369ms - Minimum: 0.290ms - Maximum: 0.473ms ### Testcase 2: `ssh` dialing Coder Connect directly without a `ProxyCommand` This is what we assume to be the 'best' way to use Coder Connect **Throughput, 10 tests, back to back:** - Average throughput across all tests: 789.50 Mbits/sec - Minimum average throughput: 708 Mbits/sec - Maximum average throughput: 839 Mbits/sec - Standard Deviation: 39.98 Mbits/sec **Latency, 100 RTTs:** - Average: 0.369ms - Minimum: 0.267ms - Maximum: 0.440ms ### Testcase 3: `coder ssh` `ProxyCommand` that creates its own Tailnet connection in-process This is what normally happens when you run `coder ssh`: **Throughput, 10 tests, back to back:** - Average throughput across all tests: 610.20 Mbits/sec - Minimum average throughput: 569 Mbits/sec - Maximum average throughput: 664 Mbits/sec - Standard Deviation: 27.29 Mbits/sec **Latency, 100 RTTs:** - Average: 0.335ms - Minimum: 0.262ms - Maximum: 0.452ms ## Analysis Performing a two-tailed, unpaired t-test against the throughput of testcases 1 and 2, we find a P value of `0.9450`. This suggests the difference between the data sets is not statistically significant. In other words, there is a 94.5% chance that the difference between the data sets is due to chance. ## Conclusion From the t-test, and by comparison to the status quo (regular `coder ssh`, which uses gvisor, and is noticeably slower), I think it's safe to say any impact on throughput or latency by the `ProxyCommand` performing a bicopy against Coder Connect is negligible. Users are very much unlikely to run into performance issues as a result of using Coder Connect via `coder ssh`, as implemented in this PR. Less scientifically, I ran these same tests on my home network with my Sydney workspace, and both throughput and latency were consistent across testcases 1 and 2. </details>
1 parent 70ea678 commit 53ba361

File tree

5 files changed

+359
-49
lines changed

5 files changed

+359
-49
lines changed

‎cli/ssh.go

+123-9
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import (
88
"fmt"
99
"io"
1010
"log"
11+
"net"
1112
"net/http"
1213
"net/url"
1314
"os"
@@ -66,6 +67,7 @@ func (r *RootCmd) ssh() *serpent.Command {
6667
stdio bool
6768
hostPrefix string
6869
hostnameSuffix string
70+
forceNewTunnel bool
6971
forwardAgent bool
7072
forwardGPG bool
7173
identityAgent string
@@ -85,6 +87,7 @@ func (r *RootCmd) ssh() *serpent.Command {
8587
containerUser string
8688
)
8789
client := new(codersdk.Client)
90+
wsClient := workspacesdk.New(client)
8891
cmd := &serpent.Command{
8992
Annotations: workspaceCommand,
9093
Use: "ssh <workspace>",
@@ -203,14 +206,14 @@ func (r *RootCmd) ssh() *serpent.Command {
203206
parsedEnv = append(parsedEnv, [2]string{k, v})
204207
}
205208

206-
deploymentSSHConfig := codersdk.SSHConfigResponse{
209+
cliConfig := codersdk.SSHConfigResponse{
207210
HostnamePrefix: hostPrefix,
208211
HostnameSuffix: hostnameSuffix,
209212
}
210213

211214
workspace, workspaceAgent, err := findWorkspaceAndAgentByHostname(
212215
ctx, inv, client,
213-
inv.Args[0], deploymentSSHConfig, disableAutostart)
216+
inv.Args[0], cliConfig, disableAutostart)
214217
if err != nil {
215218
return err
216219
}
@@ -275,10 +278,44 @@ func (r *RootCmd) ssh() *serpent.Command {
275278
return err
276279
}
277280

281+
// If we're in stdio mode, check to see if we can use Coder Connect.
282+
// We don't support Coder Connect over non-stdio coder ssh yet.
283+
if stdio && !forceNewTunnel {
284+
connInfo, err := wsClient.AgentConnectionInfoGeneric(ctx)
285+
if err != nil {
286+
return xerrors.Errorf("get agent connection info: %w", err)
287+
}
288+
coderConnectHost := fmt.Sprintf("%s.%s.%s.%s",
289+
workspaceAgent.Name, workspace.Name, workspace.OwnerName, connInfo.HostnameSuffix)
290+
exists, _ := workspacesdk.ExistsViaCoderConnect(ctx, coderConnectHost)
291+
if exists {
292+
defer cancel()
293+
294+
if networkInfoDir != "" {
295+
if err := writeCoderConnectNetInfo(ctx, networkInfoDir); err != nil {
296+
logger.Error(ctx, "failed to write coder connect net info file", slog.Error(err))
297+
}
298+
}
299+
300+
stopPolling := tryPollWorkspaceAutostop(ctx, client, workspace)
301+
defer stopPolling()
302+
303+
usageAppName := getUsageAppName(usageApp)
304+
if usageAppName != "" {
305+
closeUsage := client.UpdateWorkspaceUsageWithBodyContext(ctx, workspace.ID, codersdk.PostWorkspaceUsageRequest{
306+
AgentID: workspaceAgent.ID,
307+
AppName: usageAppName,
308+
})
309+
defer closeUsage()
310+
}
311+
return runCoderConnectStdio(ctx, fmt.Sprintf("%s:22", coderConnectHost), stdioReader, stdioWriter, stack)
312+
}
313+
}
314+
278315
if r.disableDirect {
279316
_, _ = fmt.Fprintln(inv.Stderr, "Direct connections disabled.")
280317
}
281-
conn, err := workspacesdk.New(client).
318+
conn, err := wsClient.
282319
DialAgent(ctx, workspaceAgent.ID, &workspacesdk.DialAgentOptions{
283320
Logger: logger,
284321
BlockEndpoints: r.disableDirect,
@@ -660,6 +697,12 @@ func (r *RootCmd) ssh() *serpent.Command {
660697
Value: serpent.StringOf(&containerUser),
661698
Hidden: true, // Hidden until this features is at least in beta.
662699
},
700+
{
701+
Flag: "force-new-tunnel",
702+
Description: "Force the creation of a new tunnel to the workspace, even if the Coder Connect tunnel is available.",
703+
Value: serpent.BoolOf(&forceNewTunnel),
704+
Hidden: true,
705+
},
663706
sshDisableAutostartOption(serpent.BoolOf(&disableAutostart)),
664707
}
665708
return cmd
@@ -1372,12 +1415,13 @@ func setStatsCallback(
13721415
}
13731416

13741417
type sshNetworkStats struct {
1375-
P2P bool `json:"p2p"`
1376-
Latency float64 `json:"latency"`
1377-
PreferredDERP string `json:"preferred_derp"`
1378-
DERPLatency map[string]float64 `json:"derp_latency"`
1379-
UploadBytesSec int64 `json:"upload_bytes_sec"`
1380-
DownloadBytesSec int64 `json:"download_bytes_sec"`
1418+
P2P bool `json:"p2p"`
1419+
Latency float64 `json:"latency"`
1420+
PreferredDERP string `json:"preferred_derp"`
1421+
DERPLatency map[string]float64 `json:"derp_latency"`
1422+
UploadBytesSec int64 `json:"upload_bytes_sec"`
1423+
DownloadBytesSec int64 `json:"download_bytes_sec"`
1424+
UsingCoderConnect bool `json:"using_coder_connect"`
13811425
}
13821426

13831427
func collectNetworkStats(ctx context.Context, agentConn *workspacesdk.AgentConn, start, end time.Time, counts map[netlogtype.Connection]netlogtype.Counts) (*sshNetworkStats, error) {
@@ -1448,6 +1492,76 @@ func collectNetworkStats(ctx context.Context, agentConn *workspacesdk.AgentConn,
14481492
}, nil
14491493
}
14501494

1495+
type coderConnectDialerContextKey struct{}
1496+
1497+
type coderConnectDialer interface {
1498+
DialContext(ctx context.Context, network, addr string) (net.Conn, error)
1499+
}
1500+
1501+
func WithTestOnlyCoderConnectDialer(ctx context.Context, dialer coderConnectDialer) context.Context {
1502+
return context.WithValue(ctx, coderConnectDialerContextKey{}, dialer)
1503+
}
1504+
1505+
func testOrDefaultDialer(ctx context.Context) coderConnectDialer {
1506+
dialer, ok := ctx.Value(coderConnectDialerContextKey{}).(coderConnectDialer)
1507+
if !ok || dialer == nil {
1508+
return &net.Dialer{}
1509+
}
1510+
return dialer
1511+
}
1512+
1513+
func runCoderConnectStdio(ctx context.Context, addr string, stdin io.Reader, stdout io.Writer, stack *closerStack) error {
1514+
dialer := testOrDefaultDialer(ctx)
1515+
conn, err := dialer.DialContext(ctx, "tcp", addr)
1516+
if err != nil {
1517+
return xerrors.Errorf("dial coder connect host: %w", err)
1518+
}
1519+
if err := stack.push("tcp conn", conn); err != nil {
1520+
return err
1521+
}
1522+
1523+
agentssh.Bicopy(ctx, conn, &StdioRwc{
1524+
Reader: stdin,
1525+
Writer: stdout,
1526+
})
1527+
1528+
return nil
1529+
}
1530+
1531+
type StdioRwc struct {
1532+
io.Reader
1533+
io.Writer
1534+
}
1535+
1536+
func (*StdioRwc) Close() error {
1537+
return nil
1538+
}
1539+
1540+
func writeCoderConnectNetInfo(ctx context.Context, networkInfoDir string) error {
1541+
fs, ok := ctx.Value("fs").(afero.Fs)
1542+
if !ok {
1543+
fs = afero.NewOsFs()
1544+
}
1545+
// The VS Code extension obtains the PID of the SSH process to
1546+
// find the log file associated with a SSH session.
1547+
//
1548+
// We get the parent PID because it's assumed `ssh` is calling this
1549+
// command via the ProxyCommand SSH option.
1550+
networkInfoFilePath := filepath.Join(networkInfoDir, fmt.Sprintf("%d.json", os.Getppid()))
1551+
stats := &sshNetworkStats{
1552+
UsingCoderConnect: true,
1553+
}
1554+
rawStats, err := json.Marshal(stats)
1555+
if err != nil {
1556+
return xerrors.Errorf("marshal network stats: %w", err)
1557+
}
1558+
err = afero.WriteFile(fs, networkInfoFilePath, rawStats, 0o600)
1559+
if err != nil {
1560+
return xerrors.Errorf("write network stats: %w", err)
1561+
}
1562+
return nil
1563+
}
1564+
14511565
// Converts workspace name input to owner/workspace.agent format
14521566
// Possible valid input formats:
14531567
// workspace

‎cli/ssh_internal_test.go

+85
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,17 @@ package cli
33
import (
44
"context"
55
"fmt"
6+
"io"
7+
"net"
68
"net/url"
79
"sync"
810
"testing"
911
"time"
1012

13+
gliderssh "github.com/gliderlabs/ssh"
1114
"github.com/stretchr/testify/assert"
1215
"github.com/stretchr/testify/require"
16+
"golang.org/x/crypto/ssh"
1317
"golang.org/x/xerrors"
1418

1519
"cdr.dev/slog"
@@ -220,6 +224,87 @@ func TestCloserStack_Timeout(t *testing.T) {
220224
testutil.TryReceive(ctx, t, closed)
221225
}
222226

227+
func TestCoderConnectStdio(t *testing.T) {
228+
t.Parallel()
229+
230+
ctx := testutil.Context(t, testutil.WaitShort)
231+
logger := slogtest.Make(t, nil).Leveled(slog.LevelDebug)
232+
stack := newCloserStack(ctx, logger, quartz.NewMock(t))
233+
234+
clientOutput, clientInput := io.Pipe()
235+
serverOutput, serverInput := io.Pipe()
236+
defer func() {
237+
for _, c := range []io.Closer{clientOutput, clientInput, serverOutput, serverInput} {
238+
_ = c.Close()
239+
}
240+
}()
241+
242+
server := newSSHServer("127.0.0.1:0")
243+
ln, err := net.Listen("tcp", server.server.Addr)
244+
require.NoError(t, err)
245+
246+
go func() {
247+
_ = server.Serve(ln)
248+
}()
249+
t.Cleanup(func() {
250+
_ = server.Close()
251+
})
252+
253+
stdioDone := make(chan struct{})
254+
go func() {
255+
err = runCoderConnectStdio(ctx, ln.Addr().String(), clientOutput, serverInput, stack)
256+
assert.NoError(t, err)
257+
close(stdioDone)
258+
}()
259+
260+
conn, channels, requests, err := ssh.NewClientConn(&testutil.ReaderWriterConn{
261+
Reader: serverOutput,
262+
Writer: clientInput,
263+
}, "", &ssh.ClientConfig{
264+
// #nosec
265+
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
266+
})
267+
require.NoError(t, err)
268+
defer conn.Close()
269+
270+
sshClient := ssh.NewClient(conn, channels, requests)
271+
session, err := sshClient.NewSession()
272+
require.NoError(t, err)
273+
defer session.Close()
274+
275+
// We're not connected to a real shell
276+
err = session.Run("")
277+
require.NoError(t, err)
278+
err = sshClient.Close()
279+
require.NoError(t, err)
280+
_ = clientOutput.Close()
281+
282+
<-stdioDone
283+
}
284+
285+
type sshServer struct {
286+
server *gliderssh.Server
287+
}
288+
289+
func newSSHServer(addr string) *sshServer {
290+
return &sshServer{
291+
server: &gliderssh.Server{
292+
Addr: addr,
293+
Handler: func(s gliderssh.Session) {
294+
_, _ = io.WriteString(s.Stderr(), "Connected!")
295+
},
296+
},
297+
}
298+
}
299+
300+
func (s *sshServer) Serve(ln net.Listener) error {
301+
return s.server.Serve(ln)
302+
}
303+
304+
func (s *sshServer) Close() error {
305+
return s.server.Close()
306+
}
307+
223308
type fakeCloser struct {
224309
closes *[]*fakeCloser
225310
err error

0 commit comments

Comments
 (0)