Skip to content

Commit 9f0a8f1

Browse files
authored
Merge branch 'main' into brett-14239/refactor-cli-roles-edit-command
2 parents c90838f + 184c1f0 commit 9f0a8f1

File tree

181 files changed

+12180
-979
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

181 files changed

+12180
-979
lines changed

.cursorrules

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# Cursor Rules
2+
3+
This project is called "Coder" - an application for managing remote development environments.
4+
5+
Coder provides a platform for creating, managing, and using remote development environments (also known as Cloud Development Environments or CDEs). It leverages Terraform to define and provision these environments, which are referred to as "workspaces" within the project. The system is designed to be extensible, secure, and provide developers with a seamless remote development experience.
6+
7+
# Core Architecture
8+
9+
The heart of Coder is a control plane that orchestrates the creation and management of workspaces. This control plane interacts with separate Provisioner processes over gRPC to handle workspace builds. The Provisioners consume workspace definitions and use Terraform to create the actual infrastructure.
10+
11+
The CLI package serves dual purposes - it can be used to launch the control plane itself and also provides client functionality for users to interact with an existing control plane instance. All user-facing frontend code is developed in TypeScript using React and lives in the `site/` directory.
12+
13+
The database layer uses PostgreSQL with SQLC for generating type-safe database code. Database migrations are carefully managed to ensure both forward and backward compatibility through paired `.up.sql` and `.down.sql` files.
14+
15+
# API Design
16+
17+
Coder's API architecture combines REST and gRPC approaches. The REST API is defined in `coderd/coderd.go` and uses Chi for HTTP routing. This provides the primary interface for the frontend and external integrations.
18+
19+
Internal communication with Provisioners occurs over gRPC, with service definitions maintained in `.proto` files. This separation allows for efficient binary communication with the components responsible for infrastructure management while providing a standard REST interface for human-facing applications.
20+
21+
# Network Architecture
22+
23+
Coder implements a secure networking layer based on Tailscale's Wireguard implementation. The `tailnet` package provides connectivity between workspace agents and clients through DERP (Designated Encrypted Relay for Packets) servers when direct connections aren't possible. This creates a secure overlay network allowing access to workspaces regardless of network topology, firewalls, or NAT configurations.
24+
25+
## Tailnet and DERP System
26+
27+
The networking system has three key components:
28+
29+
1. **Tailnet**: An overlay network implemented in the `tailnet` package that provides secure, end-to-end encrypted connections between clients, the Coder server, and workspace agents.
30+
31+
2. **DERP Servers**: These relay traffic when direct connections aren't possible. Coder provides several options:
32+
- A built-in DERP server that runs on the Coder control plane
33+
- Integration with Tailscale's global DERP infrastructure
34+
- Support for custom DERP servers for lower latency or offline deployments
35+
36+
3. **Direct Connections**: When possible, the system establishes peer-to-peer connections between clients and workspaces using STUN for NAT traversal. This requires both endpoints to send UDP traffic on ephemeral ports.
37+
38+
## Workspace Proxies
39+
40+
Workspace proxies (in the Enterprise edition) provide regional relay points for browser-based connections, reducing latency for geo-distributed teams. Key characteristics:
41+
42+
- Deployed as independent servers that authenticate with the Coder control plane
43+
- Relay connections for SSH, workspace apps, port forwarding, and web terminals
44+
- Do not make direct database connections
45+
- Managed through the `coder wsproxy` commands
46+
- Implemented primarily in the `enterprise/wsproxy/` package
47+
48+
# Agent System
49+
50+
The workspace agent runs within each provisioned workspace and provides core functionality including:
51+
- SSH access to workspaces via the `agentssh` package
52+
- Port forwarding
53+
- Terminal connectivity via the `pty` package for pseudo-terminal support
54+
- Application serving
55+
- Healthcheck monitoring
56+
- Resource usage reporting
57+
58+
Agents communicate with the control plane using the tailnet system and authenticate using secure tokens.
59+
60+
# Workspace Applications
61+
62+
Workspace applications (or "apps") provide browser-based access to services running within workspaces. The system supports:
63+
64+
- HTTP(S) and WebSocket connections
65+
- Path-based or subdomain-based access URLs
66+
- Health checks to monitor application availability
67+
- Different sharing levels (owner-only, authenticated users, or public)
68+
- Custom icons and display settings
69+
70+
The implementation is primarily in the `coderd/workspaceapps/` directory with components for URL generation, proxying connections, and managing application state.
71+
72+
# Implementation Details
73+
74+
The project structure separates frontend and backend concerns. React components and pages are organized in the `site/src/` directory, with Jest used for testing. The backend is primarily written in Go, with a strong emphasis on error handling patterns and test coverage.
75+
76+
Database interactions are carefully managed through migrations in `coderd/database/migrations/` and queries in `coderd/database/queries/`. All new queries require proper database authorization (dbauthz) implementation to ensure that only users with appropriate permissions can access specific resources.
77+
78+
# Authorization System
79+
80+
The database authorization (dbauthz) system enforces fine-grained access control across all database operations. It uses role-based access control (RBAC) to validate user permissions before executing database operations. The `dbauthz` package wraps the database store and performs authorization checks before returning data. All database operations must pass through this layer to ensure security.
81+
82+
# Testing Framework
83+
84+
The codebase has a comprehensive testing approach with several key components:
85+
86+
1. **Parallel Testing**: All tests must use `t.Parallel()` to run concurrently, which improves test suite performance and helps identify race conditions.
87+
88+
2. **coderdtest Package**: This package in `coderd/coderdtest/` provides utilities for creating test instances of the Coder server, setting up test users and workspaces, and mocking external components.
89+
90+
3. **Integration Tests**: Tests often span multiple components to verify system behavior, such as template creation, workspace provisioning, and agent connectivity.
91+
92+
4. **Enterprise Testing**: Enterprise features have dedicated test utilities in the `coderdenttest` package.
93+
94+
# Open Source and Enterprise Components
95+
96+
The repository contains both open source and enterprise components:
97+
98+
- Enterprise code lives primarily in the `enterprise/` directory
99+
- Enterprise features focus on governance, scalability (high availability), and advanced deployment options like workspace proxies
100+
- The boundary between open source and enterprise is managed through a licensing system
101+
- The same core codebase supports both editions, with enterprise features conditionally enabled
102+
103+
# Development Philosophy
104+
105+
Coder emphasizes clear error handling, with specific patterns required:
106+
- Concise error messages that avoid phrases like "failed to"
107+
- Wrapping errors with `%w` to maintain error chains
108+
- Using sentinel errors with the "err" prefix (e.g., `errNotFound`)
109+
110+
All tests should run in parallel using `t.Parallel()` to ensure efficient testing and expose potential race conditions. The codebase is rigorously linted with golangci-lint to maintain consistent code quality.
111+
112+
Git contributions follow a standard format with commit messages structured as `type: <message>`, where type is one of `feat`, `fix`, or `chore`.
113+
114+
# Development Workflow
115+
116+
Development can be initiated using `scripts/develop.sh` to start the application after making changes. Database schema updates should be performed through the migration system using `create_migration.sh <name>` to generate migration files, with each `.up.sql` migration paired with a corresponding `.down.sql` that properly reverts all changes.
117+
118+
If the development database gets into a bad state, it can be completely reset by removing the PostgreSQL data directory with `rm -rf .coderv2/postgres`. This will destroy all data in the development database, requiring you to recreate any test users, templates, or workspaces after restarting the application.
119+
120+
Code generation for the database layer uses `coderd/database/generate.sh`, and developers should refer to `sqlc.yaml` for the appropriate style and patterns to follow when creating new queries or tables.
121+
122+
The focus should always be on maintaining security through proper database authorization, clean error handling, and comprehensive test coverage to ensure the platform remains robust and reliable.

.github/.linkspector.yml

+1
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,6 @@ ignorePatterns:
2222
- pattern: "www.gnu.org"
2323
- pattern: "wiki.ubuntu.com"
2424
- pattern: "mutagen.io"
25+
- pattern: "docs.github.com"
2526
aliveStatusCodes:
2627
- 200

.github/workflows/ci.yaml

+5-2
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ jobs:
252252
run: |
253253
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
254254
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.34
255-
go install golang.org/x/tools/cmd/goimports@latest
255+
go install golang.org/x/tools/cmd/goimports@v0.31.0
256256
go install github.com/mikefarah/yq/v4@v4.44.3
257257
go install go.uber.org/mock/mockgen@v0.5.0
258258
@@ -299,6 +299,9 @@ jobs:
299299
- name: Setup Node
300300
uses: ./.github/actions/setup-node
301301

302+
- name: Check Go version
303+
run: IGNORE_NIX=true ./scripts/check_go_versions.sh
304+
302305
# Use default Go version
303306
- name: Setup Go
304307
uses: ./.github/actions/setup-go
@@ -860,7 +863,7 @@ jobs:
860863
run: |
861864
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
862865
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.34
863-
go install golang.org/x/tools/cmd/goimports@latest
866+
go install golang.org/x/tools/cmd/goimports@v0.31.0
864867
go install github.com/mikefarah/yq/v4@v4.44.3
865868
go install go.uber.org/mock/mockgen@v0.5.0
866869

.golangci.yaml

+4-27
Original file line numberDiff line numberDiff line change
@@ -24,30 +24,19 @@ linters-settings:
2424
enabled-checks:
2525
# - appendAssign
2626
# - appendCombine
27-
- argOrder
2827
# - assignOp
2928
# - badCall
30-
- badCond
3129
- badLock
3230
- badRegexp
3331
- boolExprSimplify
3432
# - builtinShadow
3533
- builtinShadowDecl
36-
- captLocal
37-
- caseOrder
38-
- codegenComment
3934
# - commentedOutCode
4035
- commentedOutImport
41-
- commentFormatting
42-
- defaultCaseOrder
4336
- deferUnlambda
4437
# - deprecatedComment
4538
# - docStub
46-
- dupArg
47-
- dupBranchBody
48-
- dupCase
4939
- dupImport
50-
- dupSubExpr
5140
# - elseif
5241
- emptyFallthrough
5342
# - emptyStringTest
@@ -56,56 +45,43 @@ linters-settings:
5645
# - exitAfterDefer
5746
# - exposedSyncMutex
5847
# - filepathJoin
59-
- flagDeref
60-
- flagName
6148
- hexLiteral
6249
# - httpNoBody
6350
# - hugeParam
6451
# - ifElseChain
6552
# - importShadow
6653
- indexAlloc
6754
- initClause
68-
- mapKey
6955
- methodExprCall
7056
# - nestingReduce
71-
- newDeref
7257
- nilValReturn
7358
# - octalLiteral
74-
- offBy1
7559
# - paramTypeCombine
7660
# - preferStringWriter
7761
# - preferWriteByte
7862
# - ptrToRefParam
7963
# - rangeExprCopy
8064
# - rangeValCopy
81-
- regexpMust
8265
- regexpPattern
8366
# - regexpSimplify
8467
- ruleguard
85-
- singleCaseSwitch
86-
- sloppyLen
8768
# - sloppyReassign
88-
- sloppyTypeAssert
8969
- sortSlice
9070
- sprintfQuotedString
9171
- sqlQuery
9272
# - stringConcatSimplify
9373
# - stringXbytes
9474
# - suspiciousSorting
95-
- switchTrue
9675
- truncateCmp
9776
- typeAssertChain
9877
# - typeDefFirst
99-
- typeSwitchVar
10078
# - typeUnparen
101-
- underef
10279
# - unlabelStmt
10380
# - unlambda
10481
# - unnamedResult
10582
# - unnecessaryBlock
10683
# - unnecessaryDefer
10784
# - unslice
108-
- valSwap
10985
- weakCond
11086
# - whyNoLint
11187
# - wrapperFunc
@@ -208,7 +184,7 @@ issues:
208184
- node_modules
209185
- .git
210186

211-
skip-files:
187+
exclude-files:
212188
- scripts/rules.go
213189

214190
# Rules listed here: https://github.com/securego/gosec#available-rules
@@ -224,15 +200,16 @@ issues:
224200
- path: scripts/*
225201
linters:
226202
- exhaustruct
203+
- path: scripts/rules.go
204+
linters:
205+
- ALL
227206

228207
fix: true
229208
max-issues-per-linter: 0
230209
max-same-issues: 0
231210

232211
run:
233212
timeout: 10m
234-
skip-files:
235-
- scripts/rules.go
236213

237214
# Over time, add more and more linters from
238215
# https://golangci-lint.run/usage/linters/ as the code improves.

agent/agent.go

+34-16
Original file line numberDiff line numberDiff line change
@@ -1075,7 +1075,7 @@ func (a *agent) handleManifest(manifestOK *checkpoint) func(ctx context.Context,
10751075
//
10761076
// An example is VS Code Remote, which must know the directory
10771077
// before initializing a connection.
1078-
manifest.Directory, err = expandDirectory(manifest.Directory)
1078+
manifest.Directory, err = expandPathToAbs(manifest.Directory)
10791079
if err != nil {
10801080
return xerrors.Errorf("expand directory: %w", err)
10811081
}
@@ -1115,16 +1115,35 @@ func (a *agent) handleManifest(manifestOK *checkpoint) func(ctx context.Context,
11151115
}
11161116
}
11171117

1118-
err = a.scriptRunner.Init(manifest.Scripts, aAPI.ScriptCompleted)
1118+
var (
1119+
scripts = manifest.Scripts
1120+
scriptRunnerOpts []agentscripts.InitOption
1121+
)
1122+
if a.experimentalDevcontainersEnabled {
1123+
var dcScripts []codersdk.WorkspaceAgentScript
1124+
scripts, dcScripts = agentcontainers.ExtractAndInitializeDevcontainerScripts(a.logger, expandPathToAbs, manifest.Devcontainers, scripts)
1125+
// See ExtractAndInitializeDevcontainerScripts for motivation
1126+
// behind running dcScripts as post start scripts.
1127+
scriptRunnerOpts = append(scriptRunnerOpts, agentscripts.WithPostStartScripts(dcScripts...))
1128+
}
1129+
err = a.scriptRunner.Init(scripts, aAPI.ScriptCompleted, scriptRunnerOpts...)
11191130
if err != nil {
11201131
return xerrors.Errorf("init script runner: %w", err)
11211132
}
11221133
err = a.trackGoroutine(func() {
11231134
start := time.Now()
1124-
// here we use the graceful context because the script runner is not directly tied
1125-
// to the agent API.
1135+
// Here we use the graceful context because the script runner is
1136+
// not directly tied to the agent API.
1137+
//
1138+
// First we run the start scripts to ensure the workspace has
1139+
// been initialized and then the post start scripts which may
1140+
// depend on the workspace start scripts.
1141+
//
1142+
// Measure the time immediately after the start scripts have
1143+
// finished (both start and post start). For instance, an
1144+
// autostarted devcontainer will be included in this time.
11261145
err := a.scriptRunner.Execute(a.gracefulCtx, agentscripts.ExecuteStartScripts)
1127-
// Measure the time immediately after the script has finished
1146+
err = errors.Join(err, a.scriptRunner.Execute(a.gracefulCtx, agentscripts.ExecutePostStartScripts))
11281147
dur := time.Since(start).Seconds()
11291148
if err != nil {
11301149
a.logger.Warn(ctx, "startup script(s) failed", slog.Error(err))
@@ -1851,30 +1870,29 @@ func userHomeDir() (string, error) {
18511870
return u.HomeDir, nil
18521871
}
18531872

1854-
// expandDirectory converts a directory path to an absolute path.
1855-
// It primarily resolves the home directory and any environment
1856-
// variables that may be set
1857-
func expandDirectory(dir string) (string, error) {
1858-
if dir == "" {
1873+
// expandPathToAbs converts a path to an absolute path. It primarily resolves
1874+
// the home directory and any environment variables that may be set.
1875+
func expandPathToAbs(path string) (string, error) {
1876+
if path == "" {
18591877
return "", nil
18601878
}
1861-
if dir[0] == '~' {
1879+
if path[0] == '~' {
18621880
home, err := userHomeDir()
18631881
if err != nil {
18641882
return "", err
18651883
}
1866-
dir = filepath.Join(home, dir[1:])
1884+
path = filepath.Join(home, path[1:])
18671885
}
1868-
dir = os.ExpandEnv(dir)
1886+
path = os.ExpandEnv(path)
18691887

1870-
if !filepath.IsAbs(dir) {
1888+
if !filepath.IsAbs(path) {
18711889
home, err := userHomeDir()
18721890
if err != nil {
18731891
return "", err
18741892
}
1875-
dir = filepath.Join(home, dir)
1893+
path = filepath.Join(home, path)
18761894
}
1877-
return dir, nil
1895+
return path, nil
18781896
}
18791897

18801898
// EnvAgentSubsystem is the environment variable used to denote the

0 commit comments

Comments
 (0)