Skip to content

feat(coderd): connect dbcrypt package implementation #9523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
fb953e4
feat(coderd): add dbcrypt package
johnstcn Sep 4, 2023
3b8140b
feat(coderd): plumb through dbcrypt package
johnstcn Sep 4, 2023
55b93e7
fix indentation
johnstcn Sep 5, 2023
f340cba
fixup! fix indentation
johnstcn Sep 5, 2023
feae634
check for primary key revocation on startup
johnstcn Sep 5, 2023
381f078
retry insert active key on tx serialization failure
johnstcn Sep 5, 2023
c42e6a6
fixup! retry insert active key on tx serialization failure
johnstcn Sep 5, 2023
6a50a43
use database.IsSerializedError
johnstcn Sep 5, 2023
46b1ff4
encryptFields: check for nil field or digest
johnstcn Sep 5, 2023
9c18168
rm insertDBCryptKeyNoLock
johnstcn Sep 5, 2023
6c28ce5
Merge branch 'cj/dbcrypt_redux_1' into cj/dbcrypt_redux_2
johnstcn Sep 5, 2023
c54b64a
Update enterprise/cli/dbcrypt_rotate.go
johnstcn Sep 5, 2023
5959b34
Update enterprise/coderd/coderd.go
johnstcn Sep 5, 2023
b1546b1
add unit test for ExtractAPIKeyMW
johnstcn Sep 5, 2023
3859e03
add unit test for cli.ConnectToPostgres
johnstcn Sep 5, 2023
a4f93c5
Merge remote-tracking branch 'origin/main' into cj/dbcrypt_redux_2
johnstcn Sep 6, 2023
55a0fd0
DON'T PANIC
johnstcn Sep 6, 2023
cce0244
debug log user_ids
johnstcn Sep 6, 2023
d51ec66
dbcrypt-rotate -> server dbcrypt rotate
johnstcn Sep 6, 2023
aa39fcc
refactor: move rotate logic into dbcrypt
johnstcn Sep 6, 2023
e69e3ef
add decrypt/delete commands
johnstcn Sep 6, 2023
ebf4eef
fixup! add decrypt/delete commands
johnstcn Sep 6, 2023
2de6cc3
beef up unit tests, refactor cli
johnstcn Sep 6, 2023
7774811
update golden files
johnstcn Sep 6, 2023
35ca78f
Update codersdk/deployment.go
johnstcn Sep 6, 2023
3a92a7d
Merge remote-tracking branch 'origin/main' into cj/dbcrypt_redux_2
johnstcn Sep 7, 2023
8b1f43c
revoke all active keys on dbcrypt delete
johnstcn Sep 7, 2023
270cdc1
fixup! Merge remote-tracking branch 'origin/main' into cj/dbcrypt_red…
johnstcn Sep 7, 2023
2514ffe
update docs
johnstcn Sep 7, 2023
cd351af
fixup! update docs
johnstcn Sep 7, 2023
2f5c112
fixup! update docs
johnstcn Sep 7, 2023
2450d13
soft-enforce dbcrypt in license
johnstcn Sep 7, 2023
e56b639
do not add external token encryption keys by default (as it will alwa…
johnstcn Sep 7, 2023
441fcbf
update golden files
johnstcn Sep 7, 2023
ba14128
log encryption status on startup
johnstcn Sep 7, 2023
2ae45c6
modify CLI output
johnstcn Sep 7, 2023
13451f0
Merge remote-tracking branch 'origin/main' into cj/dbcrypt_redux_2
johnstcn Sep 7, 2023
b3ff024
rm unused golden file
johnstcn Sep 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
dbcrypt-rotate -> server dbcrypt rotate
  • Loading branch information
johnstcn committed Sep 6, 2023
commit d51ec66d17f632cb1f5af8e323c43084f01e57bf
1 change: 0 additions & 1 deletion docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ Coder — A tool for provisioning self-hosted development environments with Terr
| ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
| [<code>config-ssh</code>](./cli/config-ssh.md) | Add an SSH Host entry for your workspaces "ssh coder.workspace" |
| [<code>create</code>](./cli/create.md) | Create a workspace |
| [<code>dbcrypt-rotate</code>](./cli/dbcrypt-rotate.md) | Rotate database encryption keys |
| [<code>delete</code>](./cli/delete.md) | Delete a workspace |
| [<code>dotfiles</code>](./cli/dotfiles.md) | Personalize your workspace by applying a canonical dotfiles repository |
| [<code>features</code>](./cli/features.md) | List Enterprise features |
Expand Down
1 change: 1 addition & 0 deletions docs/cli/server.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 17 additions & 0 deletions docs/cli/server_dbcrypt.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 10 additions & 5 deletions docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -547,11 +547,6 @@
"description": "Create a workspace",
"path": "cli/create.md"
},
{
"title": "dbcrypt-rotate",
"description": "Rotate database encryption keys",
"path": "cli/dbcrypt-rotate.md"
},
{
"title": "delete",
"description": "Delete a workspace",
Expand Down Expand Up @@ -711,6 +706,16 @@
"description": "Create a new admin user with the given username, email and password and adds it to every organization.",
"path": "cli/server_create-admin-user.md"
},
{
"title": "server dbcrypt",
"description": "Manage database encryption",
"path": "cli/server_dbcrypt.md"
},
{
"title": "server dbcrypt rotate",
"description": "Rotate database encryption keys",
"path": "cli/server_dbcrypt_rotate.md"
},
{
"title": "server postgres-builtin-serve",
"description": "Run the built-in PostgreSQL deployment.",
Expand Down
20 changes: 0 additions & 20 deletions enterprise/cli/dbcrypt_rotate_slim.go

This file was deleted.

1 change: 0 additions & 1 deletion enterprise/cli/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ func (r *RootCmd) enterpriseOnly() []*clibase.Cmd {
r.licenses(),
r.groups(),
r.provisionerDaemons(),
r.dbcryptRotate(),
}
}

Expand Down
4 changes: 4 additions & 0 deletions enterprise/cli/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,5 +98,9 @@ func (r *RootCmd) Server(_ func()) *clibase.Cmd {
}
return api.AGPL, api, nil
})

cmd.AddSubcommands(
r.dbcryptCmd(),
)
return cmd
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,28 @@ import (
"golang.org/x/xerrors"
)

func (*RootCmd) dbcryptRotate() *clibase.Cmd {
func (r *RootCmd) dbcryptCmd() *clibase.Cmd {
dbcryptCmd := &clibase.Cmd{
Use: "dbcrypt",
Short: "Manage database encryption.",
Handler: func(inv *clibase.Invocation) error {
return inv.Command.HelpHandler(inv)
},
}
dbcryptCmd.AddSubcommands(
r.dbcryptRotateCmd(),
)
return dbcryptCmd
}

func (*RootCmd) dbcryptRotateCmd() *clibase.Cmd {
var (
vals = new(codersdk.DeploymentValues)
opts = vals.Options()
)
cmd := &clibase.Cmd{
Use: "dbcrypt-rotate --postgres-url <postgres_url> --external-token-encryption-keys <new-key>,<old-keys>",
Short: "Rotate database encryption keys",
Use: "rotate",
Short: "Rotate database encryption keys.",
Options: clibase.OptionSet{
*opts.ByName("Postgres Connection URL"),
*opts.ByName("External Token Encryption Keys"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ func TestDBCryptRotate(t *testing.T) {
require.NoError(t, err)

// Encrypt all the data with the initial cipher.
inv, _ := newCLI(t, "dbcrypt-rotate",
inv, _ := newCLI(t, "server", "dbcrypt", "rotate",
"--postgres-url", connectionURL,
"--external-token-encryption-keys", base64.StdEncoding.EncodeToString([]byte(keyA)),
)
Expand Down Expand Up @@ -79,7 +79,7 @@ func TestDBCryptRotate(t *testing.T) {
base64.StdEncoding.EncodeToString([]byte(keyA)),
)

inv, _ = newCLI(t, "dbcrypt-rotate",
inv, _ = newCLI(t, "server", "dbcrypt", "rotate",
"--postgres-url", connectionURL,
"--external-token-encryption-keys", externalTokensArg,
)
Expand Down
1 change: 0 additions & 1 deletion enterprise/cli/testdata/coder_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ Coder v0.0.0-devel — A tool for provisioning self-hosted development environme
 $ coder templates init 

Subcommands
dbcrypt-rotate Rotate database encryption keys
features List Enterprise features
groups Manage groups
licenses Add, delete, and list licenses
Expand Down
1 change: 1 addition & 0 deletions enterprise/cli/testdata/coder_server_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Start a Coder server
create-admin-user Create a new admin user with the given username,
email and password and adds it to every
organization.
dbcrypt Manage database encryption.
postgres-builtin-serve Run the built-in PostgreSQL deployment.
postgres-builtin-url Output the connection URL for the built-in
PostgreSQL deployment.
Expand Down
9 changes: 9 additions & 0 deletions enterprise/cli/testdata/coder_server_dbcrypt_--help.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Usage: coder server dbcrypt

Manage database encryption.

Subcommands
rotate Rotate database encryption keys.

---
Run `coder --help` for a list of global options.
24 changes: 24 additions & 0 deletions enterprise/cli/testdata/coder_server_dbcrypt_rotate_--help.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Usage: coder server dbcrypt rotate [flags]

Rotate database encryption keys.

Options
--postgres-url string, $CODER_PG_CONNECTION_URL
URL of a PostgreSQL database. If empty, PostgreSQL binaries will be
downloaded from Maven (https://repo1.maven.org/maven2) and store all
data in the config root. Access the built-in database with "coder
server postgres-builtin-url".

Enterprise Options
These options are only available in the Enterprise Edition.

--external-token-encryption-keys string-array, $CODER_EXTERNAL_TOKEN_ENCRYPTION_KEYS
Encrypt OIDC and Git authentication tokens with AES-256-GCM in the
database. The value must be a comma-separated list of base64-encoded
keys. Each key, when base64-decoded, must be exactly 32 bytes in
length. The first key will be used to encrypt new values. Subsequent
keys will be used as a fallback when decrypting. During normal
operation it is recommended to only set one key.

---
Run `coder --help` for a list of global options.