Skip to content

feat: Add built-in PostgreSQL for simple production setup #2345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 15, 2022
Merged

feat: Add built-in PostgreSQL for simple production setup #2345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
dc122b6
feat: Add built-in PostgreSQL for simple production setup
kylecarbs Jun 14, 2022
0450fcb
Merge branch 'main' into embeddeddb
kylecarbs Jun 15, 2022
0856209
Merge branch 'main' into embeddeddb
kylecarbs Jun 15, 2022
7fd3560
Use fork of embedded-postgres for cache path
kylecarbs Jun 15, 2022
690c791
Merge branch 'main' into embeddeddb
kylecarbs Jun 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .goreleaser.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ nfpms:
type: "config|noreplace"
- src: coder.service
dst: /usr/lib/systemd/system/coder.service
scripts:
preinstall: preinstall.sh

# Image templates are empty on snapshots to avoid lengthy builds for development.
dockers:
Expand Down
8 changes: 6 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,14 @@ To install, run:
curl -fsSL https://coder.com/install.sh | sh
```

Once installed, you can run a temporary deployment in dev mode (all data is in-memory and destroyed on exit):
Once installed, you can start a production deployment with a single command:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, but turning the DB off if the single instance of the app goes down is not what a lot of people would consider "production"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do essentially this same thing right now on our dogfood deployment, which I'd consider a production deployment of Coder.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @f0ssel here -- I'd be more comfortable with a "traditionally-managed" database if I were setting up a prod deployment than just using the embedded one. Doesn't detract from the value of being able to instantly kick the tyres with a real database!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why though? I believe users should start real, production deployments with this setup. There isn't a technical reason why it's bad practice. Users can just backup the ~/.config/coder/postgres directory on a CRON.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair fair!

Copy link
Contributor

@f0ssel f0ssel Jun 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coupling the lifecycle of the database to the app lifecycle and running the DB on the same host are two pretty big red flags for any engineers that have to actually manage and troubleshoot the app in production.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the specific red flags? You can still run the database independently.

Copy link
Contributor

@f0ssel f0ssel Jun 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm saying that if a customer wanting to run 1000 devs and it be production ready I'd probably advise them:

  • Run the DB on a dedicated host
  • Run regular database backups and have a restore process
  • Run multiple instances of the app server, optionally on multiple hosts behind a LB, optionally across different AZs.
  • Have a way to run new webserver versions without causing downtime

Right now this command not only does none of those things, but also won't even allow you to run multiple instances of the web app since it will conflict on the DB setup.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And similar to why people recommend not putting DB healthchecks in the webserver healthchecks on kubernetes - you don't want your DB taking your webapp down and vice versa, you want their lifecycles independent for obvious reasons.


```sh
coder server --dev
# Automatically sets up an external access URL on *.try.coder.app
coder server --tunnel

# Requires a PostgreSQL instance and external access URL
coder server --postgres-url <url> --access-url <url>
Comment on lines +56 to +57
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Requires a PostgreSQL instance and external access URL
coder server --postgres-url <url> --access-url <url>
# Advanced) Requires a PostgreSQL instance and external access URL
coder server --postgres-url <url> --access-url <url>

```

Use `coder --help` to get a complete list of flags and environment variables. Use our [quickstart guide](https://coder.com/docs/coder-oss/latest/quickstart) for a full walkthrough.
Expand Down
12 changes: 12 additions & 0 deletions cli/config/file.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,18 @@ func (r Root) DotfilesURL() File {
return File(filepath.Join(string(r), "dotfilesurl"))
}

func (r Root) PostgresPath() string {
return filepath.Join(string(r), "postgres")
}

func (r Root) PostgresPassword() File {
return File(filepath.Join(r.PostgresPath(), "password"))
}

func (r Root) PostgresPort() File {
return File(filepath.Join(r.PostgresPath(), "port"))
}

// File provides convenience methods for interacting with *os.File.
type File string

Expand Down
136 changes: 76 additions & 60 deletions cli/login.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/spf13/cobra"
"golang.org/x/xerrors"

"github.com/coder/coder/cli/cliflag"
"github.com/coder/coder/cli/cliui"
"github.com/coder/coder/codersdk"
)
Expand All @@ -37,7 +38,12 @@ func init() {
}

func login() *cobra.Command {
return &cobra.Command{
var (
email string
username string
password string
)
cmd := &cobra.Command{
Use: "login <url>",
Short: "Authenticate with a Coder deployment",
Args: cobra.ExactArgs(1),
Expand Down Expand Up @@ -66,71 +72,77 @@ func login() *cobra.Command {
return xerrors.Errorf("has initial user: %w", err)
}
if !hasInitialUser {
if !isTTY(cmd) {
return xerrors.New("the initial user cannot be created in non-interactive mode. use the API")
}
_, _ = fmt.Fprintf(cmd.OutOrStdout(), caret+"Your Coder deployment hasn't been set up!\n")

_, err := cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Would you like to create the first user?",
Default: "yes",
IsConfirm: true,
})
if errors.Is(err, cliui.Canceled) {
return nil
}
if err != nil {
return err
}
currentUser, err := user.Current()
if err != nil {
return xerrors.Errorf("get current user: %w", err)
}
username, err := cliui.Prompt(cmd, cliui.PromptOptions{
Text: "What " + cliui.Styles.Field.Render("username") + " would you like?",
Default: currentUser.Username,
})
if errors.Is(err, cliui.Canceled) {
return nil
}
if err != nil {
return xerrors.Errorf("pick username prompt: %w", err)
}

email, err := cliui.Prompt(cmd, cliui.PromptOptions{
Text: "What's your " + cliui.Styles.Field.Render("email") + "?",
Validate: func(s string) error {
err := validator.New().Var(s, "email")
if err != nil {
return xerrors.New("That's not a valid email address!")
}
if username == "" {
if !isTTY(cmd) {
return xerrors.New("the initial user cannot be created in non-interactive mode. use the API")
}
_, err := cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Would you like to create the first user?",
Default: "yes",
IsConfirm: true,
})
if errors.Is(err, cliui.Canceled) {
return nil
}
if err != nil {
return err
},
})
if err != nil {
return xerrors.Errorf("specify email prompt: %w", err)
}
currentUser, err := user.Current()
if err != nil {
return xerrors.Errorf("get current user: %w", err)
}
username, err = cliui.Prompt(cmd, cliui.PromptOptions{
Text: "What " + cliui.Styles.Field.Render("username") + " would you like?",
Default: currentUser.Username,
})
if errors.Is(err, cliui.Canceled) {
return nil
}
if err != nil {
return xerrors.Errorf("pick username prompt: %w", err)
}
}

password, err := cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Enter a " + cliui.Styles.Field.Render("password") + ":",
Secret: true,
Validate: cliui.ValidateNotEmpty,
})
if err != nil {
return xerrors.Errorf("specify password prompt: %w", err)
if email == "" {
email, err = cliui.Prompt(cmd, cliui.PromptOptions{
Text: "What's your " + cliui.Styles.Field.Render("email") + "?",
Validate: func(s string) error {
err := validator.New().Var(s, "email")
if err != nil {
return xerrors.New("That's not a valid email address!")
}
return err
},
})
if err != nil {
return xerrors.Errorf("specify email prompt: %w", err)
}
}
_, err = cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Confirm " + cliui.Styles.Field.Render("password") + ":",
Secret: true,
Validate: func(s string) error {
if s != password {
return xerrors.Errorf("Passwords do not match")
}
return nil
},
})
if err != nil {
return xerrors.Errorf("confirm password prompt: %w", err)

if password == "" {
password, err = cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Enter a " + cliui.Styles.Field.Render("password") + ":",
Secret: true,
Validate: cliui.ValidateNotEmpty,
})
if err != nil {
return xerrors.Errorf("specify password prompt: %w", err)
}
_, err = cliui.Prompt(cmd, cliui.PromptOptions{
Text: "Confirm " + cliui.Styles.Field.Render("password") + ":",
Secret: true,
Validate: func(s string) error {
if s != password {
return xerrors.Errorf("Passwords do not match")
}
return nil
},
})
if err != nil {
return xerrors.Errorf("confirm password prompt: %w", err)
}
}

_, err = client.CreateFirstUser(cmd.Context(), codersdk.CreateFirstUserRequest{
Expand Down Expand Up @@ -219,6 +231,10 @@ func login() *cobra.Command {
return nil
},
}
cliflag.StringVarP(cmd.Flags(), &email, "email", "e", "CODER_EMAIL", "", "Specifies an email address to authenticate with.")
cliflag.StringVarP(cmd.Flags(), &username, "username", "u", "CODER_USERNAME", "", "Specifies a username to authenticate with.")
cliflag.StringVarP(cmd.Flags(), &password, "password", "p", "CODER_PASSWORD", "", "Specifies a password to authenticate with.")
return cmd
}

// isWSL determines if coder-cli is running within Windows Subsystem for Linux
Expand Down
20 changes: 20 additions & 0 deletions cli/login_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,26 @@ func TestLogin(t *testing.T) {
<-doneChan
})

t.Run("InitialUserFlags", func(t *testing.T) {
t.Parallel()
client := coderdtest.New(t, nil)
// The --force-tty flag is required on Windows, because the `isatty` library does not
// accurately detect Windows ptys when they are not attached to a process:
// https://github.com/mattn/go-isatty/issues/59
doneChan := make(chan struct{})
root, _ := clitest.New(t, "login", client.URL.String(), "--username", "testuser", "--email", "user@coder.com", "--password", "password")
pty := ptytest.New(t)
root.SetIn(pty.Input())
root.SetOut(pty.Output())
go func() {
defer close(doneChan)
err := root.Execute()
assert.NoError(t, err)
}()
pty.ExpectMatch("Welcome to Coder")
<-doneChan
})

t.Run("InitialUserTTYConfirmPasswordFailAndReprompt", func(t *testing.T) {
t.Parallel()
ctx, cancel := context.WithCancel(context.Background())
Expand Down
4 changes: 2 additions & 2 deletions cli/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ func Root() *cobra.Command {
SilenceUsage: true,
Long: `Coder — A tool for provisioning self-hosted development environments.
`,
Example: ` Start Coder in "dev" mode. This dev-mode requires no further setup, and your local ` + cliui.Styles.Code.Render("coder") + ` CLI will be authenticated to talk to it. This makes it easy to experiment with Coder.
` + cliui.Styles.Code.Render("$ coder server --dev") + `
Example: ` Start a Coder server.
` + cliui.Styles.Code.Render("$ coder server") + `

Get started by creating a template from an example.
` + cliui.Styles.Code.Render("$ coder templates init"),
Expand Down
Loading