From affbe382dd3e250da8d0b856d17d91efb370e7e9 Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:02:33 +0000 Subject: [PATCH 1/6] docs: external provisioners --- docs/about/architecture.md | 2 +- docs/admin/provisioners.md | 92 +++++++++++++++++++++++++++++++++++++ docs/admin/users.md | 1 + docs/images/icons/queue.svg | 1 + docs/manifest.json | 7 +++ 5 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 docs/admin/provisioners.md create mode 100644 docs/images/icons/queue.svg diff --git a/docs/about/architecture.md b/docs/about/architecture.md index 7984209531d57..4c0d941c6b41c 100644 --- a/docs/about/architecture.md +++ b/docs/about/architecture.md @@ -25,7 +25,7 @@ It offers: provisionerd is the execution context for infrastructure modifying providers. At the moment, the only provider is Terraform (running `terraform`). -> At the moment, provisionerd cannot be separated from coderd. Follow [this GitHub issue](https://github.com/coder/coder/issues/44) for more details. +By default, the Coder server runs multiple provisioner daemons. [External provisioners](../admin/provisioners.md) can be added for enhanced security or scalability. ## Agents diff --git a/docs/admin/provisioners.md b/docs/admin/provisioners.md new file mode 100644 index 0000000000000..6ed8aa028a3f9 --- /dev/null +++ b/docs/admin/provisioners.md @@ -0,0 +1,92 @@ +# External provisioners + +By default, the Coder server runs [built-in provisioner daemons](../cli/coder_server.md#provisioner-daemons), which execute `terraform` during workspace and template builds. You can learn more about `provisionerd` in our [architecture documentation](../about/architecture.md#provisionerd). + +> While external provisioners are stable, the feature is in an [alpha state](../contributing/feature-stages.md#alpha-features) and the behavior is subject to change in feature releases. Use [GitHub issues](https://github.com/coder/coder) to leave feedback. + +## Benefits of external provisioners + +There are benefits in running external provisioner servers. + +### Security + +As you add more (template) admins in Coder, there is an increased risk of malicious code being added into templates. Isolated provisioners can prevent template admins from running code directly against the Coder server, database, or host machine. + +Additionally, you can configure provisioner environments to access cloud secrets that the Coder server does not necessarily have access to. + +### Extensibility + +Instead of exposing a full API and secrets to the Coder server (e.g. Kubernetes, Docker, VMware), provisioners can run in each environment. See [Provider authentication](../templates/authentication.md) for more details. + +### Scalability + +Reduce load from the Coder server and reduce queue times and build times for users. See [Scaling Coder](./scale.md#concurrent-workspace-builds) for more details. + +## Run an external provisioner + +The [Coder CLI](../cli.md) can launch external provisioners once authenticated as a user with the Template Admin or Owner role. There are 3 types of provisioners: + +- **Generic provisioners** can pick up any build job from templates without provisioner tags. + + ```sh + coder provisionerd start + ``` + + > Ensure all provisioners (including [built-in provisioners](#disable-built-in-provisioners)) have similar configuration/cloud access or you may run into intermittent build errors, depending on which provisioner picks up a job. + +- **Tagged provisioners** can be used to pick up build jobs from workspaces and templates with matching tags. + + ```sh + coder provisionerd start \ + --tag environment=on_prem \ + --tag data_center=chicago + + # In another terminal, create/push + # a template that requires this provisioner + coder templates create on-prem \ + --provisioner-tag environment=on_prem + + # Or, match the provisioner exactly + coder templates create on-prem-chicago \ + --provisioner-tag environment=on_prem \ + --provisioner-tag data_center=chicago + ``` + + > At this time, tagged provisioners can also pick jobs from untagged templates. This behavior is [subject to change](https://github.com/coder/coder/issues/6442). + +- **User provisioners** can only pick up jobs from user-tagged templates. Unlike the other provisioner types, any Coder can run user provisioners, but they have no impact unless there is at least one template with the `scope=user` provisioner tag. + + ```sh + coder provisionerd start \ + --tag scope=user + + # In another terminal, create/push + # a template that requires user provisioners + coder templates create on-prem \ + --provisioner-tag scope=user + ``` + +## Running external provisioners via Docker + +The following command can run a Coder provisioner isolated in a Docker container. + +```sh +docker run --rm -it \ + -e CODER_URL=https://coder.example.com/ \ + -e CODER_SESSION_TOKEN=your_token \ + --entrypoint /opt/coder \ + ghcr.io/coder/coder:latest \ + provisionerd start +``` + +Be sure to replace `https://coder.example.com` with your [access URL](./configure.md#access-url) and `your_token` with an [API token](../api.md). + +To include [provider secrets](../templates/authentication.md), you can modify the command to mount environment variables or external volumes. Alternatively, you can create a custom provisioner image that includes the Coder CLI. + +## Disable built-in provisioners + +As mentioned above, the Coder server will run built-in provisioners by default. This can be disabled with a server-wide [flag or environment variable](../cli/coder_server.md#provisioner-daemons). + +```sh +coder server --provisioner-daemons=0 +``` diff --git a/docs/admin/users.md b/docs/admin/users.md index 3ab7c40f48213..9c07996b66fe7 100644 --- a/docs/admin/users.md +++ b/docs/admin/users.md @@ -13,6 +13,7 @@ Coder offers these user roles in the community edition: | Change User roles | | | | ✅ | | Manage **ALL** Templates | | | ✅ | ✅ | | View, update and delete **ALL** Workspaces | | | ✅ | ✅ | +| Run [external provisioners](./provisioners.md) | | | ✅ | ✅ | | Execute and use **ALL** Workspaces | | | | ✅ | | View all user operation [Audit Logs](./audit-logs.md) | ✅ | | | ✅ | diff --git a/docs/images/icons/queue.svg b/docs/images/icons/queue.svg new file mode 100644 index 0000000000000..628b54f66652c --- /dev/null +++ b/docs/images/icons/queue.svg @@ -0,0 +1 @@ + diff --git a/docs/manifest.json b/docs/manifest.json index 78d9903fd1ce6..ece4030883a90 100644 --- a/docs/manifest.json +++ b/docs/manifest.json @@ -259,6 +259,13 @@ "path": "./admin/scale.md", "icon_path": "./images/icons/scale.svg" }, + { + "title": "External Provisioners", + "description": "Run provisioners isolated from the Coder server", + "path": "./admin/provisioners.md", + "icon_path": "./images/icons/queue.svg", + "state": "enterprise" + }, { "title": "Audit Logs", "description": "Learn how to use Audit Logs in your Coder deployment", From 20efbc8f1bbf8640428c840d68e9e0bc2885ae7f Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:04:44 +0000 Subject: [PATCH 2/6] better wording --- docs/about/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/about/architecture.md b/docs/about/architecture.md index 4c0d941c6b41c..791cb6ec76db8 100644 --- a/docs/about/architecture.md +++ b/docs/about/architecture.md @@ -25,7 +25,7 @@ It offers: provisionerd is the execution context for infrastructure modifying providers. At the moment, the only provider is Terraform (running `terraform`). -By default, the Coder server runs multiple provisioner daemons. [External provisioners](../admin/provisioners.md) can be added for enhanced security or scalability. +By default, the Coder server runs multiple provisioner daemons. [External provisioners](../admin/provisioners.md) can be added for security or scalability purposes. ## Agents From be16ceb71e9169eeb4c84cd144f67c8edc5778bd Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:05:07 +0000 Subject: [PATCH 3/6] fix wording --- docs/admin/provisioners.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/provisioners.md b/docs/admin/provisioners.md index 6ed8aa028a3f9..18669f0cec8a2 100644 --- a/docs/admin/provisioners.md +++ b/docs/admin/provisioners.md @@ -2,7 +2,7 @@ By default, the Coder server runs [built-in provisioner daemons](../cli/coder_server.md#provisioner-daemons), which execute `terraform` during workspace and template builds. You can learn more about `provisionerd` in our [architecture documentation](../about/architecture.md#provisionerd). -> While external provisioners are stable, the feature is in an [alpha state](../contributing/feature-stages.md#alpha-features) and the behavior is subject to change in feature releases. Use [GitHub issues](https://github.com/coder/coder) to leave feedback. +> While external provisioners are stable, the feature is in an [alpha state](../contributing/feature-stages.md#alpha-features) and the behavior is subject to change in future releases. Use [GitHub issues](https://github.com/coder/coder) to leave feedback. ## Benefits of external provisioners From bb14ef81b71762bbc9f353e2f8cc952862f03f2b Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:10:18 +0000 Subject: [PATCH 4/6] fix wording in various places --- docs/admin/provisioners.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/admin/provisioners.md b/docs/admin/provisioners.md index 18669f0cec8a2..0c147f4fa5a74 100644 --- a/docs/admin/provisioners.md +++ b/docs/admin/provisioners.md @@ -12,15 +12,15 @@ There are benefits in running external provisioner servers. As you add more (template) admins in Coder, there is an increased risk of malicious code being added into templates. Isolated provisioners can prevent template admins from running code directly against the Coder server, database, or host machine. -Additionally, you can configure provisioner environments to access cloud secrets that the Coder server does not necessarily have access to. +Additionally, you can configure provisioner environments to access cloud secrets that you would like to conceal from the Coder server. ### Extensibility -Instead of exposing a full API and secrets to the Coder server (e.g. Kubernetes, Docker, VMware), provisioners can run in each environment. See [Provider authentication](../templates/authentication.md) for more details. +Instead of exposing an entire API and secrets (e.g. Kubernetes, Docker, VMware) to the Coder server, you can run provisioners in each environment. See [Provider authentication](../templates/authentication.md) for more details. ### Scalability -Reduce load from the Coder server and reduce queue times and build times for users. See [Scaling Coder](./scale.md#concurrent-workspace-builds) for more details. +External provisioners can reduce load and build queue times from the Coder server. See [Scaling Coder](./scale.md#concurrent-workspace-builds) for more details. ## Run an external provisioner @@ -32,9 +32,9 @@ The [Coder CLI](../cli.md) can launch external provisioners once authenticated a coder provisionerd start ``` - > Ensure all provisioners (including [built-in provisioners](#disable-built-in-provisioners)) have similar configuration/cloud access or you may run into intermittent build errors, depending on which provisioner picks up a job. + > Ensure all provisioners (including [built-in provisioners](#disable-built-in-provisioners)) have similar configuration/cloud access. Otherwise, users may run into intermittent build errors depending on which provisioner picks up a job. -- **Tagged provisioners** can be used to pick up build jobs from workspaces and templates with matching tags. +- **Tagged provisioners** can be used to pick up build jobs from templates (and corresponding workspaces) with matching tags. ```sh coder provisionerd start \ @@ -81,7 +81,7 @@ docker run --rm -it \ Be sure to replace `https://coder.example.com` with your [access URL](./configure.md#access-url) and `your_token` with an [API token](../api.md). -To include [provider secrets](../templates/authentication.md), you can modify the command to mount environment variables or external volumes. Alternatively, you can create a custom provisioner image that includes the Coder CLI. +To include [provider secrets](../templates/authentication.md), modify the `docker run` command to mount environment variables or external volumes. Alternatively, you can create a custom provisioner image. ## Disable built-in provisioners From c4d1657151d333560fae72ef1824c4f6105f8017 Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:11:25 +0000 Subject: [PATCH 5/6] keep fixing wording --- docs/admin/provisioners.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin/provisioners.md b/docs/admin/provisioners.md index 0c147f4fa5a74..ee1a26a68721f 100644 --- a/docs/admin/provisioners.md +++ b/docs/admin/provisioners.md @@ -24,7 +24,7 @@ External provisioners can reduce load and build queue times from the Coder serve ## Run an external provisioner -The [Coder CLI](../cli.md) can launch external provisioners once authenticated as a user with the Template Admin or Owner role. There are 3 types of provisioners: +Once authenticated as a user with the Template Admin or Owner role, the [Coder CLI](../cli.md) can launch external provisioners. There are 3 types of provisioners: - **Generic provisioners** can pick up any build job from templates without provisioner tags. From bf2f017ad180d88d4913248b61c7a27946aae764 Mon Sep 17 00:00:00 2001 From: Ben Date: Fri, 3 Mar 2023 23:32:22 +0000 Subject: [PATCH 6/6] add another note about template management --- docs/admin/users.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/admin/users.md b/docs/admin/users.md index 9c07996b66fe7..6363d64aa2b53 100644 --- a/docs/admin/users.md +++ b/docs/admin/users.md @@ -20,6 +20,12 @@ Coder offers these user roles in the community edition: A user may have one or more roles. All users have an implicit Member role that may use personal workspaces. +## Security notes + +A malicious Template Admin could write a template that executes commands on the host (or `coder server` container), which potentially escalates their privileges or shuts down the Coder server. To avoid this, run [external provisioners](./provisioners.md). + +In low-trust environments, we do not recommend giving users direct access to edit templates. Instead, use [CI/CD pipelines to update templates](../templates/change-management.md) with proper security scans and code reviews in place. + ## Create a user To create a user with the web UI: