Skip to content

docs: external provisioners #6443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/about/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ It offers:
provisionerd is the execution context for infrastructure modifying providers.
At the moment, the only provider is Terraform (running `terraform`).

> At the moment, provisionerd cannot be separated from coderd. Follow [this GitHub issue](https://github.com/coder/coder/issues/44) for more details.
By default, the Coder server runs multiple provisioner daemons. [External provisioners](../admin/provisioners.md) can be added for security or scalability purposes.

## Agents

Expand Down
92 changes: 92 additions & 0 deletions docs/admin/provisioners.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# External provisioners

By default, the Coder server runs [built-in provisioner daemons](../cli/coder_server.md#provisioner-daemons), which execute `terraform` during workspace and template builds. You can learn more about `provisionerd` in our [architecture documentation](../about/architecture.md#provisionerd).

> While external provisioners are stable, the feature is in an [alpha state](../contributing/feature-stages.md#alpha-features) and the behavior is subject to change in future releases. Use [GitHub issues](https://github.com/coder/coder) to leave feedback.

## Benefits of external provisioners

There are benefits in running external provisioner servers.

### Security

As you add more (template) admins in Coder, there is an increased risk of malicious code being added into templates. Isolated provisioners can prevent template admins from running code directly against the Coder server, database, or host machine.

Additionally, you can configure provisioner environments to access cloud secrets that you would like to conceal from the Coder server.

### Extensibility

Instead of exposing an entire API and secrets (e.g. Kubernetes, Docker, VMware) to the Coder server, you can run provisioners in each environment. See [Provider authentication](../templates/authentication.md) for more details.

### Scalability

External provisioners can reduce load and build queue times from the Coder server. See [Scaling Coder](./scale.md#concurrent-workspace-builds) for more details.

## Run an external provisioner

Once authenticated as a user with the Template Admin or Owner role, the [Coder CLI](../cli.md) can launch external provisioners. There are 3 types of provisioners:

- **Generic provisioners** can pick up any build job from templates without provisioner tags.

```sh
coder provisionerd start
```

> Ensure all provisioners (including [built-in provisioners](#disable-built-in-provisioners)) have similar configuration/cloud access. Otherwise, users may run into intermittent build errors depending on which provisioner picks up a job.

- **Tagged provisioners** can be used to pick up build jobs from templates (and corresponding workspaces) with matching tags.

```sh
coder provisionerd start \
--tag environment=on_prem \
--tag data_center=chicago

# In another terminal, create/push
# a template that requires this provisioner
coder templates create on-prem \
--provisioner-tag environment=on_prem

# Or, match the provisioner exactly
coder templates create on-prem-chicago \
--provisioner-tag environment=on_prem \
--provisioner-tag data_center=chicago
```

> At this time, tagged provisioners can also pick jobs from untagged templates. This behavior is [subject to change](https://github.com/coder/coder/issues/6442).

- **User provisioners** can only pick up jobs from user-tagged templates. Unlike the other provisioner types, any Coder can run user provisioners, but they have no impact unless there is at least one template with the `scope=user` provisioner tag.

```sh
coder provisionerd start \
--tag scope=user

# In another terminal, create/push
# a template that requires user provisioners
coder templates create on-prem \
--provisioner-tag scope=user
```

## Running external provisioners via Docker

The following command can run a Coder provisioner isolated in a Docker container.

```sh
docker run --rm -it \
-e CODER_URL=https://coder.example.com/ \
-e CODER_SESSION_TOKEN=your_token \
--entrypoint /opt/coder \
ghcr.io/coder/coder:latest \
provisionerd start
```

Be sure to replace `https://coder.example.com` with your [access URL](./configure.md#access-url) and `your_token` with an [API token](../api.md).

To include [provider secrets](../templates/authentication.md), modify the `docker run` command to mount environment variables or external volumes. Alternatively, you can create a custom provisioner image.

## Disable built-in provisioners

As mentioned above, the Coder server will run built-in provisioners by default. This can be disabled with a server-wide [flag or environment variable](../cli/coder_server.md#provisioner-daemons).

```sh
coder server --provisioner-daemons=0
```
7 changes: 7 additions & 0 deletions docs/admin/users.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,19 @@ Coder offers these user roles in the community edition:
| Change User roles | | | | ✅ |
| Manage **ALL** Templates | | | ✅ | ✅ |
| View, update and delete **ALL** Workspaces | | | ✅ | ✅ |
| Run [external provisioners](./provisioners.md) | | | ✅ | ✅ |
| Execute and use **ALL** Workspaces | | | | ✅ |
| View all user operation [Audit Logs](./audit-logs.md) | ✅ | | | ✅ |

A user may have one or more roles. All users have an implicit Member role
that may use personal workspaces.

## Security notes

A malicious Template Admin could write a template that executes commands on the host (or `coder server` container), which potentially escalates their privileges or shuts down the Coder server. To avoid this, run [external provisioners](./provisioners.md).

In low-trust environments, we do not recommend giving users direct access to edit templates. Instead, use [CI/CD pipelines to update templates](../templates/change-management.md) with proper security scans and code reviews in place.

## Create a user

To create a user with the web UI:
Expand Down
1 change: 1 addition & 0 deletions docs/images/icons/queue.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 7 additions & 0 deletions docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,13 @@
"path": "./admin/scale.md",
"icon_path": "./images/icons/scale.svg"
},
{
"title": "External Provisioners",
"description": "Run provisioners isolated from the Coder server",
"path": "./admin/provisioners.md",
"icon_path": "./images/icons/queue.svg",
"state": "enterprise"
},
{
"title": "Audit Logs",
"description": "Learn how to use Audit Logs in your Coder deployment",
Expand Down