feat: add provisioner chart to release and docs #9050
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,22 +10,23 @@ By default, the Coder server runs [built-in provisioner daemons](../cli/server.m | |
|
|
||
| - **Reduce server load**: External provisioners reduce load and build queue times from the Coder server. See [Scaling Coder](./scale.md#concurrent-workspace-builds) for more details. | ||
|
|
||
| Each provisioner can run a single [concurrent workspace build](./scale.md#concurrent-workspace-builds). For example, running 30 provisioner containers will allow 30 users to start workspaces at the same time. | ||
|
|
||
| Provisioners are started with the [coder provisionerd start](../cli/provisionerd_start.md) command. | ||
|
|
||
| ## Authentication | ||
|
|
||
| The provisioner daemon must authenticate with your Coder deployment. | ||
|
|
||
| Set a [provisioner daemon pre-shared key (PSK)](../cli/server.md#--provisioner-daemon-psk) on the Coder server and start the provisioner with | ||
| `coder provisionerd start --psk <your-psk>`. If you are [installing with Helm](../install/kubernetes#install-coder-with-helm), | ||
| see the [Helm example](#example-running-an-external-provisioner-with-helm) below. | ||
|
|
||
| > Coder still supports authenticating the provisioner daemon with a [token](../cli.md#--token) from a user with the | ||
| > Template Admin or Owner role. This method is deprecated in favor of the PSK, which only has permission to access | ||
| > provisioner daemon APIs. We recommend migrating to the PSK as soon as practical. | ||
|
|
||
| ## Types of provisioners | ||
|
|
||
| - **Generic provisioners** can pick up any build job from templates without provisioner tags. | ||
|
|
||
|
|
@@ -65,7 +66,68 @@ The provisioner server must authenticate with your Coder deployment. There are t | |
| --provisioner-tag scope=user | ||
| ``` | ||
|
|
||
| ## Example: Running an external provisioner with Helm | ||
|
|
||
| Coder provides a Helm chart for running external provisioner daemons, which you will use in concert with the Helm chart | ||
| for deploying the Coder server. | ||
|
|
||
| 1. Create a long, random pre-shared key (PSK) and store it in a Kubernetes secret | ||
|
|
||
| ```shell | ||
| kubectl create secret generic coder-provisioner-psk --from-literal=psk=`head /dev/urandom | tr -dc A-Za-z0-9 | head -c 26` | ||
| ``` | ||
|
|
||
| 1. Modify your Coder `values.yaml` to include | ||
|
|
||
| ```yaml | ||
| provisionerDaemon: | ||
| pskSecretName: "coder-provisioner-psk" | ||
| ``` | ||
|
|
||
| 1. Redeploy Coder with the new `values.yaml` to roll out the PSK. You can omit `--version <your version>` to also upgrade | ||
| Coder to the latest version. | ||
|
|
||
| ```shell | ||
| helm upgrade coder coder-v2/coder \ | ||
| --namespace coder \ | ||
| --version <your version> \ | ||
| --values values.yaml | ||
| ``` | ||
|
|
||
| 1. Create a `provisioner-values.yaml` file for the provisioner daemons Helm chart. For example | ||
|
|
||
| ```yaml | ||
| coder: | ||
| env: | ||
| - name: CODER_URL | ||
| value: "https://coder.example.com" | ||
| replicaCount: 10 | ||
| provisionerDaemon: | ||
| pskSecretName: "coder-provisioner-psk" | ||
| tags: | ||
| location: auh | ||
| kind: k8s | ||
|
Comment on lines
+105
to
+109
There was a problem hiding this comment. It's a bit confusing to me how the provisioner chart and the Coder chart seemingly accept the same Helm values format despite being different charts. If I am understanding correctly, when: provisionerDaemon:
pskSecretName: "coder-provisioner-psk"is set for the server values, it will set However, when the same values file is used set for the provisioner server, it passes the PSK to the provisioner. Is this a normal practice for Helm charts? If so, I'm on board. I also wonder, in a follow-up PR, if we should rewrite our workspace proxies Helm to do something similar. Our workspace proxies Helm is currently on the Coder chart and when enabled, Helm changes the entrypoint/command to start a wsproxy server instead of a full server. Our current wsproxy approach is easier for me to wrap my head around, but I can see how it may not be a best practice. There was a problem hiding this comment. The I kept the I think having different charts for coderd and provisionerd is conceptually easier to understand, rather than a single massive chart with lots of values that don't make sense when running provisioners. Workspace proxies are conceptually much more similar to Coderd in that they serve traffic, so maybe it's ok to keep those two use cases in a single chart. We should keep an eye on how much they diverge and split it into two charts if we're finding there is a lot of either-or config knobs. |
||
| ``` | ||
|
|
||
| This example creates a deployment of 10 provisioner daemons (for 10 concurrent builds) with the listed tags. For | ||
| generic provisioners, remove the tags. | ||
|
|
||
| > Refer to the [values.yaml](https://github.com/coder/coder/blob/main/helm/provisioner/values.yaml) file for the | ||
| > coder-provisioner chart for information on what values can be specified. | ||
|
|
||
| 1. Install the provisioner daemon chart | ||
|
|
||
| ```shell | ||
| helm install coder-provisioner coder-v2/coder-provisioner \ | ||
| --namespace coder \ | ||
| --version <your version> \ | ||
| --values provisioner-values.yaml | ||
| ``` | ||
|
|
||
| You can verify that your provisioner daemons have successfully connected to Coderd by looking for a log with message | ||
| `provisionerd successfully connected to coderd` from each Pod. | ||
|
|
||
| ## Example: Running an external provisioner on a VM | ||
|
|
||
| ```sh | ||
| curl -L https://coder.com/install.sh | sh | ||
|
|
@@ -74,7 +136,7 @@ export CODER_SESSION_TOKEN=your_token | |
| coder provisionerd start | ||
| ``` | ||
|
|
||
| ## Example: Running an external provisioner via Docker | ||
|
|
||
| ```sh | ||
| docker run --rm -it \ | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Coder Helm Chart | ||
|
|
||
| This directory contains the Helm chart used to deploy Coder provisioner daemons onto a Kubernetes | ||
| cluster. | ||
|
|
||
| External provisioner daemons are an Enterprise feature. Contact sales@coder.com. | ||
|
|
||
| ## Getting Started | ||
|
|
||
| > **Warning**: The main branch in this repository does not represent the | ||
| > latest release of Coder. Please reference our installation docs for | ||
| > instructions on a tagged release. | ||
|
|
||
| View | ||
| [our docs](https://coder.com/docs/v2/latest/admin/provisioners) | ||
| for detailed installation instructions. | ||
|
|
||
| ## Values | ||
|
|
||
| Please refer to [values.yaml](values.yaml) for available Helm values and their | ||
| defaults. | ||
|
|
||
| A good starting point for your values file is: | ||
|
|
||
| ```yaml | ||
| coder: | ||
| env: | ||
| - name: CODER_URL | ||
| value: "https://coder.example.com" | ||
| # This env enables the Prometheus metrics endpoint. | ||
| - name: CODER_PROMETHEUS_ADDRESS | ||
| value: "0.0.0.0:2112" | ||
| replicaCount: 10 | ||
| provisionerDaemon: | ||
| pskSecretName: "coder-provisioner-psk" | ||
| ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we supported API token scopes, would you be open to us supporting both, or are there other reasons for deprecating user token auth?
Something I like about relying on user authentication is that this token could be rotated without restarting the Coder server and #7992 could be used down the road to limit specific user tokens to specific provisioners versus full API access.
It does feel a little odd passing
CODER_SESSION_TOKENto the provisioner, especially if it is a long-lived token, but that could be said about any automation somebody is trying to do with Coder.I know it also requires that you create a token prior to starting a provisioner and you can only create a token on behalf of another user (machine account) via the REST API which is clunky.
Perhaps, for now, we mention the drawbacks of the token admin auth and avoid "deprecation" language and then we can support it again as a first-class citizen when we add scopes to API tokens.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to keep supporting user token auth anyway because of local provisioners.
I agree that we'll want to add support for tightly-scoped credentials that can be rotated without restarting Coderd in the future. At present I'm agnostic about whether they are API tokens or some other kind of credential (as a specific example: a JWT signed by an external authority). We should speak with our large customers to get their take on how they'd like to do this auth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bet you a 🌮 someone is going to ask for Azure Workload Identity