From b7b8c2765dfd107bbe0d3eea3c34bc008e51f3e6 Mon Sep 17 00:00:00 2001 From: Danny Kopping Date: Tue, 5 Nov 2024 18:30:44 +0100 Subject: [PATCH 1/3] Clarify template caching doc Signed-off-by: Danny Kopping --- .../best-practices/speed-up-templates.md | 97 ++++++++++++------- 1 file changed, 61 insertions(+), 36 deletions(-) diff --git a/docs/tutorials/best-practices/speed-up-templates.md b/docs/tutorials/best-practices/speed-up-templates.md index ddf08b5e51d75..948f219736aea 100644 --- a/docs/tutorials/best-practices/speed-up-templates.md +++ b/docs/tutorials/best-practices/speed-up-templates.md @@ -21,7 +21,7 @@ potentially optimize within the template. ![Screenshot of a workspace and its build timeline](../../images/best-practice/build-timeline.png) -Adjust this request to match your Coder access URL and workspace: +You can retrieve this detail programmatically from the API, as well: ```shell curl -X GET https://coder.example.com/api/v2/workspacebuilds/{workspacebuild}/timings \ @@ -36,9 +36,9 @@ for more information. ### Coder Observability Chart Use the [Observability Helm chart](https://github.com/coder/observability) for a -pre-built set of dashboards to monitor your control plane over time. It includes -Grafana, Prometheus, Loki, and Alert Manager out-of-the-box, and can be deployed -on your existing Grafana instance. +pre-built set of dashboards to monitor your Coder deployments over time. It +includes pre-configured Grafana, Prometheus, Loki, and Alertmanager instances to +ingest and display key observability data. We recommend that all administrators deploying on Kubernetes or on an existing Prometheus or Grafana stack set the observability bundle up with the control @@ -48,40 +48,44 @@ or our [Kubernetes installation guide](../../install/kubernetes.md). ### Enable Prometheus metrics for Coder -[Prometheus.io](https://prometheus.io/docs/introduction/overview/#what-is-prometheus) -is included as part of the [observability chart](#coder-observability-chart). It -offers a variety of -[available metrics](../../admin/integrations/prometheus.md#available-metrics), +Coder exposes a variety of +[application metrics](../../admin/integrations/prometheus.md#available-metrics), such as `coderd_provisionerd_job_timings_seconds` and -`coderd_agentstats_startup_script_seconds`, which measure how long the workspace -takes to provision and how long the startup script takes. +`coderd_agentstats_startup_script_seconds`, which measure how long the +workspaces take to provision and how long the startup scripts take. -You can -[install it separately](https://prometheus.io/docs/prometheus/latest/getting_started/) -if you prefer. +To make use of these metrics, you will need to +[enable Prometheus metrics](../../admin/integrations/prometheus#enable-prometheus-metrics) +exposition. + +If you are not using the [Observability Chart](#coder-observability-chart), you +will need to install Prometheus and configure it to scrape the metrics from your +Coder installation. ## Provisioners -`coder server` defaults to three provisioner daemons. Each provisioner daemon -can handle one single job, such as start, stop, or delete at a time and can be -resource intensive. When all provisioners are busy, workspaces enter a "pending" -state until a provisioner becomes available. +`coder server` by default provides three built-in provisioner daemons +(controlled by the +[`CODER_PROVISIONER_DAEMONS`](../../reference/cli/server#--provisioner-daemons) +config option). Each provisioner daemon can handle one single job (such as +start, stop, or delete) at a time and can be resource intensive. When all +provisioners are busy, workspaces enter a "pending" state until a provisioner +becomes available. ### Increase provisioner daemons Provisioners are queue-based to reduce unpredictable load to the Coder server. -However, they can be scaled up to allow more concurrent provisioners. You risk -overloading the central Coder server if you use too many built-in provisioners, -so we recommend a maximum of five provisioners. For more than five provisioners, -we recommend that you move to -[external provisioners](../../admin/provisioners.md). - -If you can’t move to external provisioners, use the `provisioner-daemons` flag -to increase the number of provisioner daemons to five: - -```shell -coder server --provisioner-daemons=5 -``` +If you require a higher bandwidth of provisioner jobs, you can do so by +increasing the +[`CODER_PROVISIONER_DAEMONS`](../../reference/cli/server#--provisioner-daemons) +config option. + +You risk overloading Coder if you use too many built-in provisioners, so we +recommend a maximum of five built-in provisioners per `coderd` replica. For more +than five provisioners, we recommend that you move to +[External Provisioners](../../admin/provisioners.md) and also consider +[High Availability](../../admin/networking/high-availability) to run multiple +`coderd` replicas. Visit the [CLI documentation](../../reference/cli/server.md#--provisioner-daemons) for @@ -116,21 +120,28 @@ for more information. ## Set up Terraform provider caching -By default, Coder downloads each Terraform provider when a workspace starts. -This can create unnecessary network and disk I/O. +### Template lock file + +On each workspace build, Terraform will examine the providers used by the +template and attempt to download the latest version of each provider (unless +constrained to a specific version). Terraform exposes a mechanism to build a +static list of provider versions, which improves cacheability. + +Without caching, Terraform will need to download each provider on each build, +and this can create unnecessary network and disk I/O. `terraform init` generates a `.terraform.lock.hcl` which instructs Coder provisioners to cache specific versions of your providers. -To use `terraform init` to cache providers: +To use `terraform init` to build the static provider version list: -1. Pull the templates to your local device: +1. Pull your template to your local device: ```shell - coder templates pull + coder templates pull