|
| 1 | +# Prebuilt workspaces (beta) |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +Prebuilt workspaces let you pre-provision and maintain a pool of ready-to-claim workspaces. |
| 6 | +When a developer requests a workspace matching a preset, Coder assigns an existing instance instead of creating a new |
| 7 | +one, reducing setup time significantly. |
| 8 | + |
| 9 | +## Prerequisites |
| 10 | + |
| 11 | +- Premium license |
| 12 | +- Use `coder/coder` Terraform provider `>= 2.3.0-pre2` in your template (**TODO: update with latest version**) |
| 13 | +- Enable the `workspace-prebuilds` [experiment](https://coder.com/docs/reference/cli/server#--experiments) |
| 14 | + |
| 15 | +## Configuration |
| 16 | + |
| 17 | +1. In your Terraform template, add a `prebuilds` block within a `coder_workspace_preset` block: |
| 18 | + |
| 19 | + ```hcl |
| 20 | + data "coder_workspace_preset" "goland" { |
| 21 | + name = "GoLand: Large" |
| 22 | + parameters = { |
| 23 | + jetbrains_ide = "GO" |
| 24 | + cpus = 8 |
| 25 | + memory = 16 |
| 26 | + } |
| 27 | + prebuilds { |
| 28 | + instances = 3 |
| 29 | + } |
| 30 | + } |
| 31 | + ``` |
| 32 | + |
| 33 | +2. Publish and import the template |
| 34 | +3. An internal reconciliation loop maintains exactly the specified `instances` of prebuilt workspaces. |
| 35 | + |
| 36 | +_This model of declarative configuration plus a reconciliation loop is similar to Kubernetes._ |
| 37 | + |
| 38 | +## Ownership |
| 39 | + |
| 40 | +When prebuilt workspaces are created, they are owned by the pseudo-user `prebuilds`. This user has no permissions, and |
| 41 | +is simply a mechanism to identify unclaimed prebuilt workspaces. |
| 42 | + |
| 43 | +The `prebuilds` user is as a member of the `Everyone` group, and can be added to other groups. |
| 44 | + |
| 45 | +## Viewing prebuilt workspaces |
| 46 | + |
| 47 | +Given that prebuilt workspaces are just regular workspaces, you can view them in the **Workspaces** view in the |
| 48 | +frontend: |
| 49 | + |
| 50 | + |
| 51 | + |
| 52 | +## Claiming |
| 53 | + |
| 54 | +A prebuilt workspace is automatically and transparently assigned to a user when the following occurs: |
| 55 | + |
| 56 | +1. The user creates a new workspace via the API or the Coder web UI |
| 57 | +2. The user selected a preset in #1 which has been configured for prebuilds |
| 58 | +3. A prebuilt workspace is in eligible state |
| 59 | + |
| 60 | +The ownership of the prebuilt workspace will change to the requesting user, and this is referred to as a "claim". |
| 61 | + |
| 62 | +## Eligibility |
| 63 | + |
| 64 | +When a prebuilt workspace is running, and its agent has completed all of its bootstrap procedures and executed its |
| 65 | +startup scripts, the workspace will be marked eligible to be claimed. |
| 66 | + |
| 67 | +## Relationship to workspace presets |
| 68 | + |
| 69 | +[Workspace presets](https://coder.com/docs/admin/templates/extending-templates/parameters#workspace-presets-beta) allow |
| 70 | +you to configure commonly used combinations of parameters into a single option, which makes it easier for developers to |
| 71 | +pick one that fits |
| 72 | +their needs. |
| 73 | + |
| 74 | +Prebuilt workspaces need to have a preset defined to match the _base configuration_ of a workspace, i.e. the preset |
| 75 | +needs to define all the required parameters needed to build a workspace. These parameters are necessary in order to |
| 76 | +build workspaces in the background. |
| 77 | + |
| 78 | +Parameters which are not required or not part of a preset can still be used with prebuilt workspaces. The preset defines |
| 79 | +the minimum required set of parameters, and these are immutable. |
| 80 | + |
| 81 | +## Invalidation |
| 82 | + |
| 83 | +Prebuilt workspaces are _never_ updated after they are created. |
| 84 | + |
| 85 | +Whenever a template version changes, all prebuilt workspaces relating to an inactive template version will be destroyed. |
| 86 | +New prebuilt workspaces will be provisioned for the active template version. |
| 87 | + |
| 88 | +Invalidating prebuilt workspaces is useful when your template version does not change but a referenced dependency does, |
| 89 | +which is necessary for running an up-to-date workspace. For example, if |
| 90 | +an [AMI](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) which is referenced by your template is updated, |
| 91 | +you can simply delete the prebuilt workspaces, and they will be recreated with the latest AMI. |
| 92 | + |
| 93 | +_In future releases, we will allow operators to invalidate their prebuilt workspaces programmatically._ |
| 94 | + |
| 95 | +## Quotas |
| 96 | + |
| 97 | +Prebuilt workspaces can be used in conjunction with [Resource Quotas](https://coder.com/docs/admin/users/quotas). Given |
| 98 | +that all unclaimed prebuilt workspaces are [owned](#ownership) by the `prebuilds` user, you may configure a quota for |
| 99 | +any group which this user appears in. |
| 100 | + |
| 101 | +Once the quota is exceeded, prebuilt workspaces will fail provisioning like regular workspaces would. |
| 102 | + |
| 103 | +## Current Limitations |
| 104 | + |
| 105 | +### Organizations |
| 106 | + |
| 107 | +Prebuilt workspaces can only be utilized by the default organization. |
| 108 | + |
| 109 | +https://github.com/coder/internal/issues/364 is open to track this feature, and will be implemented in a future release. |
| 110 | + |
| 111 | +### Autoscaling |
| 112 | + |
| 113 | +Prebuilt workspaces will remain running indefinitely until they are claimed. We do not at present have an autoscaling |
| 114 | +mechanism to reduce the number of instances after working hours. |
| 115 | + |
| 116 | +https://github.com/coder/internal/issues/312 is open to track this feature, and will be implemented in a future release. |
| 117 | + |
| 118 | +## Gotchas |
| 119 | + |
| 120 | +### Resource Replacement |
| 121 | + |
| 122 | +When a prebuilt workspace is created, it is initially [owned](#ownership) by the `prebuilds` user and a random name |
| 123 | +is generated for it. When `terraform apply` runs, it will provide these values during provisioning in the |
| 124 | +[`coder_workspace`](https://registry.terraform.io/providers/coder/coder/latest/docs/data-sources/workspace) and |
| 125 | +[`coder_workspace_owner`](https://registry.terraform.io/providers/coder/coder/latest/docs/data-sources/workspace_owner) |
| 126 | +datasources. |
| 127 | + |
| 128 | +Once a prebuilt workspace is claimed, the ownership of that workspace changes to the requesting user and |
| 129 | +`terraform apply` is run again, now with updated values for the aforementioned datasources. |
| 130 | + |
| 131 | +If your template has used these datasources in immutable fields (i.e. the |
| 132 | +[`user_data`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#user_data-1) field in |
| 133 | +an `aws_instance` resource), Terraform will interpret these changes as _drift_ and will therefore destroy and recreate |
| 134 | +the resource. |
| 135 | + |
| 136 | +This is obviously undesirable because the prebuilt workspace will now have to provision _again_, while the user waits, |
| 137 | +eliminating the value of the prior pre-provisioning. |
| 138 | + |
| 139 | +Should this occur when a prebuilt workspace is claimed, all Template Admins will receive a notification which will |
| 140 | +link them to the build logs to investigate which resource was being replaced. |
| 141 | + |
| 142 | + |
| 143 | + |
| 144 | +To avoid this problem, you will need to add a `lifecycle` block to your resource: |
| 145 | + |
| 146 | +```hcl |
| 147 | +resource "docker_container" "workspace" { |
| 148 | + lifecycle { |
| 149 | + ignore_changes = all |
| 150 | + } |
| 151 | +
|
| 152 | + count = data.coder_workspace.me.start_count |
| 153 | + name = "coder-${data.coder_workspace_owner.me.name}-${lower(data.coder_workspace.me.name)}" |
| 154 | + ... |
| 155 | +} |
| 156 | +``` |
| 157 | + |
| 158 | +In the above example, the `docker_container` would be created with a `name` attribute which includes references to the |
| 159 | +initial owner (i.e. `prebuilds`), and will never change - even when the values of `data.coder_workspace_owner.me.name` |
| 160 | +and `data.coder_workspace.me.name` change in the above example. `name` is immutable like `user_data` above. |
| 161 | + |
| 162 | +You can read more about `ignore_changes` |
| 163 | +here: https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle#ignore_changes |
| 164 | + |
| 165 | +Should certain mutable attributes be required to change, you can use a more targeted approach by providing a list of |
| 166 | +attributes to `ignore_changes`: |
| 167 | + |
| 168 | +```hcl |
| 169 | +resource "docker_container" "workspace" { |
| 170 | + lifecycle { |
| 171 | + ignore_changes = [name] |
| 172 | + } |
| 173 | +
|
| 174 | + count = data.coder_workspace.me.start_count |
| 175 | + name = "coder-${data.coder_workspace_owner.me.name}-${lower(data.coder_workspace.me.name)}" |
| 176 | + ... |
| 177 | +} |
| 178 | +``` |
| 179 | + |
| 180 | +## Troubleshooting |
| 181 | + |
| 182 | +### Metrics |
| 183 | + |
| 184 | +- `coderd_prebuilt_workspaces_created_total` (counter): Total number of prebuilt workspaces that have been created to |
| 185 | + meet the desired instance count of each template preset |
| 186 | +- `coderd_prebuilt_workspaces_failed_total` (counter): Total number of prebuilt workspaces that failed to build |
| 187 | +- `coderd_prebuilt_workspaces_claimed_total` (counter): Total number of prebuilt workspaces which were claimed by users. |
| 188 | + Claiming refers to creating a workspace with a preset selected for which eligible prebuilt workspaces are available |
| 189 | + and one is reassigned to a user |
| 190 | +- `coderd_prebuilt_workspaces_desired` (gauge): Target number of prebuilt workspaces that should be available for each |
| 191 | + template preset |
| 192 | +- `coderd_prebuilt_workspaces_running` (gauge): Current number of prebuilt workspaces that are in a running state. These |
| 193 | + workspaces have started successfully but may not yet be claimable by users (see `coderd_prebuilt_workspaces_eligible`) |
| 194 | +- `coderd_prebuilt_workspaces_eligible` (gauge): Current number of prebuilt workspaces that are eligible to be claimed |
| 195 | + by users. These are workspaces that have completed their build process with their agent reporting 'ready' status |
| 196 | + |
| 197 | +### Logs |
| 198 | + |
| 199 | +Search for `coderd.prebuilds:` to gain insight into the behaviour of the reconciliation loop |
0 commit comments