This is a combination of just documentation of best practices, as well as requiring that the end-user build some scripts/automation against our API? This won't be native?

It could be native! We could support "pooling" where, at a given time, a template has n running workspaces, which are then renamed to the user ID and given a new owner in the database, when a workspace is started.

For example, during an OpenShift workshop, there could be 100 clusters "ready" to be assigned to users, built in advance.

I helped product manage this feature at a previous role, and here are some complications to consider:

• People set up pool sizes, and then forget about it. It turns into a cost hog. We'd want metrics, warnings, or UI elements that help people recognize this waste
• Pool size needs it's cost allocated to someone, but it's in a limbo. Who "owns" the template?
• Scheduled start/stop on pools. Boot up 10 workspaces by 8am cst, and drop it down to 4 workspaces by 8pm cst.
• UI indicators that help people understand the state of a pool/exhausted pool

For our UseCase a workspace is defined in a coder.yaml. So every start of a new workspace can have different settings.

Related to #9611

I think we can implement this relatively cleanly by splitting provisioning into two phases:

1. Warming — we terraform apply the template without any user information. This state becomes an anonymous workspace.
2. Assigning — we terraform apply the anonymous workspace with user information. If the template is well-designed only cheap resources will be rebuilt.

Then users can make their templates warm-able by following these two practices:

1. Be resilient to missing user information when building
2. Only depend on user information in the cheapest recourses. E.g., use a random ID for an instance name.

Edit April 3rd, 2024:

Instead of complicated the provisioning layer, we could wait until Workspace transfers are robust, and then assign pre-built workspaces to a dummy account until they're needed. Each transfer will have to constitute a rebuild.

Just my two cents about prebuilds for devcontainer.
I know it does not generalize outside devcontainers, but I think coder should have proper support for devcontainers anyway.

Resources:

• https://containers.dev/implementors/reference/#prebuilding
• https://containers.dev/guide/prebuild
• https://docs.github.com/en/codespaces/prebuilding-your-codespaces/configuring-prebuilds

They are just images pushed into an oci registry.

We could use any CI to build and push prebuilds. But would be nice to get a "ready" to use solution.
Codespaces automate this via github actions, we could use coder workspaces as builder.

However coder must be aware of git repositories to filter prebuilds from a given branch/ref. (using the git provider api)

The configuration should be specific to each repository.
Github do it via the settings tab of each repository, coder may use a file per repository.

# .coder/config.yaml
devcontainers:
  template_id: # id of the devcontainer terraform template to use ?
  prebuilds:
    configuration: .devcontainer/devcontainer.json # default value
    triggers:            # some params in trigger should not be set together.
      schedule: ... 
      on_change:
       - ".devcontainer/**/*"
       - "!.devcontainer/**/*.md"
       - "Dockerfile"
      commit_interval: 4 # new prebuild every 4 commits
      enable_manual_trigger: true # allow manual trigger via coder cli or UI ?
    branches:    # which branches to prebuild
      - main
      - develop:
        configuration: # override configuration for release/1.*
          - .devcontainer/devcontainer.json
          - .devcontainer/frontend/devcontainer.json
      - release/2.* :
        triggers:
          on_change: # override on_change
           - .devcontainer/**/*
           - old-folder/Dockerfile
    notify_failure: # notify coder users when failure happen.
      - dev1
      - dev2

Coder can schedule workspace creation based on the config in each repository.
During workspace creation coder fill in the repo, branch and configuration, run the prebuild script and then delete the workspace.

When a user create a workspace from a given branch/ref, they should see if a prebuild is available.

Or even have the possibility to select one among a list of prebuilds.
This probably requires asking parameters in a specific order, maybe a new field page to coder_parameters could to the job.
The page =1 to ask repository, branch, and configuration to use. (completion like codespaces would be nice)
The page =2 would ask which prebuild to use and other things like dotfile, etc...
(IMO paging improve the UX by avoiding scrolling down parameters.)

# Generic devcontainer template using envbuilder or devcontainer cli

resource "a_cloud_vm_instance" {
   startup_script = data.coder_agent.main.init_script
   # [...]
}

# get information about the repository
data "coder_repository" "me"{
  # external auth to assume the git provider for api calls (bitbucket, gitlab, github...)
  git_external_auth_id  : data.coder_external_auth.my_git_provider.id    
  
  repository_url = data.coder_param.repository_url.value
  # or
  repository_id = my_project/_my_repo
  selected_branch = data.coder_param.branch.value
} 

resource "coder_devcontainer_prebuild" { 
  # Can set default values, enforce quotas/limits or configure admin configuration ?
  registry: "dockerhub.com/org/${data.coder_repository.me.repository_name}:${data.coder_repository.me.head_short_sha}"
  script: <<-EOT
    # customize prebuild script if needed.
    devcontainer build --push true --image-name dockerhub.com/org/${data.coder_repository.me.repository_name}:${data.coder_repository.me.head_short_sha}
    # [...]
  EOT
}

locals {
  init_script = <<-EOT
  if [[ ${data.coder_repository.prebuild} ]]
      # use prebuild.
  else 
     # devcontainer build ...
  EOT
}

# [...]

Update: @dannykopping is actively working on this and we'll likely have a generally available solution in the next couple of months.

That's right! We're actively working on this. I added some updates to the issue body to reflect our current progress.

This is now actively under development: https://github.com/orgs/coder/projects/44/views/3