Skip to content

docs: improve workspaces and templates doc #9139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Oct 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
224bf4c
Reorg, updated/new screenshots, consistent terminology
marcpaq Aug 16, 2023
d494d28
First pass
marcpaq Aug 17, 2023
eabd21f
Another pass
marcpaq Aug 17, 2023
a92693d
Added integration section
marcpaq Aug 21, 2023
46618e4
New outline for template pages, small updates
marcpaq Aug 22, 2023
85535d0
Revised outline for templates, added tutorial
marcpaq Aug 23, 2023
bbd4928
First pass at tutorial
marcpaq Aug 23, 2023
3cc2dbb
Some feedback from Ben.
marcpaq Aug 23, 2023
cb39a6d
Update docs/workspaces.md
marcpaq Aug 25, 2023
6706032
Update docs/workspaces.md
marcpaq Aug 25, 2023
4e4b9e5
Update docs/workspaces.md
marcpaq Aug 25, 2023
351c457
Fixed typos
marcpaq Aug 25, 2023
50e9041
Expanded tutorial
marcpaq Aug 28, 2023
ed27c28
New screenshots, improved tutorial, revised anatomy
marcpaq Sep 7, 2023
744bad0
Improved tutorial. Anatomy is now a guided tour.
marcpaq Sep 8, 2023
58d5c9e
First pass at guided tour
marcpaq Sep 8, 2023
bf5228c
Updated authentication info
marcpaq Sep 11, 2023
92eea5f
Reorganized the guided tour
marcpaq Sep 12, 2023
302f5dd
Edited more template pages
marcpaq Sep 13, 2023
c80c0e3
Update docs/templates/tour.md
marcpaq Sep 15, 2023
e308e5d
Update docs/templates/tour.md
marcpaq Sep 15, 2023
22570dd
Update docs/templates/tour.md
marcpaq Sep 15, 2023
cbeac94
Update docs/templates/tutorial.md
marcpaq Sep 15, 2023
b12c1d6
Update docs/templates/tour.md
marcpaq Sep 15, 2023
b4f33d5
Update docs/templates/tour.md
marcpaq Sep 15, 2023
ec1741e
Update docs/templates/tour.md
marcpaq Sep 15, 2023
223dc9a
Update docs/templates/tour.md
marcpaq Sep 15, 2023
cb266c6
Update docs/templates/tour.md
marcpaq Sep 15, 2023
b5c0371
Revised devcontainers and docker-in-workspaces
marcpaq Sep 15, 2023
9f16953
Edited and added screenshots
marcpaq Sep 18, 2023
6bda970
Prepared first draft, except docs/templates/open-in-coder.md
marcpaq Sep 19, 2023
c2d992b
Fix typo
marcpaq Oct 3, 2023
c9a6c3c
remove legacy parameters and migration guide
matifali Oct 9, 2023
3efae01
Use coder templates create
marcpaq Oct 11, 2023
267639c
Added screenshot for workspace template variables
marcpaq Oct 12, 2023
5c64352
Made it prettier
marcpaq Oct 12, 2023
dbfc80d
Fixed minor typos and markdown problems
marcpaq Oct 12, 2023
6506168
edits to repairing workspaces
bpmct Oct 15, 2023
588a08a
fix broken links in product
bpmct Oct 15, 2023
151687d
Added troubleshooting, minor corrections.
marcpaq Oct 16, 2023
456f5d5
fix terminal links
bpmct Oct 16, 2023
59528ff
fmt
matifali Oct 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/admin/provisioners.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Provisioners
# External provisioners

By default, the Coder server runs [built-in provisioner daemons](../cli/server.md#provisioner-daemons), which execute `terraform` during workspace and template builds. However, there are sometimes benefits to running external provisioner daemons:

Expand Down
Binary file modified docs/images/autostart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/autostop.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/creating-workspace-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/schedule.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/template-variables.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/build-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/choosing-edit-template.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/coder-login-web.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/coder-session-token.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/create-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/create-workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/edit-files.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/edit-source-code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/new-workspace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/publish.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/select-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/source-code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/starter-templates.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-tour.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/template-variables.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/update.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/use-template.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/templates/workspace-apps.png
Binary file added docs/images/templates/workspace-ready.png
Binary file added docs/images/workspace-update.png
92 changes: 54 additions & 38 deletions docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -134,75 +134,86 @@
},
{
"title": "Templates",
"description": "Learn about templates, which define the infrastructure underlying workspaces",
"description": "Templates define the infrastructure for workspaces",
"path": "./templates/index.md",
"icon_path": "./images/icons/picture.svg",
"children": [
{
"title": "Templates & Terraform",
"description": "Learn what makes up a template",
"path": "./templates/concepts.md",
"title": "Working with templates",
"description": "Creating, editing, and updating templates",
"path": "./templates/creating.md"
},
{
"title": "Your first template",
"description": "A tutorial for creating and editing your first template",
"path": "./templates/tutorial.md"
},
{
"title": "Guided tour",
"description": "Create a template from scratch",
"path": "./templates/tour.md"
},
{
"title": "Setting up templates",
"description": "Best practices for writing templates",
"path": "./templates/best-practices.md",
"children": [
{
"title": "Parameters",
"description": "Use parameters to customize templates",
"path": "./templates/parameters.md",
"icon_path": "./images/icons/code.svg"
"title": "Change management",
"description": "Versioning templates with git and CI",
"path": "./templates/change-management.md",
"icon_path": "./images/icons/git.svg"
},
{
"title": "Terraform Modules",
"description": "Reuse code across Coder templates",
"path": "./templates/modules.md"
"title": "Provider authentication",
"description": "Authenticate the provisioner",
"path": "./templates/authentication.md",
"icon_path": "./images/icons/key.svg"
},
{
"title": "Agent Metadata",
"description": "Learn how to expose live agent information to users",
"path": "./templates/agent-metadata.md",
"icon_path": "./images/icons/table-rows.svg"
"title": "Resource persistence",
"description": "How resource persistence works in Coder",
"path": "./templates/resource-persistence.md",
"icon_path": "./images/icons/infinity.svg"
},
{
"title": "Resource Metadata",
"description": "Learn how to expose resource data to users",
"path": "./templates/resource-metadata.md",
"icon_path": "./images/icons/table-rows.svg"
"title": "Terraform modules",
"description": "Reuse code across Coder templates",
"path": "./templates/modules.md"
}
]
},
{
"title": "Best Practices",
"description": "Best practices for writing/managing templates",
"path": "./templates/best-practices.md",
"title": "Customizing templates",
"description": "Give information and options to workspace users",
"path": "./templates/customizing.md",
"children": [
{
"title": "Provider Authentication",
"description": "Learn how to authenticate the provisioner",
"path": "./templates/authentication.md",
"icon_path": "./images/icons/key.svg"
"title": "Agent metadata",
"description": "Show operational metrics in the workspace",
"path": "./templates/agent-metadata.md"
},
{
"title": "Resource Persistence",
"description": "Learn how resource persistence works in Coder",
"path": "./templates/resource-persistence.md",
"icon_path": "./images/icons/infinity.svg"
"title": "Resource metadata",
"description": "Show information in the workspace about template resources",
"path": "./templates/resource-metadata.md"
},
{
"title": "Change Management",
"description": "Learn how to source-control templates with git and CI",
"path": "./templates/change-management.md",
"icon_path": "./images/icons/git.svg"
"title": "Parameters",
"description": "Prompt the user for additional information about a workspace",
"path": "./templates/parameters.md"
}
]
},

{
"title": "Open in Coder",
"description": "Learn how to add an \"Open in Coder\" button to your repos",
"description": "Add an \"Open in Coder\" button to your repos",
"path": "./templates/open-in-coder.md",
"icon_path": "./images/icons/key.svg"
},
{
"title": "Docker in Workspaces",
"description": "Use docker inside containerized templates",
"title": "Docker in workspaces",
"description": "Use Docker inside containerized templates",
"path": "./templates/docker-in-workspaces.md",
"icon_path": "./images/icons/docker.svg"
},
Expand All @@ -211,6 +222,11 @@
"description": "Use devcontainers in workspaces",
"path": "./templates/devcontainers.md",
"state": "alpha"
},
{
"title": "Troubleshooting templates",
"description": "Fix common template problems",
"path": "./templates/troubleshooting.md"
}
]
},
Expand Down
61 changes: 35 additions & 26 deletions docs/templates/agent-metadata.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,26 @@
# Agent Metadata
# Agent metadata

![agent-metadata](../images/agent-metadata.png)

With Agent Metadata, template admins can expose operational metrics from
their workspaces to their users. It is the dynamic complement of [Resource Metadata](./resource-metadata.md).
You can show live operational metrics to workspace users with agent
metadata. It is the dynamic complement of [Resource
Metadata](./resource-metadata.md).

See the [Terraform reference](https://registry.terraform.io/providers/coder/coder/latest/docs/resources/agent#metadata).
You specify agent metadata in the
[`coder_agent`](https://registry.terraform.io/providers/coder/coder/latest/docs/resources/agent).

## Examples

All of these examples use [heredoc strings](https://developer.hashicorp.com/terraform/language/expressions/strings#heredoc-strings) for the script declaration. With heredoc strings, you
can script without messy escape codes, just as if you were working in your terminal.
All of these examples use [heredoc
strings](https://developer.hashicorp.com/terraform/language/expressions/strings#heredoc-strings)
for the script declaration. With heredoc strings, you can script
without messy escape codes, just as if you were working in your
terminal.

Some of the below examples use the [`coder stat`](../cli/stat.md) command.
This is useful for determining CPU/memory usage inside a container, which
can be tricky otherwise.
Some of the examples use the [`coder stat`](../cli/stat.md) command.
This is useful for determining CPU and memory usage of the VM or
container that the workspace is running in, which is more accurate
than resource usage about the workspace's host.

Here's a standard set of metadata snippets for Linux agents:

Expand Down Expand Up @@ -82,44 +88,46 @@ resource "coder_agent" "main" {
}
```

## Utilities
## Useful utilities

You can also show agent metadata for information about the workspace's host.

[top](https://linux.die.net/man/1/top) is available in most Linux
distributions and provides virtual memory, CPU and IO statistics. Running `top`
produces output that looks like:
distributions and provides virtual memory, CPU and IO statistics.
Running `top` produces output that looks like:

```text
%Cpu(s): 65.8 us, 4.4 sy, 0.0 ni, 29.3 id, 0.3 wa, 0.0 hi, 0.2 si, 0.0 st
MiB Mem : 16009.0 total, 493.7 free, 4624.8 used, 10890.5 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 11021.3 avail Mem
```

[vmstat](https://linux.die.net/man/8/vmstat) is available in most Linux
distributions and provides virtual memory, CPU and IO statistics. Running `vmstat`
produces output that looks like:
[vmstat](https://linux.die.net/man/8/vmstat) is available in most
Linux distributions and provides virtual memory, CPU and IO
statistics. Running `vmstat` produces output that looks like:

```text
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 19580 4781680 12133692 217646944 0 2 4 32 1 0 1 1 98 0 0
```

[dstat](https://linux.die.net/man/1/dstat) is considerably more parseable
than `vmstat` but often not included in base images. It is easily installed by
most package managers under the name `dstat`. The output of running `dstat 1 1` looks
like:
[dstat](https://linux.die.net/man/1/dstat) is considerably more
parseable than `vmstat` but often not included in base images. It is
easily installed by most package managers under the name `dstat`. The
output of running `dstat 1 1` looks like:

```text
--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
1 1 98 0 0|3422k 25M| 0 0 | 153k 904k| 123k 174k
```

## DB Write Load
## Managing the database load

Agent metadata can generate a significant write load and overwhelm your
database if you're not careful. The approximate writes per second can be
calculated using the formula:
Agent metadata can generate a significant write load and overwhelm
your Coder database if you're not careful. The approximate writes per
second can be calculated using the formula:

```text
(metadata_count * num_running_agents * 2) / metadata_avg_interval
Expand All @@ -131,7 +139,8 @@ For example, let's say you have
- each with 6 metadata snippets
- with an average interval of 4 seconds

You can expect `(10 * 6 * 2) / 4` or 30 writes per second.
You can expect `(10 * 6 * 2) / 4`, or 30 writes per second.

One of the writes is to the `UNLOGGED` `workspace_agent_metadata` table and
the other to the `NOTIFY` query that enables live stats streaming in the UI.
One of the writes is to the `UNLOGGED` `workspace_agent_metadata`
table and the other to the `NOTIFY` query that enables live stats
streaming in the UI.
48 changes: 31 additions & 17 deletions docs/templates/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,42 @@
</p>
</blockquote>

Coder's provisioner process needs to authenticate with cloud provider APIs to provision
workspaces. You can either pass credentials to the provisioner as parameters or execute Coder
in an environment that is authenticated with the cloud provider.
The Coder server's
[provisioner](https://registry.terraform.io/providers/coder/coder/latest/docs/data-sources/provisioner)
process needs to authenticate with other provider APIs to provision
workspaces. There are two approaches to do this:

We encourage the latter where supported. This approach simplifies the template, keeps cloud
provider credentials out of Coder's database (making it a less valuable target for attackers),
and is compatible with agent-based authentication schemes (that handle credential rotation
and/or ensure the credentials are not written to disk).
- Pass credentials to the provisioner as parameters.
- Preferred: Execute the Coder server in an environment that is
authenticated with the provider.

Cloud providers for which the Terraform provider supports authenticated environments include
We encourage the latter approach where supported:

- Simplifies the template.
- Keeps provider credentials out of Coder's database, making it
a less valuable target for attackers.
- Compatible with agent-based authentication schemes, which handle
credential rotation or ensure the credentials are not written to disk.

Generally, you can set up an environment to provide credentials to
Coder in these ways:

- A well-known location on disk. For example, `~/.aws/credentials` for
AWS on POSIX systems.
- Environment variables.

It is usually sufficient to authenticate using the CLI or SDK for the
provider before running Coder, but check the Terraform provider's
documentation for details.

These platforms have Terraform providers that support authenticated
environments:

- [Google Cloud](https://registry.terraform.io/providers/hashicorp/google/latest/docs)
- [Amazon Web Services](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
- [Microsoft Azure](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
- [Kubernetes](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs)

Additional providers may be supported; check the
[documentation of the Terraform provider](https://registry.terraform.io/browse/providers) for
details.

The way these generally work is via the credentials being available to Coder either in some
well-known location on disk (e.g. `~/.aws/credentials` for AWS on posix systems), or via
environment variables. It is usually sufficient to authenticate using the CLI or SDK for the
cloud provider before running Coder for this to work, but check the Terraform provider
documentation for details.
Other providers might also support authenticated environments. Check
the [documentation of the Terraform
provider](https://registry.terraform.io/browse/providers) for details.
6 changes: 5 additions & 1 deletion docs/templates/best-practices.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Template best practices

Templates are very flexible
We recommend a few ways to manage workspace resources, authentication,
and versioning.

<children>
</children>
12 changes: 7 additions & 5 deletions docs/templates/change-management.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Template Change Management

We recommend source controlling your templates as you would other code. [Install Coder](../install/) in CI/CD pipelines to push new template versions.
We recommend source-controlling your templates as you would other
code. You can [install Coder](../install/) in CI/CD pipelines to push new
template versions.

```console
# Install the Coder CLI
Expand All @@ -25,7 +27,7 @@ coder templates push --yes $CODER_TEMPLATE_NAME \
--name=$CODER_TEMPLATE_VERSION # Version name is optional
```

> Looking for an example? See how we push our development image
> and template [via GitHub actions](https://github.com/coder/coder/blob/main/.github/workflows/dogfood.yaml).

> To cap token lifetime on creation, [configure Coder server to set a shorter max token lifetime](../cli/server.md#--max-token-lifetime)
To cap token lifetime on creation, [configure Coder server to set a
shorter max token lifetime](../cli/server.md#--max-token-lifetime).
For an example, see how we push our development image and template
[with GitHub actions](https://github.com/coder/coder/blob/main/.github/workflows/dogfood.yaml).
Loading