Skip to content

add: ECS example template #3915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 8, 2022
Merged

add: ECS example template #3915

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c8d11d0
add: ECS example template
ericpaulsen Sep 6, 2022
49e1bd5
fix: empty main.tf
ericpaulsen Sep 6, 2022
deeaba6
cleanup
ericpaulsen Sep 6, 2022
70f9537
rm: cluster & compute
ericpaulsen Sep 7, 2022
d9c91f4
set CPU & memory vars
ericpaulsen Sep 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rm: cluster & compute
  • Loading branch information
@ericpaulsen
ericpaulsen committed Sep 7, 2022
commit 70f9537eccc9fa927a095ad36bef29da2792659c
74 changes: 2 additions & 72 deletions examples/templates/ecs-container/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,86 +6,16 @@ tags: [cloud, aws]

# aws-ecs

This is a sample template for running a Coder workspace on ECS.

## Required permissions / policy

The following sample policy allows Coder to create EC2 instances and modify
instances provisioned by Coder:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:GetDefaultCreditSpecification",
"ec2:DescribeIamInstanceProfileAssociations",
"ec2:DescribeTags",
"ec2:CreateTags",
"ec2:RunInstances",
"ec2:DescribeInstanceCreditSpecifications",
"ec2:DescribeImages",
"ec2:ModifyDefaultCreditSpecification",
"ec2:DescribeVolumes"
],
"Resource": "*"
},
{
"Sid": "CoderResources",
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeInstanceAttribute",
"ec2:UnmonitorInstances",
"ec2:TerminateInstances",
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:DeleteTags",
"ec2:MonitorInstances",
"ec2:CreateTags",
"ec2:RunInstances",
"ec2:ModifyInstanceAttribute",
"ec2:ModifyInstanceCreditSpecification"
],
"Resource": "arn:aws:ec2:*:*:instance/*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/Coder_Provisioned": "true"
}
}
}
]
}
```

Additionally, the `AmazonEC2ContainerServiceforEC2Role` managed policy should be
attached to the container instance IAM role, otherwise you will receive an error
when creating the ECS cluster.

This is represented as the `iam_instance_role` argument of the `launch_template`
resource. Please see the [AWS documentation for configuring this instance role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html#instance-iam-role-verify).
This is a sample template for running a Coder workspace on ECS. It assumes there
is a pre-existing ECS cluster with EC2-based compute to host the workspace.

## Architecture

This workspace is built using the following AWS resources:

- Launch template - this defines the EC2 instance(s) to host the container
- Auto-scaling group - EC2 auto-scaling group configuration
- ECS cluster - logical grouping of containers to be run in ECS
- Capacity provider - ECS-specific resource that ties in the auto-scaling group
- Task definition - the container definition, includes the image, command, volume(s)
- ECS service - manages the task definition

## User data

This template includes a two-part user data configuration, represented as the
`cloudinit_config` data source. There is an ECS-specific user data definition,
which is required for the EC2 instances to join the ECS cluster. Additionally, the
Coder user data (defined in the `locals` block) is needed to stop/start the instance(s).

## code-server

`code-server` is installed via the `startup_script` argument in the `coder_agent`
Expand Down
162 changes: 13 additions & 149 deletions examples/templates/ecs-container/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,155 +8,18 @@ terraform {
source = "coder/coder"
version = "~> 0.4.9"
}
cloudinit = {
source = "hashicorp/cloudinit"
version = "2.2.0"
}
}
}

# required for multi-user data config
provider "cloudinit" {}
variable "ecs-cluster" {
description = "Input the ECS cluster ARN to host the workspace"
default = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
default = ""
sensitive = true

We're currently using this to indicate whether this is a developer-level variable or template-wide variable. I assume the intended behavior is that the admin picks which cluster workspaces are deploying to?

I also suggest not setting a default value since it will skip it on coder templates create

Some docs: https://coder.com/docs/coder-oss/latest/templates#parameters

}

# configure AWS provider with creds present on Coder server host
provider "aws" {
region = "us-east-1"
shared_config_files = ["/home/coder/.aws/config"]
shared_credentials_files = ["/home/coder/.aws/credentials"]
}

locals {

# User data is used to stop/start AWS instances. See:
# https://github.com/hashicorp/terraform-provider-aws/issues/22

user_data_start = <<EOT
Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]
hostname: ${lower(data.coder_workspace.me.name)}

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
sudo -u ubuntu sh -c '${coder_agent.coder.init_script}'
--//--
EOT

user_data_end = <<EOT
Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0
--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"
#cloud-config
cloud_final_modules:
- [scripts-user, always]
--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"
#!/bin/bash
sudo shutdown -h now
--//--
EOT
}

# user data definitions, for both ECS & Coder
data "cloudinit_config" "main" {
base64_encode = true
gzip = false

part {
content = data.coder_workspace.me.transition == "start" ? local.user_data_start : local.user_data_end
}
# required for the EC2 instances to join the ECS cluster
part {
content = "#!/bin/bash\necho ECS_CLUSTER=${aws_ecs_cluster.main.name} >> /etc/ecs/ecs.config"
}
}

# set launch template for EC2 auto-scaling group
resource "aws_launch_template" "coder-oss-ubuntu" {
name_prefix = "coder-oss"
# ECS optimized AMI - https://github.com/awsdocs/amazon-ecs-developer-guide/blob/master/doc_source/ecs-optimized_AMI.md
image_id = "ami-03f8a7b55051ae0d4"
instance_type = "t2.medium"
iam_instance_profile {
arn = "arn:aws:iam::816024705881:instance-profile/coder-oss-ecs-role"
}
user_data = data.cloudinit_config.main.rendered
}

# provision auto-scaling group to host ECS task definitions
resource "aws_autoscaling_group" "main" {
name = "coder-ecs-auto-scaling-group"
min_size = 1
max_size = 1
desired_capacity = 1
availability_zones = ["us-east-1a"]
protect_from_scale_in = true

launch_template {
id = aws_launch_template.coder-oss-ubuntu.id
version = "$Latest"
}

tag {
key = "AmazonECSManaged"
value = true
propagate_at_launch = true
}
}

# create AWS ECS cluster
resource "aws_ecs_cluster" "main" {
name = "coder-oss-ecs"

setting {
name = "containerInsights"
value = "enabled"
}
}

# create capacity provider & tie in autoscaling group
resource "aws_ecs_capacity_provider" "main" {
name = "coder-oss-capacity-provider"

auto_scaling_group_provider {
auto_scaling_group_arn = aws_autoscaling_group.main.arn
managed_termination_protection = "ENABLED"

managed_scaling {
maximum_scaling_step_size = 1000
minimum_scaling_step_size = 1
status = "ENABLED"
target_capacity = 10
}
}
}

# attach capacity provider to cluster
resource "aws_ecs_cluster_capacity_providers" "main" {
cluster_name = aws_ecs_cluster.main.name

capacity_providers = [aws_ecs_capacity_provider.main.name]
shared_config_files = ["$HOME/.aws/config"]
shared_credentials_files = ["$HOME/.aws/credentials"]
}

# coder workspace, created as an ECS task definition
Expand All @@ -168,7 +31,7 @@ resource "aws_ecs_task_definition" "workspace" {
memory = 2048
container_definitions = jsonencode([
{
name = "coder-workspace-1"
name = "coder-workspace-${data.coder_workspace.me.id}"
image = "codercom/enterprise-base:ubuntu"
cpu = 1024
memory = 2048
Expand All @@ -184,7 +47,7 @@ resource "aws_ecs_task_definition" "workspace" {
mountPoints = [
{
# the name of the volume to mount
sourceVolume = "home-dir"
sourceVolume = "home-dir-${data.coder_workspace.me.id}"
# path on the container to mount the volume at
containerPath = "/home/coder"
}
Expand All @@ -200,7 +63,7 @@ resource "aws_ecs_task_definition" "workspace" {

# workspace persistent volume definition
volume {
name = "home-dir"
name = "home-dir-${data.coder_workspace.me.id}"

docker_volume_configuration {
# "shared" ensures that the disk is persisted upon workspace restart
Expand All @@ -212,10 +75,11 @@ resource "aws_ecs_task_definition" "workspace" {
}

resource "aws_ecs_service" "workspace" {
name = "workspace"
cluster = aws_ecs_cluster.main.id
name = "workspace-${data.coder_workspace.me.id}"
cluster = var.ecs-cluster
task_definition = aws_ecs_task_definition.workspace.arn
desired_count = 1
# scale the service to zero when the workspace is stopped
desired_count = data.coder_workspace.me.start_count
}

data "coder_workspace" "me" {}
Expand Down