Skip to content

Commit 5c9410c

Browse files
bpmctKatie Hornejawnsy
authored
add k3s cluster setup doc (coder#427)
* add k3s setup * fix github link * fix wording * edits from feedback * clarify wording * add TURN port * Edit text * Edit text * Lint * Update text * networking v2 -> TURNS Co-authored-by: Jonathan Yu <jonathan@coder.com> * apply jawnsy feedback * mention networking v2 * Edit text Co-authored-by: Katie Horne <katie@coder.com> Co-authored-by: Jonathan Yu <jonathan@coder.com>
1 parent 7be637f commit 5c9410c

File tree

2 files changed

+144
-0
lines changed

2 files changed

+144
-0
lines changed

‎manifest.json

+3
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,9 @@
128128
{
129129
"path": "./setup/kubernetes/local-preview.md"
130130
},
131+
{
132+
"path": "./setup/kubernetes/k3s.md"
133+
},
131134
{
132135
"path": "./setup/kubernetes/aws.md"
133136
},

‎setup/kubernetes/k3s.md

+141
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
---
2+
title: "K3s"
3+
description: Set up K3s on an Ubuntu machine to deploy Coder.
4+
---
5+
6+
This article will show you how to install K3s onto a new Ubuntu 20.04 LTS
7+
machine for use with Coder.
8+
9+
[K3s](https://k3s.io/) is a lightweight Kubernetes distribution that works well
10+
for single-node or multi-node clusters. This guide covers the installation of
11+
K3s onto a new Ubuntu 20.04 LTS machine. If you want to install Coder on a local
12+
machine or an existing host, a [kind cluster](./kind.md) or [k3d
13+
cluster](https://k3d.io/) may be a better choice, as it leverages Docker to set
14+
up/tear down clusters with little hassle.
15+
16+
> This installation method is not officially supported or tested by Coder. If
17+
> you have questions or run into issues, feel free to reach out using our
18+
> [community Slack channel](https://cdr.co/join-community).
19+
>
20+
> **We do not recommend using K3s for production deployments of Coder.**
21+
22+
## Prerequisites
23+
24+
Before proceeding, please make sure that:
25+
26+
- You have an **Ubuntu 20.04 machine**: This can be a bare metal or a virtual
27+
machine.
28+
29+
Ensure that the machine's specs satisfy Coder's [resource
30+
requirements](../requirements.md), since your experience with Coder is
31+
dependent on your system specs.
32+
33+
- You have the following software installed on your machine:
34+
35+
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)
36+
- [helm](https://helm.sh/docs/intro/install/)
37+
38+
- Your network policy or firewall accepts incoming traffic on:
39+
40+
- Port 22 (SSH)
41+
- Port 80 (HTTP)
42+
- Port 443 (HTTPS)
43+
- Port 5349 (TURNS)
44+
- **Optional**: Port 8443 (Kubernetes API)
45+
46+
## Step 1: Change the default SSH port
47+
48+
> If you've enabled Networking v2 after installing Coder (you can do so by going
49+
to **Manage** > **Admin** > **Infrastructure**), this step to SSH into
50+
workspaces isn't necessary, since TURNS is used instead.
51+
52+
To allow [SSH into workspaces](../../workspaces/ssh), you must change the host's
53+
default SSH port to free up port `22`. You may also need to modify your firewall
54+
to accept incoming traffic from the alternative port (e.g., if you rename port
55+
`22` to `5522`, then your firewall must accept traffic from `5522`).
56+
57+
> If you don't know how to change the SSH port in Linux, please review this
58+
> [guide from
59+
> Linuxize](https://linuxize.com/post/how-to-change-ssh-port-in-linux/)
60+
61+
## Step 2: Install K3s with Calico
62+
63+
The following steps are based on [Calico's quickstart
64+
guide](https://docs.projectcalico.org/getting-started/kubernetes/k3s/quickstart)
65+
for setting up K3s. However, you will disable K3s' default network policies and
66+
Traefik in favor of Calico and nginx-ingress.
67+
68+
1. Create a single-node K3s cluster:
69+
70+
```console
71+
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="--flannel-backend=none --cluster-cidr=192.168.0.0/16 --disable-network-policy --disable=traefik" sh -
72+
```
73+
74+
> Per the [Calico
75+
> docs](https://docs.projectcalico.org/getting-started/kubernetes/k3s/quickstart):
76+
>
77+
> If `192.168.0.0/16` is already in use within your network, you must select
78+
> a different pod network CIDR by replacing `192.168.0.0/16` in the above
79+
> command.
80+
>
81+
> K3s installer generates kubeconfig file in `/etc` with limited permissions;
82+
> by using the `K3S_KUBECONFIG_MODE` environment, you are assigning the
83+
> necessary permissions to the file and making it accessible for other users.
84+
85+
1. Install the Calico operator and CRDs (Calico implements Kubernetes pod
86+
networking and policy enforcement):
87+
88+
```console
89+
kubectl create -f https://docs projectcalico.org/manifests tigera-operator.yaml
90+
91+
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml
92+
```
93+
94+
1. Confirm that all of the pods are running:
95+
96+
```console
97+
watch kubectl get pods --all-namespaces
98+
```
99+
100+
## Step 3: Allow IP Forwarding
101+
102+
Modify Calico to enable IP forwarding, which is needed for container networking.
103+
104+
```console
105+
vim /etc/cni/net.d/10-canal.conflist
106+
107+
kubectl edit cm cni-config -n calico-system
108+
```
109+
110+
Under `container_settings`, set `allow_ip_forwarding` to `true`:
111+
112+
```json
113+
"container_settings": {
114+
"allow_ip_forwarding": true
115+
}
116+
```
117+
118+
## Step 4: Copy over the kubeconfig
119+
120+
Occasionally, Helm will not recognize the K3s cluster (see
121+
k3s-io/[k3s#1126](https://github.com/k3s-io/k3s/issues/1126) for more
122+
information).
123+
124+
If this happens, but you want to interface with the cluster from your local
125+
machine, copy `/etc/rancher/k3s/k3s.yaml` to `~/.kube/config`.
126+
127+
After copying this file from the K3s node to your local workstation:
128+
129+
- Ensure that you replace `localhost` or `127.0.0.1` with the host's public IP
130+
address in the copied file
131+
- Ensure that your firewall permits traffic through port `8443`
132+
133+
```console
134+
# on the host machine:
135+
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
136+
```
137+
138+
## Next steps
139+
140+
At this point, you're ready to proceed to [installing
141+
Coder](../installation.md).

0 commit comments

Comments
 (0)