Skip to content

Added draft of Coder and Nix article #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
4ebfb3a
Added draft of Coder and Nix article
unforswearing Mar 30, 2023
3ccfaaf
Updating content, remove yaml header.
unforswearing Mar 31, 2023
b5a97d2
Update posts/coder-and-nix/index.md
unforswearing Apr 15, 2023
f4c4345
Update posts/coder-and-nix/index.md
unforswearing Apr 15, 2023
c836084
Update posts/coder-and-nix/index.md
unforswearing Apr 15, 2023
5a321f5
Update posts/coder-and-nix/index.md
unforswearing Apr 15, 2023
ba585b1
fix spelling
unforswearing Apr 17, 2023
bb8ad97
Mentioned Docker issues with the Apple M1 chip
unforswearing Apr 19, 2023
8cffaa5
Apr 20, 2023, 1:18 PM
unforswearing Apr 20, 2023
029d825
Apr 20, 2023, 1:53 PM
unforswearing Apr 20, 2023
4cd5d60
Apr 20, 2023, 1:55 PM
unforswearing Apr 20, 2023
b1d8c3f
Apr 20, 2023, 1:59 PM
unforswearing Apr 20, 2023
8100e7d
Apr 20, 2023, 2:02 PM
unforswearing Apr 20, 2023
8424138
Apr 20, 2023, 2:09 PM
unforswearing Apr 20, 2023
110b60a
Apr 20, 2023, 2:14 PM
unforswearing Apr 20, 2023
6630127
Apr 20, 2023, 2:16 PM
unforswearing Apr 20, 2023
df7cf3d
Apr 20, 2023, 2:24 PM
unforswearing Apr 20, 2023
1aac9d6
Apr 20, 2023, 2:28 PM
unforswearing Apr 20, 2023
730e446
Apr 20, 2023, 2:38 PM
unforswearing Apr 20, 2023
2936625
Apr 20, 2023, 2:42 PM
unforswearing Apr 20, 2023
a0a773f
Apr 20, 2023, 2:49 PM
unforswearing Apr 20, 2023
3444f8e
Apr 20, 2023, 2:58 PM
unforswearing Apr 20, 2023
0d24787
Apr 20, 2023, 3:21 PM
unforswearing Apr 20, 2023
16e5be1
Apr 20, 2023, 3:22 PM
unforswearing Apr 20, 2023
8a2f42c
Apr 20, 2023, 3:24 PM
unforswearing Apr 20, 2023
01e11e4
Apr 20, 2023, 3:32 PM
unforswearing Apr 20, 2023
cadbe4c
Apr 20, 2023, 4:29 PM
unforswearing Apr 20, 2023
02268b4
Apr 20, 2023, 4:30 PM
unforswearing Apr 20, 2023
4eef905
Apr 20, 2023, 4:32 PM
unforswearing Apr 20, 2023
0fa5dd4
Apr 21, 2023, 11:47 AM
unforswearing Apr 21, 2023
5d3c5a7
Apr 21, 2023, 1:10 PM
unforswearing Apr 21, 2023
7d84389
Apr 21, 2023, 1:38 PM
unforswearing Apr 21, 2023
dd38030
Apr 21, 2023, 1:50 PM
unforswearing Apr 21, 2023
f229709
Apr 21, 2023, 2:06 PM
unforswearing Apr 21, 2023
62ceb5b
Apr 21, 2023, 2:08 PM
unforswearing Apr 21, 2023
bbb82b7
Apr 21, 2023, 2:14 PM
unforswearing Apr 21, 2023
03b2560
Apr 21, 2023, 2:16 PM
unforswearing Apr 21, 2023
36b193b
changed heading title
unforswearing Apr 21, 2023
4c61ae2
looking for a replacement for todomvc-nix...
unforswearing Apr 21, 2023
d6a9e05
todo: update last section to include
unforswearing Apr 21, 2023
f37f591
fixed issues with Dockerfile, add instructions, diagrams
unforswearing Apr 22, 2023
4e04ed8
change header name to match command
unforswearing Apr 22, 2023
4854f57
add content to Using Other Applications section
unforswearing Apr 22, 2023
3c55783
add nodejs.nix to example
unforswearing Apr 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 189 additions & 0 deletions posts/coder-and-nix/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
# Coder and Nix

## What is Coder

[Coder](https://coder.com) is an open source remote development platform that allows you to share an environment with your entire development team. Coder [eliminates many common issues](https://coder.com/why) with remote development by creating a shared environment for all your resources that is designed to streamline the development process for any type of development -- from DevOps and IT to Data Scientists. Additionally Coder is available through many of the popular cloud providers, allowing you to develop from anywhere using many different operating systems.

Since Coder is designed to be a shared development platform for your team, it makes sense that Coder would work well with a tool like Nix. In fact, the team at [Coder uses Nix](https://coder.com/docs/v2/latest/CONTRIBUTING#requirements) for our development process

## What is Nix?

Nix is a tool and operating system that treats every environment, including the os, as a list of dependencies. The Nix toolchain was created with the understanding that all software depends on other software to function. Because of this, Nix can let you set up your environment by declaring a list of dependencies. And according to [nix.dev](https://nix.dev/) this ecosystem aims to achieve reproducible development environments, seamless transfer of software environments between computers, and atomic upgrades and rollbacks.

You can get started with the Nix package manager immediately on any OS. Instructions for installing Nix on your system are located at the [nix installation guide](https://nixos.org/download.html).

Nix is a very robust, and at times complicated, platform. This article will attempt to describe how to use Nix with Coder at a high level. For additional information about Nix, please visit [https://nixos.org/](https://nixos.org/).

### Benefits to using Nix
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish there were some diagrams we could use to break up these sections, but I couldn't find anything particularly simple. Any ideas?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can come up with some diagrams over the weekend and will add them to the article by Monday.


At its core, Nix aims to provide packages that are reproducible by isolating each package build. This means that the dependencies in each package exist in a separate environment from other packages. Nix is somewhat unique in that you may use the package management features of Nix without the full OS. Best of all, Nix can allow you to declare a default configuration for your system which can be transferred to any new machine using a shared configuration file. You can search for Nix packages at [search.nixos.org](https://search.nixos.org/).

The reproducibility of Nix environments allows development teams allowing developers to share configuration with teams [and contributors](https://coder.com/docs/v2/latest/CONTRIBUTING#requirements). Declarative Nix environments ensures that the environment will contain the same resources and tools for each developer on your team.

Developers working with Nix won't have to worry about their builds failing due to outdated dependencies and can save time by avoiding resource-intensive workarounds. If you do happen to encounter an error with your Nix configuration you can roll back your Nix flake to the previous version to ensure your team can get work done right away.

![Docker workflow for development and production infrastructure](static/docker_infra_and_dev.png)

## What About Docker?

There are many benefits to using Docker to manage your development environment. You can prepare a Docker image with all the tooling needed to work on your app and have it ready to go with a basic `docker run` command. This environment ensures that all developers on your team are using the same version of the languages and resources that you need to maintain your application.

Docker also helps to abstract away the gritty details of using a particular service or language. With Docker you wont need to manage each process independently - it is all included in the docker image. Docker is great for deploying applications, but it gets a bit complicated when using Docker to develop them. And using the same environment to develop and deploy your application can get very complicated very quickly. Each additional layer of complexity added to the development environment is another thing that can break without warning.

However mixing this infrastructure with the development environment can sometimes limit the developers ability to use the tools they are comfortable with. Developers must give up their preferred shell configuration and tooling in favor of a base set of tools and settings determined by the infrastructure. Many developers can pick up new tooling without much difficulty, but having a top-down approach to the development environment means not only are these tools fixed for the project, they may eventually break if the resources in the Dockerfile become outdated.

Additionally, Docker requires that your team understands Docker and how it works to some degree. This is particularly important if something does happen to break within Docker -- your development team has to know how to troubleshoot the issue before they can resolve it. This is particularly difficult since Docker works differently between platforms. An issue one team member may have on Windows can be vastly different from issues MacOS based developers may face. And Docker has [quite a few issues running on MacOS](https://docs.docker.com/desktop/troubleshoot/known-issues/), especially with Apple's [new M1 chip](https://discussions.apple.com/thread/252268744). There is a potential for a huge amount of lost productivity when these issues come up, both in time and brainpower spent trying to fix the issue. And when its only your setup that doesn't work with Docker you find yourself facing the [it works on my machine](https://coder.com/blog/it-s-works-on-my-machine-explained) problem.

You may also need to develop within the container itself, and you may loose your own personal development configuration. This configuration could be added to the docker container however your entire team would need to use this config and thus agree on the parts that may or may not be included. This becomes a problem when you have a custom configuration or use different tools than the other members in the team. Yes, you can learn the tools used by the team as a whole, however your productivity may suffer for it.

Nix solves this problem with the `nix-shell` command, and the associated `shell.nix` configuration files.

### Bring Your Tools with Nix

As a developer using Nix as part of a project, you can bring your own tools by using Nix shell. By default, Docker does not allow you to bring your own custom configuration to the development environment. For example, say you have spent some time configuring your shell environment with themes and plugins. To use these tools with Docker you would have to add the shell configuration files to your Docker image, and everyone else on your team would need to use the same shell configuration. This becomes an issue if your teammates don't use the same tools and plugins. In order for you to use your own configuration you would have to maintain your own separate container containing your personal configuration and if that container fails, you are responsible for troubleshooting and fixing everything in order to continue development on your app.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's mention dotfiles as an option to customize your shell, even with Docker. A lot of our users use this, but it does require extra work.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dotfiles link you included resolves to a 404, but I did find this discussion. Is it related?


The nix-shell command resolves many of the uncertainties of the shared development process. You can use nix-shell from the command line by using the nix-shell command directly to install a package. For example, if you use the [fzf](https://github.com/junegunn/fzf) tool as a part of your workflow you can install it using the following command:

```bash
nix-shell -p fzf
```

The above command will add `fzf` to your local nix store and temporarily modify your $PATH variable to include the command in your shell. `fzf` will be available immediately.

On the other hand, if you're getting started with nix and want to create an environment that contains many of your favorite tools you can create a `shell.nix` file. This file will contain the commands needed to make the tools available for your current session, and requires the Nix builtin function (nix calls these functions [derivations](https://nixos.org/manual/nix/stable/language/derivations.html)) `mkshell`. The following command will install `zsh`, `fzf`, `autoenv`, and the `powerlevel10k` theme.

```nix
# shell.nix

with (import <nixpkgs> {});
mkshell {
buildInputs = [
fzf
zsh
zsh-autoenv
zsh-powerlevel10k
];
}
```

To use this `shell.nix` file you can run `nix-shell` without any arguments. The `nix-shell` tool will look for this file by default.

![Nix Shell Resources](static/nix_tooling.png)

### Adding Your Tools to A Shared `shell.nix` Configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is super solid, let's explain how this shell.nix can be used inside a Coder workspace in the next section

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this under the "NixOS, Docker, and nix-shell" section


The main benefit over using Docker for shared development environments is that Nix has the ability to extend this shared configuration with your own personal settings. For example, if you want to add your toolchain to a [nodejs.nix development environment](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/web/nodejs/nodejs.nix), you can modify your `shell.nix` file with the following code. Note that the `nodejs.nix` file is imported as part of the `buildInputs` array.

```nix
# shell.nix

with (import <nixpkgs> {});
mkshell {
buildInputs = [
(import ./nodejs.nix { inherit pkgs; })
fzf
zsh
zsh-autoenv
zsh-powerlevel10k
];
}
```

Now running the `nix-shell` command will create an environment containing the shared NodeJS nix configuration and your custom toolchain.

## Using Nix with Coder
Copy link
Member

@bpmct bpmct Apr 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's replace with:

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added instructions for this process, let me know if they need any adjusting. I ran into an issue buidling the todomvc-nix project so I'm currently looking for replacement.


At this point you're ready to start using Nix with Coder. There are a few different ways you can combine thee two tools, but one really effective method to get the most of both environments is to run Coder within a Nix environment by using Docker. In this example we will be using modifying the Docker template in Coder to add the [nix-devcontainer image](https://github.com/xtruder/nix-devcontainer).

### NixOS, Docker, and `nix-env`

Much of the functionality of Coder comes from the flexibility of [template](https://coder.com/blog/managing-templates-in-coder). To get started you will need to copy the Docker template in Coder. You copy this template using the [Coder CLI](https://github.com/coder/coder) by entering `coder templates init` into your terminal and selecting the `Develop in Docker` option.

<!-- ![screenshot of selecting docker](static/develop_in_docker.png) -->

```txt
> coder templates init
A template defines infrastructure as code to be provisioned for individual
developer workspaces. Select an example to be copied to the active directory:

Type to search

> Develop in Docker
Run workspaces on a Docker host using registry images
https://github.com/coder/coder/tree/main/examples/templates/docker
```

`Develop in Docker` will create a directory named `docker` in your current directory.

<!-- ![screenshot of extracing template](static/extract_docker_template.png) -->

```txt
Extracting docker to ./docker...
Create your template by running:

cd ./docker && coder templates create

Examples provide a starting point and are expected to be edited! 🎨
```

The `docker` directory that contains your newly copied template should look something like the diagram below:

```txt
├── docker
│   ├── README.md
│   ├── build
│   │   ├── Dockerfile
│   └── main.tf
```

Next, be sure to add the tools from your `shell.nix` file to the Docker image. To do this we will create a new file called `tools.nix`. This file will be loaded using the `nix-env` to load your tools into the current shell by default, whereas `shell.nix` can be used for ad-hoc dependencies.

The contents of your `tools.nix` should contain the following contents:

```nix
with import <nixpkgs>{}; [ fzf zsh zsh-autoenv zsh-powerlevel10k (import ./nodejs.nix { inherit pkgs; })]
```

After saving the above code to the `tools.nix` file, be sure to move this file to the same directory as your `Dockerfile`. In this example, this file will live in the `docker/build` directory

```txt
├── docker
│   ├── README.md
│   ├── build
│   │   ├── Dockerfile
│   │   └── tools.nix # docker/build/tools.nix
│   └── main.tf
```

Finally, we need to tell Docker how to load the NixOS container, create a user named `coder`, and install the tools from your `tools.nix` file into your local environment. To do this we will replace the default Dockerfile located at `docker/build/Dockerfile` with the new code below. Edit your Dockerfile to include on the following commands:

```dockerfile
FROM ghcr.io/xtruder/nix-devcontainer:v1
ARG USER=coder
ADD tools.nix /tools.nix
RUN nix-env -if /tools.nix && echo "tools.nix installed"
```

To use this new template with Coder, you have to add the template to your running Coder instance. You can do this from your terminal:

1. Navigte to the root of the template `docker` template directory
2. Next, register your updated template with Coder by running `coder templates edit docker`
3. Now upload the template to your Coder instance by running `coder templates push docker`

Once this template is uploaded you can use this template to create a new [workspace](https://coder.com/docs/v1/latest/workspaces). This workspace will create a full NixOS environment that contains all of the tools you defined in your `tools.nix` file.

### Using Other Applications

Since Nix is an OS and a package manager you can use nix tools to install just about any software to your environment. Let's say you want to use [emanote](https://emanote.srid.ca/start) to take project notes and share those notes with your team. You can install `emanote` to your [nix profile](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-profile.html) by running the following command:

```bash
nix profile install github:srid/emanote
```

Nix will ask you to confirm a few settings and then will install `emanote` in your environment. Point `emanote` to your notes folder and enter `emanote run --port 8080` to start a server to view your notes.

## Conclusion

In this article you learned a bit about Coder and Nix, and how you can combine the two platforms to create a robust remote development environment that is reproducible and can be extended to incorporate your own tools. Nix and Coder both aim to provide developers with a worry-free environment, allowing your team to focus on work and not troubleshooting.

Get started with Coder now by visiting [https://coder.com/](https://coder.com/).
Binary file added posts/coder-and-nix/static/develop_in_docker.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added posts/coder-and-nix/static/nix_tooling.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.