Skip to content

chore: reduce build time by converting docker image to build with nix #9099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Aug 18, 2023
Merged

chore: reduce build time by converting docker image to build with nix #9099

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
3fb7c88
Add Docker image
kylecarbs Aug 15, 2023
bcc79ec
Try building the container in CI
kylecarbs Aug 15, 2023
bc1643a
Uncomment testing lines
kylecarbs Aug 15, 2023
3d58363
Trim image step
kylecarbs Aug 15, 2023
3d7539d
Hit 'em with a Docker load
kylecarbs Aug 15, 2023
faefce9
Oopsie!
kylecarbs Aug 15, 2023
35b546c
Add a tag and push!
kylecarbs Aug 15, 2023
f87b926
Fix image name 🤦
kylecarbs Aug 15, 2023
b8b476f
Fix sudo
kylecarbs Aug 15, 2023
43d8174
Fix target name
kylecarbs Aug 15, 2023
c058bcc
Build and push
kylecarbs Aug 15, 2023
2895f39
Fix login to docker hub
kylecarbs Aug 15, 2023
6496df8
Revert to Docker push
kylecarbs Aug 15, 2023
b9fb225
Fix PATH
kylecarbs Aug 17, 2023
2f35e09
Fix SHA
kylecarbs Aug 17, 2023
0e910b1
Fix ca certs
kylecarbs Aug 17, 2023
467918a
Fix ca certs
kylecarbs Aug 17, 2023
e6ec959
Fix coping files in
kylecarbs Aug 17, 2023
6890ee7
Fix docker init
kylecarbs Aug 17, 2023
d900e19
Fix Docker group and init
kylecarbs Aug 17, 2023
6b4578c
Add comments to our Nix
kylecarbs Aug 17, 2023
7cfa73f
Fix build stage
kylecarbs Aug 17, 2023
3437e07
Add some more comments
kylecarbs Aug 17, 2023
aed76a8
Remove old dogfood image files
kylecarbs Aug 17, 2023
b0f34cb
Tag and push with branch name
kylecarbs Aug 17, 2023
6c896ca
Merge branch 'main' into nixdev
kylecarbs Aug 17, 2023
a9f623c
Fix tag passing
kylecarbs Aug 17, 2023
986c1a8
Fix tag passing
kylecarbs Aug 17, 2023
600e9b2
Remove old pull triggers
kylecarbs Aug 17, 2023
c00fca7
Convert gen to use Nix
kylecarbs Aug 17, 2023
18f58c4
Add protobuf to the flake
kylecarbs Aug 17, 2023
eb95b64
Add prettier to the dev shell
kylecarbs Aug 17, 2023
24e998d
Swap to the faster Nix cache action
kylecarbs Aug 17, 2023
bb489cd
Add the correct yq
kylecarbs Aug 17, 2023
0b27d66
Fix gen
kylecarbs Aug 17, 2023
a361ca8
Add make to the flake
kylecarbs Aug 17, 2023
6c07b2a
Update extensions
kylecarbs Aug 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 5 additions & 29 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,38 +169,14 @@ jobs:
with:
fetch-depth: 1

- name: Setup Node
uses: ./.github/actions/setup-node

- name: Setup Go
uses: ./.github/actions/setup-go

- name: Setup sqlc
uses: ./.github/actions/setup-sqlc

- name: go install tools
run: |
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.30
go install storj.io/drpc/cmd/protoc-gen-go-drpc@v0.0.33
go install golang.org/x/tools/cmd/goimports@latest
go install github.com/mikefarah/yq/v4@v4.30.6
go install github.com/golang/mock/mockgen@v1.6.0
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@v4

- name: Install Protoc
run: |
# protoc must be in lockstep with our dogfood Dockerfile or the
# version in the comments will differ. This is also defined in
# security.yaml
set -x
cd dogfood
DOCKER_BUILDKIT=1 docker build . --target proto -t protoc
protoc_path=/usr/local/bin/protoc
docker run --rm --entrypoint cat protoc /tmp/bin/protoc > $protoc_path
chmod +x $protoc_path
protoc --version
- name: Run the Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@v2

- name: make gen
run: "make --output-sync -j -B gen"
run: "nix-shell --command 'make --output-sync -j -B gen'"

- name: Check for unstaged files
run: ./scripts/check_unstaged.sh
Expand Down
30 changes: 17 additions & 13 deletions .github/workflows/dogfood.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,15 @@ on:
branches:
- main
paths:
- "flake.nix"
- "flake.lock"
- "dogfood/**"
- ".github/workflows/dogfood.yaml"
# Uncomment these lines when testing with CI.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment might be out of date?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I almost missed this ty!

# pull_request:
# paths:
# - "flake.nix"
# - "flake.lock"
# - "dogfood/**"
# - ".github/workflows/dogfood.yaml"
workflow_dispatch:
Expand All @@ -18,6 +22,9 @@ jobs:
deploy_image:
runs-on: buildjet-4vcpu-ubuntu-2204
steps:
- name: Checkout
uses: actions/checkout@v3

- name: Get branch name
id: branch-name
uses: tj-actions/branch-names@v6.5
Expand All @@ -30,27 +37,24 @@ jobs:
tag=${tag//\//--}
echo "tag=${tag}" >> $GITHUB_OUTPUT

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Run the Magic Nix Cache
uses: DeterminateSystems/magic-nix-cache-action@v2

- run: nix build .#devEnvImage && ./result | docker load

- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}

- name: Build and push
uses: docker/build-push-action@v4
with:
context: "{{defaultContext}}:dogfood"
pull: true
push: true
tags: "codercom/oss-dogfood:${{ steps.docker-tag-name.outputs.tag }},codercom/oss-dogfood:latest"
cache-from: type=registry,ref=codercom/oss-dogfood:latest
cache-to: type=inline
- name: Tag and Push
run: |
docker tag codercom/oss-dogfood:latest codercom/oss-dogfood:${{ steps.docker-tag-name.outputs.tag }}
docker push codercom/oss-dogfood -a

deploy_template:
needs: deploy_image
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,6 @@ site/stats/
./scaletest/terraform/.terraform.lock.hcl
scaletest/terraform/secrets.tfvars
.terraform.tfstate.*

# Nix
result
3 changes: 3 additions & 0 deletions .prettierignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,9 @@ site/stats/
./scaletest/terraform/.terraform.lock.hcl
scaletest/terraform/secrets.tfvars
.terraform.tfstate.*

# Nix
result
# .prettierignore.include:
# Helm templates contain variables that are invalid YAML and can't be formatted
# by Prettier.
Expand Down
Loading