`coder templates create` using the template `docker-image-builds` and derivatives fails to build #2815

wale · 2022-07-04T00:12:51Z

Problem

The images folder is seemingly not pushed to the host when running coder templates create on Windows clients. The second validation rule in the below template snippet seemingly always fails regardless of the availability of the relevant images folder:

variable "docker_image" {
  description = "What Docker image would you like to use for your workspace?"
  default     = "alpine"

  # List of images available for the user to choose from.
  # Delete this condition to give users free text input.
  validation {
    condition = contains([
      "alpine",
      "ubuntu"
    ], var.docker_image)
    error_message = "Invalid Docker image!"
  }

  # Prevents admin errors when the image is not found
  validation {
    condition     = fileexists("images/${var.docker_image}.Dockerfile")
    error_message = "Invalid Docker image. The file does not exist in the images directory."
  }
}

Logs:

Directory tree:

This is also reproducible with the default docker-image-builds template without any changes.

Coder version (on both host and client): Coder v0.7.6+a494489

The text was updated successfully, but these errors were encountered:

wale · 2022-07-06T04:38:19Z

Apparently this is a bug that doesn't import Dockerfiles in a given directory, but importing them from the template root works fine?

phorcys420 · 2022-07-20T15:28:16Z

Apparently this is a bug that doesn't import Dockerfiles in a given directory, but importing them from the template root works fine?

Can confirm, the same error is occuring in a template of mine.
Maybe this issue should be re-named to something more generic ? Also, the weird part is that it doesn't seem to come from the docker provider itself, since even fileexists fails.

bpmct · 2022-07-25T21:29:31Z

@wale @phorcys420 Are you on Windows clients? I was only able to reproduce this when trying the docker-image-builds using coder on Windows. This appears to be a Coder bug as it's not recursively uploading the folder and subfolders to Coder.t

I also confirmed moving the images/ to root on WIndows fixed it.

phorcys420 · 2022-07-25T21:31:19Z

Are you on Windows clients?

Yeah.

This appears to be a Coder bug as it's not recursively uploading the folder and subfolders to Coder

Maybe, but doing coder templates pull still contains my subfolders.

bpmct · 2022-07-25T21:57:16Z

@phorcys420 You're able to make it through coder templates create and pull the template? Mine is failing at the create step because of the validation rule.

phorcys420 · 2022-07-25T21:58:59Z

@phorcys420 You're able to make it through coder templates create and pull the template? Mine is failing at the create step because of the validation rule.

No, I had to remove that one, and it still didn't work afterwards. I'm just pointing out that it does upload them because I can pull the subfolders afterwards.

phorcys420 · 2022-08-01T18:42:25Z

Stumbled across this today while pulling my template (v0.8.2+00c5116), maybe this has something to do with this issue?

phorcys420 · 2022-10-02T15:31:26Z

any news ?

matifali · 2022-10-08T21:42:55Z

I also faced this when trying to ADD files in my Dockerfile. Somehow docker provider doesn't see other files in the images subdirectory

phorcys420 · 2022-10-11T15:01:17Z

I also faced this when trying to add files in my Dockerfile. Somehow docker provider doesn't see other files in the images subdirectory

apparently it doesn't even see other files at all, I tried to ADD files from the same directory and it didn't see the file.

matifali · 2022-10-11T15:03:11Z

Is it a coder issue or the docker provide issue?

jsjoeio · 2022-10-11T18:11:28Z

I was able to reproduce this using Coder v0.9.8+b50bb99 Mon Oct 10 18:45:03 UTC 2022.

Logs:

PS C:\repro\coder_0.9.8_windows_amd64\docker-build-images> ..\coder.exe templates create      
> Create and upload "C:\\repro\\coder_0.9.8_windows_amd64\\docker-build-images"? (yes/no) yes
✔ Queued [261ms]
✔ Setting up [66ms]
✔ Adding README.md... [65ms]
✔ Parsing template parameters [144ms]
⧗  Detecting persistent resources
  Terraform 1.1.9

  Error: Invalid value for variable
  Invalid Docker image. The file does not exist in the images directory.

This was checked by the validation rule at main.tf:62,3-13.
✔ Detecting persistent resources [2130ms]
✘ Cleaning Up [111ms]
template import provision for start: recv import provision: plan terraform: terraform plan: exit status 1
Run 'coder templates create --help' for usage.

I'm going to dig in and see if I can find the root cause.

jsjoeio · 2022-10-11T19:55:13Z

I'm currently getting a Windows workspace fully setup then my plan is to see if this is on the Coder side or Terraform side by:

removing validation lines in main.tf
running coder templates create
after success coder templates pull
verify subdirectory images is there

If so, then it reproduces what @phorcys420 shared and would lead me to believe this is on the provider side. Stay tuned for more info.

jsjoeio · 2022-10-11T21:07:26Z

I was able to do the above + coder templates pull <template name> file.tar.gz and then extract it to see that the coder CLI does indeed upload the subdirectories.

Therefore, I believe this issue is in the docker provider we're using. @bpmct did share this issue he found which may be related: kreuzwerker/terraform-provider-docker#325

Workarounds:

remove validation steps
move images/ to root as suggested by @bpmct here

@bpmct how would you like to proceed from here?

phorcys420 · 2022-10-12T06:26:49Z

move images/ to root

please be aware that even if you did that you couldn't add files to the CT at all

matifali · 2022-10-12T06:58:14Z

move images/ to root

please be aware that even if you did that you couldn't add files to the CT at all

yes, I face this issue while using ADD in my docker file. It does not see any files in the images/ directory.
PS. I am using a Linux client and server.

jsjoeio · 2022-10-13T20:22:17Z

Thank you both for the additional context. I chatted a bit offline with @bpmct. He was surprised removing the validation steps allows to template to be uploaded.

Variable validation has nothing to do with the Docker provider.

Maybe it's an issue with Terraform on Windows? We'd have to dig in.

Also, the provisioning itself happens using the Linux/Mac version of the Terraform provider on the server but I guess it’s uploaded by the Windows coder client?

Additional notes shared by Ben.

I'm going to ping a couple other team members and see if they have insight. Thanks all for your patience!

jsjoeio · 2022-10-13T21:39:18Z

yes, I face this issue while using ADD in my docker file. It does not see any files in the images/ directory.

Thank you! I wonder if that's a separate issue.

jsjoeio · 2022-10-13T21:49:34Z

I'm digging into this more to make sure we can identify what the actual problem is.

My first thought is, "Can I create workspace using the docker template on a Windows client?"

Steps to reproduce

coder template init -> select "docker"
cd .\docker\ && coder template create
coder create --template="docker" test-docker

Expected
It builds the workspace fine and I can ssh into it.

Actual
Fails with error:

✔ Starting workspace [1258ms]
✘ Cleaning Up [119ms]
send job update: insert job logs: pq: invalid byte sequence for encoding "UTF8": 0x00
Run 'coder create --help' for usage.

Workspace Logs from UI:

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Terraform 1.3.0
data.coder_workspace.me: Refreshing...
data.coder_provisioner.me: Refreshing...
data.coder_provisioner.me: Refresh complete after 0s [id=eface01e-4cae-47b5-8b70-71db4f0c2f9b]
data.coder_workspace.me: Refresh complete after 0s [id=27cf56ec-2fc7-4b32-bba2-dd8fef127d74]
coder_agent.main: Plan to create
coder_app.code-server: Plan to create
docker_volume.home_volume: Plan to create
docker_image.main: Plan to create
docker_container.workspace[0]: Plan to create
Plan: 5 to add, 0 to change, 0 to destroy.
coder_agent.main: Creating...
coder_agent.main: Creation complete after 0s [id=dd7195be-2ed7-4b87-81a8-51d52cf4fd12]
docker_volume.home_volume: Creating...
coder_app.code-server: Creating...
docker_image.main: Creating...
coder_app.code-server: Creation complete after 0s [id=3fa2a7de-3f02-44c8-a922-aa3c39b476fc]
docker_volume.home_volume: Creation complete after 0s [id=coder-jp-test-docker-home]
docker_image.main: Creation errored after 0s

"What is docker_image.main?"
Here is the code:

resource "docker_image" "main" {
  name = "coder-${data.coder_workspace.me.id}"
  build {
    path = "./build"
  }
  triggers = {
    dir_sha1 = sha1(join("", [for f in fileset(path.module, "build/*") : filesha1(f)]))
  }
}

As you can see, this is relying on a subdirectory ./build in the template. That's one common denominator between this and the docker-image-builds failure.

jsjoeio · 2022-10-13T21:54:59Z

"Now let's try the docker-code-server template which doesn't rely on any subdirectories"

Steps to reproduce

coder template init -> select "docker-code-server"
cd .\docker\ && coder template create
coder create --template="docker-code-server" test-cs

Expected
It builds the workspace fine and I can ssh into it.

Actual
It builds the workspace fine and I can ssh into it.

So here, not using subdirectories, everything works as expected ✅

jsjoeio · 2022-10-13T21:57:05Z

Thus far, we have two factors that may be related to this:

Windows client
subdirectories in the template

Let's see if we can eliminate the first one.

"Can I create workspace using the docker template on a macOS client?"

WIP

On Windows, files in tar archives were stored with Windows path-separators resulting in them being individual files as opposed to contained in a folder. This commit ensures Unix-based paths (slash) are being used inside tar archives. Exmple of previous output: ``` /tmp/provisionerd673501182/images: /tmp/provisionerd673501182/: README.md images images\base.Dockerfile images\java.Dockerfile images\node.Dockerfile main.tf ``` Fixes #2815

mafredri · 2022-10-14T09:21:40Z

The issue seems to have been that we were saving Windows-based paths (backslash) vs Unix-based (slash) in the tar archives, fix in #4548.

@jsjoeio I'm unsure if this helps with the other issues you discovered?

matifali · 2022-10-14T09:28:20Z

What about Dockerfile unable to use any ADD commands because it can not see the external files?

mafredri · 2022-10-14T09:31:48Z

@matifali that should also be fixed by this since the files will now be in the correct folder.

matifali · 2022-10-14T09:38:43Z

@matifali that should also be fixed by this since the files will now be in the correct folder.

but I faced that on Linux client. so a path fix. I will test again after you merge.

mafredri · 2022-10-14T09:50:48Z

@matifali if the archive was created on Windows, then it would affect a Linux client as well. But if the flow was Linux -> Linux then this change would have no effect. If that's still a problem for you we should perhaps move it to a new separate issue?

bpmct · 2022-10-14T12:44:55Z

I originally thought this was a Windows-only issue but I may have missed something!

mafredri · 2022-10-14T13:06:41Z

@bpmct I would agree with that assessment. However, it can affect other OS-es via Windows systems.

So let's say you have the following structure:

data/bootstrap.sh
Dockerfile
main.tf

Relevant directive in Dockerfile would be ADD data/bootstrap.sh.

Iff this template is uploaded by Windows, then it would succeed (assuming main.tf doesn't do any special checks) but the tar file will not contain data/bootstrap.sh, instead it will be named data\\bootstrap.sh (single file, no folder).

Now when a Linux client tries to create the workspace and triggers the Docker build, it can't find the file.

^ This is the only way I see this issue affecting other operating systems.

jsjoeio · 2022-10-14T16:14:34Z

Nice find @mafredri 🙌🏼

Once it's merged and a new release is cut, I'll re-test and verify the fix.

On Windows, files in tar archives were stored with Windows path-separators resulting in them being individual files as opposed to contained in a folder. This commit ensures Unix-based paths (slash) are being used inside tar archives. Exmple of previous output: ``` /tmp/provisionerd673501182/images: /tmp/provisionerd673501182/: README.md images images\base.Dockerfile images\java.Dockerfile images\node.Dockerfile main.tf ``` Fixes #2815

phorcys420 · 2022-10-17T16:00:09Z

I can confirm this is a Windows-only issue, just tried this on a Linux host and it worked just fine.
I will confirm whether it works or not on windows with the next release.

jsjoeio · 2022-10-17T21:20:36Z

This is out now: https://github.com/coder/coder/releases/tag/v0.10.0

phorcys420 · 2022-10-18T08:24:08Z

Work for me on Windows with v0.10.1 !

jsjoeio · 2022-10-18T15:18:49Z

Woohoo! Thanks for confirming!

wale changed the title ~~coder templates create using the template docker-image-builds fails to build~~ coder templates create using the template docker-image-builds and derivatives fails to build Jul 4, 2022

kylecarbs mentioned this issue Jul 19, 2022

July Roadmap 2022 #3042

Closed

20 tasks

tjcran added the bug label Jul 21, 2022

tjcran mentioned this issue Jul 29, 2022

August Roadmap 2022 #3182

Closed

25 tasks

misskniss assigned dwahler Aug 9, 2022

This comment was marked as outdated.

Sign in to view

kylecarbs unassigned dwahler Aug 24, 2022

bpmct added the s0 Major regression, all-hands-on-deck to fix label Oct 10, 2022

jsjoeio self-assigned this Oct 11, 2022

mafredri mentioned this issue Oct 14, 2022

fix: Fix template create with sub-folders on Windows #4548

Merged

mafredri closed this as completed in #4548 Oct 14, 2022

bpmct assigned mafredri Oct 17, 2022

matifali added the bug label Jun 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`coder templates create` using the template `docker-image-builds` and derivatives fails to build #2815

`coder templates create` using the template `docker-image-builds` and derivatives fails to build #2815

wale commented Jul 4, 2022 •

edited by misskniss

Loading

wale commented Jul 6, 2022

phorcys420 commented Jul 20, 2022

bpmct commented Jul 25, 2022

phorcys420 commented Jul 25, 2022 •

edited

Loading

bpmct commented Jul 25, 2022 •

edited

Loading

phorcys420 commented Jul 25, 2022

phorcys420 commented Aug 1, 2022

This comment was marked as outdated.

phorcys420 commented Oct 2, 2022

matifali commented Oct 8, 2022 •

edited

Loading

phorcys420 commented Oct 11, 2022

matifali commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

phorcys420 commented Oct 12, 2022

matifali commented Oct 12, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022 •

edited

Loading

jsjoeio commented Oct 13, 2022

mafredri commented Oct 14, 2022

matifali commented Oct 14, 2022

mafredri commented Oct 14, 2022

matifali commented Oct 14, 2022

mafredri commented Oct 14, 2022

bpmct commented Oct 14, 2022

mafredri commented Oct 14, 2022

jsjoeio commented Oct 14, 2022

phorcys420 commented Oct 17, 2022

jsjoeio commented Oct 17, 2022

phorcys420 commented Oct 18, 2022

jsjoeio commented Oct 18, 2022

coder templates create using the template docker-image-builds and derivatives fails to build #2815

coder templates create using the template docker-image-builds and derivatives fails to build #2815

Comments

wale commented Jul 4, 2022 • edited by misskniss Loading

Problem

wale commented Jul 6, 2022

phorcys420 commented Jul 20, 2022

bpmct commented Jul 25, 2022

phorcys420 commented Jul 25, 2022 • edited Loading

bpmct commented Jul 25, 2022 • edited Loading

phorcys420 commented Jul 25, 2022

phorcys420 commented Aug 1, 2022

This comment was marked as outdated.

phorcys420 commented Oct 2, 2022

matifali commented Oct 8, 2022 • edited Loading

phorcys420 commented Oct 11, 2022

matifali commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

jsjoeio commented Oct 11, 2022

phorcys420 commented Oct 12, 2022

matifali commented Oct 12, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

jsjoeio commented Oct 13, 2022 • edited Loading

jsjoeio commented Oct 13, 2022

mafredri commented Oct 14, 2022

matifali commented Oct 14, 2022

mafredri commented Oct 14, 2022

matifali commented Oct 14, 2022

mafredri commented Oct 14, 2022

bpmct commented Oct 14, 2022

mafredri commented Oct 14, 2022

jsjoeio commented Oct 14, 2022

phorcys420 commented Oct 17, 2022

jsjoeio commented Oct 17, 2022

phorcys420 commented Oct 18, 2022

jsjoeio commented Oct 18, 2022

`coder templates create` using the template `docker-image-builds` and derivatives fails to build #2815

`coder templates create` using the template `docker-image-builds` and derivatives fails to build #2815

wale commented Jul 4, 2022 •

edited by misskniss

Loading

phorcys420 commented Jul 25, 2022 •

edited

Loading

bpmct commented Jul 25, 2022 •

edited

Loading

matifali commented Oct 8, 2022 •

edited

Loading

jsjoeio commented Oct 13, 2022 •

edited

Loading