Skip to content

fix(provisioner/terraform/tfparse): skip evaluation of unrelated parameters #16023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 3, 2025

Conversation

johnstcn
Copy link
Member

@johnstcn johnstcn commented Jan 3, 2025

Fixes #16021

tfparse had been enforcing the "no functions, no locals" rule for all coder_parameter default values, where we only want to enforce this for ones referenced by coder_workspace_tags.

  • Improves tfparse test coverage to include more parameter types and values
  • Adds tests with unrelated parameters that should be ignored by tfparse
  • Modifies tfparse to only attempt evaluation of variables/parameters referenced by coder_workspace_tags

@johnstcn johnstcn self-assigned this Jan 3, 2025
@johnstcn johnstcn changed the title fix(provisioner/terraform/tfparse): skip evaluation of defaults if no workspace_tags found fix(provisioner/terraform/tfparse): skip evaluation of unrelated parameters Jan 3, 2025
"stringparam": "a",
"numparam": "1",
"boolparam": "true",
"listparam": `["a", "b"]`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: this ends up being allowed due to the default of coder_parameter being a string.

}`,
},
expectTags: nil,
expectError: `can't convert variable default value to string: unsupported type []interface {}`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: this is also not allowed by the provider:

│   on main.tf line 38, in data "coder_workspace_tags" "tags":
│   38:   tags = {
│   39:     "listvar" = ["a"]
│   40:   }
│
│ Inappropriate value for attribute "tags": element "listvar": string required.

}`,
},
expectTags: nil,
expectError: `can't convert variable default value to string: unsupported type map[string]interface {}`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: also not allowed by the provider:

│   on main.tf line 38, in data "coder_workspace_tags" "tags":
│   38:   tags = {
│   39:     "mapvar" = {a = "b"}
│   40:   }
│
│ Inappropriate value for attribute "tags": element "mapvar": string required.

@@ -290,6 +305,18 @@ func (p *Parser) CoderParameterDefaults(ctx context.Context, varsDefaults map[st
continue
}

var found bool
needle := strings.Join([]string{"data", dataResource.Type, dataResource.Name}, ".")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, there is no way to break a variable reference over multiple lines e.g.

data.\
coder_parameter.\
foo.\
value

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct 👍

@johnstcn johnstcn requested review from mtojek and SasSwart January 3, 2025 12:25
Copy link
Member

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve this change although I'm concerned about the overall parsing logic. With another patch it will become harder to debug or touch internals in the future.

@@ -290,6 +305,18 @@ func (p *Parser) CoderParameterDefaults(ctx context.Context, varsDefaults map[st
continue
}

var found bool
needle := strings.Join([]string{"data", dataResource.Type, dataResource.Name}, ".")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct 👍

Comment on lines 265 to 269
for _, tv := range tags {
if strings.Contains(tv, v.Name) {
found = true
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could extract this block and the one below to a common helper function

@johnstcn johnstcn force-pushed the cj/16021/skip-no-workspace-tags branch from ce68fe2 to e56cac7 Compare January 3, 2025 13:19
@johnstcn
Copy link
Member Author

johnstcn commented Jan 3, 2025

Smoke-tested on personal deployment with example from linked issue.

Screen.Recording.2025-01-03.at.13.27.45.mov

Note that issue described in #16022 is still present if that parameter is referenced in coder_workspace_tags.

m := make(map[string]string)
for _, v := range p.module.Variables {
if v == nil {
continue
}
var found bool
for _, tv := range tags {
if strings.Contains(tv, v.Name) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if strings.Contains(tv, v.Name) {
if strings.Contains(tv, "var."+v.Name) {

m := make(map[string]string)
for _, v := range p.module.Variables {
if v == nil {
continue
}
var found bool
for _, tv := range tags {
if strings.Contains(tv, v.Name) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

won't a strings.Contains also contain variable names that are subsets?

Like var.Foo will match both Foo and var.FooBar

Copy link
Member

@Emyrk Emyrk Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a test case:

{
   name: "overlapping var name",
   files: map[string]string{
      `main.tf`: `
      // This file is the same as the above, except for this comment.
      variable "a" {
         type = string
         default = "1"
      }
      variable "ab" {
       description = "This is a variable of type string"
       type        = string
       default     = jsonencode(["a", "b"])
      }
      data "coder_workspace_tags" "tags" {
         tags = {
            "foo": "bar",
            "a": var.a,
         }
      }`,
   },
   reqTags:  map[string]string{"foo": "noclobber"},
   wantTags: map[string]string{"owner": "", "scope": "organization", "foo": "bar", "a": "1"},
},

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, we may then need to do some more "intelligent" parsing.
I hesitate to reach for a regexp.... but \b may be exactly what we need here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the reason you do a contains because it could be an expression like foo(var.name)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to parse the expression and get the actual tokens? Or we only have string in the HCL?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I'm looking into HCL traversals now as they might allow us to do this cleanly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've modified the implementation to instead:

  • Gather the set of variables referenced by the tags up-front,
  • Only evaluate the default value(s) for coder_parameters referenced by those tags.
  • As default values of coder_parameters can reference other variables, to avoid needing to perform a graph traversal of all the dependencies here I've elected to simply continue passing in all the variable default values. We don't immediately enforce default values for all variables.
  • To avoid unrelated variables with default values of complex types being rejected, I added some 'last-ditch' support for JSON-encoding them.

@johnstcn johnstcn requested a review from Emyrk January 3, 2025 18:43
@johnstcn johnstcn merged commit 1ab10cf into main Jan 3, 2025
30 checks passed
@johnstcn johnstcn deleted the cj/16021/skip-no-workspace-tags branch January 3, 2025 19:32
@github-actions github-actions bot locked and limited conversation to collaborators Jan 3, 2025
@matifali
Copy link
Member

matifali commented Jan 6, 2025

/cherry-pick release-2.18

@matifali
Copy link
Member

/cherry-pick release/2.18

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

coder_parameter regression: cannot use lists and functions anymore
5 participants