Skip to content

bug: template tarball is missing "plan" part #10599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
sreya opened this issue Nov 8, 2023 · 7 comments
Closed

bug: template tarball is missing "plan" part #10599

sreya opened this issue Nov 8, 2023 · 7 comments
Assignees
@mtojek
Labels
s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks

Comments

@sreya
Copy link
Collaborator

sreya commented Nov 8, 2023 

  metricscollector_test.go:137: 
       	Error Trace:	/home/runner/actions-runner/_work/coder/coder/coderd/prometheusmetrics/insights/metricscollector_test.go:137
       	Error:      	Condition never satisfied
       	Test:       	TestCollectInsights
       	Messages:   	template insights are inconsistent with golden files, got: map[coderd_insights_applications_usage_seconds[application_name=Golden Slug,slug=golden-slug,template_name=golden-template]:120 coderd_insights_applications_usage_seconds[application_name=JetBrains,slug=,template_name=golden-template]:0 coderd_insights_applications_usage_seconds[application_name=SSH,slug=,template_name=golden-template]:60 coderd_insights_applications_usage_seconds[application_name=Visual Studio Code,slug=,template_name=golden-template]:0 coderd_insights_applications_usage_seconds[application_name=Web Terminal,slug=,template_name=golden-template]:0 coderd_insights_templates_active_users[template_name=golden-template]:1]

Seen here
@cdr-bot cdr-bot bot added the chore label Nov 8, 2023
@sreya sreya added flake and removed chore labels Nov 8, 2023
@matifali matifali added the s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks label Nov 9, 2023
@mtojek
Copy link
Member

mtojek commented Nov 16, 2023 •
edited
Loading

This is suspicious: it looks like provisioner ignored/didn't receive rich_parameter_values:

t.go:84: 2023-11-08 22:07:59.655 [debu]  provisionerd.runner: acquired job is workspace provision  job_id=53423578-5242-47de-bf92-4fd2fda6f31c  template_name=golden-template  template_version=busy_blackwell2  workspace_build_id=5c70e1fe-5417-4003-b202-afbded83eb3a  workspace_id=9adb0895-5591-42aa-b29b-0d6253dadff4  workspace_name=amazing-hellman2-cvT  workspace_owner=testuser  workspace_transition=start  workspace_name=amazing-hellman2-cvT  state_length=0  rich_parameter_values=[]  variable_values=[]

hence no parameter-related metrics. When I'm running it locally, rich_parameter_values is populated:

/Users/mtojek/code/coder/coderd/prometheusmetrics/insights/t.go:84: 2023-11-16 10:51:40.107 [debu]  provisionerd.runner: acquired job is workspace provision  job_id=db5e6d57-0b1c-4564-b5e9-a92e56bd99c5  template_name=golden-template  template_version=vigilant_knuth9  workspace_build_id=b3e451b9-1a21-4908-924a-1b72db4ef026  workspace_id=c08461ea-57e6-458b-8eda-9e20ab8cf8f3  workspace_name=inspiring-torvalds9-4ju  workspace_owner=testuser  workspace_transition=start  workspace_name=inspiring-torvalds9-4ju  state_length=0  rich_parameter_values="[name:\"first_parameter\" value:\"Foobar\" name:\"second_parameter\" value:\"true\" name:\"third_parameter\" value:\"789\"]"  variable_values=[]

EDIT:

These log lines are missing too:

2023-11-16 10:51:39.850 [info]  inmem-provisionerd-funny_solomon2: inserting template import job parameter  job_id=82148863-a8ed-4b2b-992e-dc5af703b64e  parameter_name=first_parameter  type=string  ephemeral=false
    /Users/mtojek/code/coder/coderd/prometheusmetrics/insights/t.go:84: 2023-11-16 10:51:39.851 [debu]  echo: cleaned up work directory  session_id=331cc747-10e2-4fa5-a8fc-45595b8dc3f0
2023-11-16 10:51:39.851 [info]  inmem-provisionerd-funny_solomon2: inserting template import job parameter  job_id=82148863-a8ed-4b2b-992e-dc5af703b64e  parameter_name=second_parameter  type=bool  ephemeral=false
2023-11-16 10:51:39.851 [info]  inmem-provisionerd-funny_solomon2: inserting template import job parameter  job_id=82148863-a8ed-4b2b-992e-dc5af703b64e  parameter_name=third_parameter  type=number  ephemeral=false

@mtojek
Copy link
Member

mtojek commented Nov 16, 2023 •
edited
Loading

Another finding:

expected:

2023-11-16 10:51:39.849 [debu]  echo: extracted file  session_id=331cc747-10e2-4fa5-a8fc-45595b8dc3f0  size_bytes=2  path=/var/folders/82/7tbcycl16pz_7rxv981qdqsh0000gn/T/TestCollectInsights2469533705/002/Session331cc747-10e2-4fa5-a8fc-45595b8dc3f0/0.parse.protobuf  mode=-rw-r--r--
2023-11-16 10:51:39.849 [debu]  echo: extracted file  session_id=331cc747-10e2-4fa5-a8fc-45595b8dc3f0  size_bytes=168  path=/var/folders/82/7tbcycl16pz_7rxv981qdqsh0000gn/T/TestCollectInsights2469533705/002/Session331cc747-10e2-4fa5-a8fc-45595b8dc3f0/0.apply.protobuf  mode=-rw-r--r--
2023-11-16 10:51:39.849 [debu]  echo: extracted file  session_id=331cc747-10e2-4fa5-a8fc-45595b8dc3f0  size_bytes=88  path=/var/folders/82/7tbcycl16pz_7rxv981qdqsh0000gn/T/TestCollectInsights2469533705/002/Session331cc747-10e2-4fa5-a8fc-45595b8dc3f0/0.plan.protobuf  mode=-rw-r--r--

but was:

2023-11-08 22:07:59.399 [debu]  echo: extracted file  session_id=b61c66ef-5856-4298-b32d-b3cdc294a8ce  size_bytes=2  path=/tmp/TestCollectInsights2132138616/002/Sessionb61c66ef-5856-4298-b32d-b3cdc294a8ce/0.parse.protobuf  mode=-rw-r--r--
2023-11-08 22:07:59.399 [debu]  echo: extracted file  session_id=b61c66ef-5856-4298-b32d-b3cdc294a8ce  size_bytes=168  path=/tmp/TestCollectInsights2132138616/002/Sessionb61c66ef-5856-4298-b32d-b3cdc294a8ce/0.apply.protobuf  mode=-rw-r--r--
2023-11-08 22:07:59.399 [debu]  echo: extracted file  session_id=b61c66ef-5856-4298-b32d-b3cdc294a8ce  size_bytes=2  path=/tmp/TestCollectInsights2132138616/002/Sessionb61c66ef-5856-4298-b32d-b3cdc294a8ce/0.plan.protobuf  mode=-rw-r--r--

File 0.plan.protobuf is empty or chopped.

@mtojek mtojek mentioned this issue Nov 16, 2023
Merged
@mtojek mtojek changed the title flake: TestCollectInsights bug: template tarball is missing "plan" part Nov 16, 2023
@mtojek
Copy link
Member

mtojek commented Nov 16, 2023

Enhanced logging is merged now. Let me know if you observe a similar error, so we can continue the investigation.

@spikecurtis
Copy link
Contributor

https://github.com/coder/coder/runs/18774973730

@mtojek
Copy link
Member

mtojek commented Nov 17, 2023

This is a different root cause as the parameters are present:

        	Messages:   	template insights are inconsistent with golden files, got: map[coderd_insights_applications_usage_seconds[application_name=Golden Slug,slug=golden-slug,template_name=golden-template]:120 coderd_insights_applications_usage_seconds[application_name=JetBrains,slug=,template_name=golden-template]:0 coderd_insights_applications_usage_seconds[application_name=SSH,slug=,template_name=golden-template]:60 coderd_insights_applications_usage_seconds[application_name=Visual Studio Code,slug=,template_name=golden-template]:0 coderd_insights_applications_usage_seconds[application_name=Web Terminal,slug=,template_name=golden-template]:0 coderd_insights_parameters[parameter_name=first_parameter,parameter_type=string,parameter_value=Foobar,template_name=golden-template]:1 coderd_insights_parameters[parameter_name=second_parameter,parameter_type=bool,parameter_value=true,template_name=golden-template]:1 coderd_insights_parameters[parameter_name=third_parameter,parameter_type=number,parameter_value=789,template_name=golden-template]:1 coderd_insights_templates_active_users[template_name=golden-template]:1]

map[coderd_insights_applications_usage_seconds[application_name=Golden Slug,slug=golden-slug,template_name=golden-template]:120

it looks like one metric slipped here, most likely due to the truncation.

I will disable the test for now.

This was referenced Nov 17, 2023
Merged
Closed
Merged
@mtojek
Copy link
Member

mtojek commented Nov 20, 2023

I merged a fix for the second issue in tests and will move the task to "blocked" until we get more data for the first one.

@mtojek
Copy link
Member

mtojek commented Nov 23, 2023

I haven't seen any reports so closing for now.

@mtojek mtojek closed this as completed Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mtojek mtojek

Labels
s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sreya @spikecurtis @matifali @mtojek