Skip to content

test flake: TestProvisionerd/MaliciousTar #9895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
johnstcn opened this issue Sep 27, 2023 · 0 comments · Fixed by #10158
Closed

test flake: TestProvisionerd/MaliciousTar #9895

johnstcn opened this issue Sep 27, 2023 · 0 comments · Fixed by #10158
Assignees
@spikecurtis
Labels
s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks

Comments

@johnstcn
Copy link
Member

johnstcn commented Sep 27, 2023 •
edited
Loading

Seen here: https://github.com/coder/coder/actions/runs/6328705590/job/17187479528?pr=9894

Looks like the testServer got an unexpected EOF when sending the AcquiredJob with the payload:

					err := stream.Send(&proto.AcquiredJob{
						JobId:       "test",
						Provisioner: "someprovisioner",
						TemplateSourceArchive: createTar(t, map[string]string{
							"../../../etc/passwd": "content",
						}),
						Type: &proto.AcquiredJob_TemplateImport_{
							TemplateImport: &proto.AcquiredJob_TemplateImport{
								Metadata: &sdkproto.Metadata{},
							},
						},
					})
					assert.NoError(t, err) <--here
Output: 
=== Failed
=== FAIL: provisionerd TestProvisionerd/MaliciousTar (0.03s)
    t.go:84: 2023-09-27 16:25:42.876 [debu]  provisionerd: dialing coderd
    t.go:84: 2023-09-27 16:25:42.876 [info]  provisionerd: successfully connected to coderd
    t.go:84: 2023-09-27 16:25:42.876 [debu]  provisionerd: start of acquireAndRunOne
    t.go:84: 2023-09-27 16:25:42.890 [debu]  provisionerd: graceful acquire done  job_id=test  error=<nil>
    t.go:84: 2023-09-27 16:25:42.890 [debu]  provisionerd: acquired job  initiator_username=""  provisioner=someprovisioner  job_id=test
    t.go:84: 2023-09-27 16:25:42.891 [debu]  provisionerd.runner: acquired job is template import  job_id=test  user_variable_values=[]
    t.go:84: 2023-09-27 16:25:42.892 [info]  test-provisioner: unpacking template source archive  session_id=0b0e1ca9-3785-457b-9bc5-90f7f7d17bad  size_bytes=1024
    t.go:84: 2023-09-27 16:25:42.892 [debu]  test-provisioner: cleaned up work directory  session_id=0b0e1ca9-3785-457b-9bc5-90f7f7d17bad
    t.go:84: 2023-09-27 16:25:42.896 [debu]  provisionerd.runner: sending FailedJob  job_id=test
    t.go:84: 2023-09-27 16:25:42.897 [debu]  provisionerd.runner: sent FailedJob  job_id=test
    t.go:84: 2023-09-27 16:25:42.898 [debu]  provisionerd: start of acquireAndRunOne
    t.go:84: 2023-09-27 16:25:42.897 [info]  provisionerd: closing provisionerd
    t.go:84: 2023-09-27 16:25:42.900 [debu]  provisionerd: waiting for goroutines to exit
    t.go:84: 2023-09-27 16:25:42.900 [debu]  provisionerd: connect loop exited
    provisionerd_test.go:166: 
        	Error Trace:	/home/runner/actions-runner/_work/coder/coder/provisionerd/provisionerd_test.go:166
        	            				/home/runner/actions-runner/_work/coder/coder/provisionerd/provisionerd_test.go:1201
        	            				/home/runner/actions-runner/_work/coder/coder/provisionerd/proto/provisionerd_drpc.pb.go:197
        	            				/home/runner/go/pkg/mod/storj.io/drpc@v0.0.33-0.20230420154621-9716137f6037/drpcmux/handle_rpc.go:33
        	            				/home/runner/go/pkg/mod/storj.io/drpc@v0.0.33-0.20230420154621-9716137f6037/drpcserver/server.go:124
        	            				/home/runner/go/pkg/mod/storj.io/drpc@v0.0.33-0.20230420154621-9716137f6037/drpcserver/server.go:66
        	            				/home/runner/go/pkg/mod/storj.io/drpc@v0.0.33-0.20230420154621-9716137f6037/drpcserver/server.go:114
        	            				/home/runner/go/pkg/mod/storj.io/drpc@v0.0.33-0.20230420154621-9716137f6037/drpcctx/tracker.go:35
        	            				/opt/hostedtoolcache/go/1.20.7/x64/src/runtime/asm_amd64.s:1598
        	Error:      	Received unexpected error:
        	            	EOF
        	Test:       	TestProvisionerd/MaliciousTar
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: graceful acquire done  job_id=""  error="context canceled"
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: exiting acquire; provisionerd is closing
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: acquire loop exited
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: closing server with error  error=<nil>
    t.go:84: 2023-09-27 16:25:42.901 [info]  provisionerd: attempting graceful shutdown
    t.go:84: 2023-09-27 16:25:42.901 [info]  provisionerd: gracefully shutdown
    t.go:84: 2023-09-27 16:25:42.901 [info]  provisionerd: closing provisionerd
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: waiting for first closer to complete
    t.go:84: 2023-09-27 16:25:42.901 [debu]  provisionerd: first closer completed

=== FAIL: provisionerd TestProvisionerd (0.00s)
@johnstcn johnstcn added flake s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks labels Sep 27, 2023
@sreya sreya assigned code-asher and deansheather and unassigned code-asher Oct 2, 2023
@spikecurtis spikecurtis mentioned this issue Oct 10, 2023
Merged
spikecurtis added a commit that referenced this issue Oct 10, 2023
@spikecurtis
fix: fix MaliciousTar test case (#10158)
211718f 
fixes #9895

Problem was that provisionerd tries to acquire the next job, and races with shutdown, triggering the assert in the handler.  Switches this test case to use the more robust handler.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spikecurtis spikecurtis

Labels
s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: fix MaliciousTar test case coder/coder
4 participants
@johnstcn @spikecurtis @deansheather @code-asher