Skip to content

flaky: gotestsum can't find tag "--- PASS:" #5247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mtojek opened this issue Dec 2, 2022 · 2 comments · Fixed by #5336
Closed

flaky: gotestsum can't find tag "--- PASS:" #5247

mtojek opened this issue Dec 2, 2022 · 2 comments · Fixed by #5336
Assignees
@mtojek

Comments

@mtojek
Copy link
Member

mtojek commented Dec 2, 2022 •
edited
Loading

Spotted in: https://github.com/coder/coder/actions/runs/3598057251/jobs/6060563465

=== Failed
=== FAIL: enterprise/cli TestLicensesAddFake (unknown)
    ptytest.go:98: 2022-12-02 01:43:02.532: cmd: stdout: "License with ID 1 added"
    licenses_test.go:49: 2022-12-02 01:43:02.532: cmd: matched "License with ID 1 added" = "License with ID 1 added"
    licenses_test.go:61: 2022-12-02 01:43:02.563: cmd: matched "Paste license:" = "> Paste license:"
    licenses_test.go:62: 2022-12-02 01:43:02.563: cmd: stdin: "test.jwt.sig\r\n"
    ptytest.go:98: 2022-12-02 01:43:02.563: cmd: stdout: "> Paste license: License with ID 1 added"
    licenses_test.go:64: 2022-12-02 01:43:02.563: cmd: matched "License with ID 1 added" = " License with ID 1 added"
    ptytest.go:98: 2022-12-02 01:43:02.554: cmd: stdout: "License with ID 1 added"
    licenses_test.go:81: 2022-12-02 01:43:02.554: cmd: matched "License with ID 1 added" = "License with ID 1 added"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "{"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "  \"id\": 1,"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "  \"uuid\": \"00000000-0000-0000-0000-000000000000\","
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "  \"uploaded_at\": \"2022-12-02T01:43:02.5502473Z\","
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "  \"claims\": {"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "    \"features\": {"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "      \"f1\": 1,"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "      \"f2\": 2"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "    },"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "    \"h1\": \"claim1\""
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "  }"
    ptytest.go:98: 2022-12-02 01:43:02.551: cmd: stdout: "}"
    licenses_test.go:119: 2022-12-02 01:43:02.551: cmd: matched "\"f2\": 2" = "{\n  \"id\": 1,\n  \"uuid\": \"00000000-0000-0000-0000-000000000000\",\n  \"uploaded_at\": \"2022-12-02T01:43:02.550[247](https://github.com/coder/coder/actions/runs/3598057251/jobs/6060563465#step:9:248)3Z\",\n  \"claims\": {\n    \"features\": {\n      \"f1\": 1,\n      \"f2\": 2"

... and few more inside.

Apparently, all test cases are valid and passed.
@mtojek mtojek self-assigned this Dec 2, 2022
@mtojek mtojek added the flake label Dec 2, 2022
@mtojek mtojek mentioned this issue Dec 2, 2022
Merged
@mtojek mtojek changed the title flaky: enterprise/cli TestLicensesAddFake flaky: gotestsum can't find tag "--- PASS:" Dec 2, 2022
@mtojek
Copy link
Member Author

mtojek commented Dec 2, 2022

It seems to me like the outcome of gotestyourself/gotestsum#141.

I suspect that there is an interleave between some logger calls, and gotestsum can't find the marker --- PASS:. Let's enable --debug mode for now and try to find that place.

@mafredri mafredri mentioned this issue Dec 2, 2022
Closed
mafredri added a commit that referenced this issue Dec 2, 2022
@mafredri
fix: Ensure pty is closed in licenses tests
9d4d09f 
The license tests fail inexplicably on Windows, this may be because we
are leaving pty's behind.

References #5247
@mafredri mafredri mentioned this issue Dec 2, 2022
Merged
@mtojek
Copy link
Member Author

mtojek commented Dec 6, 2022 •
edited
Loading

Builds:
https://github.com/coder/coder/actions/runs/3629622036/jobs/6122380989
https://github.com/coder/coder/actions/runs/3630060378/jobs/6122972527

Most likely these builds had like similar problems. Matchers matched, but tests failed.

This was referenced Dec 6, 2022
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mtojek mtojek

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore: collect gotestsum TestEvents as workflow artifacts mtojek/coder
1 participant
@mtojek