Repo: Failure on main: Run Unit Tests with Experimental TSServer (eslint-plugin) #8131

JoshuaKGoldberg · 2023-12-25T14:53:15Z

Suggestion

It started after merging #7752. https://github.com/typescript-eslint/typescript-eslint/actions/runs/7317242945/job/19932592486:

Summary of all failing tests
 FAIL  tests/rules/no-unused-vars/no-unused-vars-eslint.test.ts (17.679 s, 1025 MB heap size)
  ● no-unused-vars › invalid › 
/*global ����, ��*/
����;
      
    assert(received)
    Expected value to be equal to:
      true
    Received:
      false
    
    Message:
      A fatal parsing error occurred: Parsing error: Invalid character.

      356 |         }
      357 |         const fatalErrorMessage = messages.find(m => m.fatal);
    > 358 |         (0, node_assert_1.default)(!fatalErrorMessage, `A fatal parsing error occurred: ${fatalErrorMessage?.message}`);
          |                                 ^
      359 |         // Verify if autofix makes a syntax error or not.
      360 |         if (messages.some(m => m.fix)) {
      361 |             output = SourceCodeFixer.applyFixes(code, messages).output;

      at RuleTester.runRuleForItem (../rule-tester/dist/RuleTester.js:358:33)
      at RuleTester.runRuleForItem (../rule-tester/dist/RuleTester.js:503:25)
    Message:
      A fatal parsing error occurred: Parsing error: Invalid character.

      356 |         }
      357 |         const fatalErrorMessage = messages.find(m => m.fatal);
    > 358 |         (0, node_assert_1.default)(!fatalErrorMessage, `A fatal parsing error occurred: ${fatalErrorMessage?.message}`);
          |                                 ^
      359 |         // Verify if autofix makes a syntax error or not.
      360 |         if (messages.some(m => m.fix)) {
      361 |             output = SourceCodeFixer.applyFixes(code, messages).output;

      at RuleTester.runRuleForItem (../rule-tester/dist/RuleTester.js:358:33)
      at RuleTester.runRuleForItem (../rule-tester/dist/RuleTester.js:503:25)
      at Object.call (../rule-tester/dist/RuleTester.js:237:119)
 FAIL  tests/eslint-rules/no-undef.test.ts (13.747 s, 631 MB heap size)
  ● no-undef › valid › 
interface IteratorCallback<Subject, Key, Value> {
  (this: Subject, value: Value, key: Key, subject: Subject): void | false;
}
function eachr<Value>(
  subject: Array<Value>,
  callback: IteratorCallback<typeof subject, number, Value>,
): typeof subject;
function eachr<Key, Value>(
  subject: Map<Key, Value>,
  callback: IteratorCallback<typeof subject, Key, Value>,
): typeof subject;
function eachr<Object extends object, Key extends keyof Object>(
  subject: Object,
  callback: IteratorCallback<Object, Key, Object[Key]>,
): Object;
function eachr<Object extends object, Key, Value>(
  subject: Object | Array<Value> | Map<Key, Value>,
  callback: IteratorCallback<typeof subject, Key, Value>,
): typeof subject {
  return subject;
}
    
    assert.strictEqual(received, expected)
    Expected value to strictly be equal to:
      0
    Received:
      2
    
    Message:
      Should have no errors but had 2: [
      {
        ruleId: 'no-undef',
        severity: 2,
        message: "'Map' is not defined.",
        line: 10,
        column: 12,
        nodeType: 'Identifier',
        messageId: 'undef',
        endLine: 10,
        endColumn: 15
      },
      {
        ruleId: 'no-undef',
        severity: 2,
        message: "'Map' is not defined.",
        line: 18,
        column: 36,
        nodeType: 'Identifier',
        messageId: 'undef',
        endLine: 18,
        endColumn: 39
      }
    ]

      484 |     const result = this.runRuleForItem(ruleName, rule, item);
      485 |     const messages = result.messages;
    > 486 |     node_assert_1.default.strictEqual(messages.length, 0, node_util_1.default.format('Should have no errors but had %d: %s', messages.length, node_util_1.default.inspect(messages)));
          |                           ^
      487 |     assertASTDidntChange(result.beforeAST, result.afterAST);
      488 | }, _RuleTester_testInvalidTemplate = function _RuleTester_testInvalidTemplate(ruleName, rule, item) {
      489 |     node_assert_1.default.ok(typeof item.code === 'string', "Test case must specify a string value for 'code'");

      at RuleTester.strictEqual (../rule-tester/dist/RuleTester.js:486:27)
      at Object.call (../rule-tester/dist/RuleTester.js:222:117)
  ● no-undef › valid › 
function eachr<Key, Value>(subject: Map<Key, Value>): typeof subject;
    
    assert.strictEqual(received, expected)
    Expected value to strictly be equal to:
      0
    Received:
      1
    
    Message:
      Should have no errors but had 1: [
      {
        ruleId: 'no-undef',
        severity: 2,
        message: "'Map' is not defined.",
        line: 2,
        column: 37,
        nodeType: 'Identifier',
        messageId: 'undef',
        endLine: 2,
        endColumn: 40
      }
    ]

      484 |     const result = this.runRuleForItem(ruleName, rule, item);
      485 |     const messages = result.messages;
    > 486 |     node_assert_1.default.strictEqual(messages.length, 0, node_util_1.default.format('Should have no errors but had %d: %s', messages.length, node_util_1.default.inspect(messages)));
          |                           ^
      487 |     assertASTDidntChange(result.beforeAST, result.afterAST);
      488 | }, _RuleTester_testInvalidTemplate = function _RuleTester_testInvalidTemplate(ruleName, rule, item) {
      489 |     node_assert_1.default.ok(typeof item.code === 'string', "Test case must specify a string value for 'code'");

      at RuleTester.strictEqual (../rule-tester/dist/RuleTester.js:486:27)
      at Object.call (../rule-tester/dist/RuleTester.js:222:117)
Test Suites: 3 failed, 174 passed, 177 total
Tests:       5 failed, 1 skipped, 28600 passed, 28606 total
Snapshots:   136 passed, 136 total
Time:        822.794 s
Ran all test suites.

I honestly thought this was some kind of weird merge conflict. Seemed odd to me that the change would cause it. Sigh.

The text was updated successfully, but these errors were encountered:

JoshuaKGoldberg · 2023-12-25T18:49:03Z

I don't have much time today (it's Christmas!), but taking a brief look between activities...

The first one, /*global 𠮷𩸽, 𠮷*/, is the first of two parse diagnostics reported by TypeScript. Both are messageText: 'Invalid character.'.

[0]: start: 21, length: 2
[1]: start: 23, length: 2

The other two are from Map not being defined. Interesting.

bradzacher · 2023-12-25T21:51:59Z

@JoshuaKGoldberg if map isn't defined then perhaps the lib isn't defined?

auvred · 2023-12-26T17:12:31Z

I reverted this removal of the early return, and now all failed tests pass on my machine: https://github.com/typescript-eslint/typescript-eslint/pull/7752/files#diff-b089b5bc90fd2a532136946c025806e6daa02b7deda1f704829d736dfdb4b3c8L18-L19

Ref: #7752 (review)

JoshuaKGoldberg · 2023-12-26T18:15:28Z

Oy vey: #8136.

Proposal: to stop failures on main (and improve CI performance!), let's:

Merge fix(typescript-estree): only create project service from env setting if project is enabled #8136
File a followup to add a job that:
1. runs all unit tests with project & project service on main after each commit
2. pings us in some way (issue? Discord message?) on each failure

IMO having allowDefaultProjectForFiles is an important enough user-facing feature that we'd really want to leave it in.

Zamiell · 2024-01-04T22:59:03Z

Setting up GitHub Actions to put failures to Discord is pretty easy, I do it like this:

  # To enable CI failure notifications over Discord:
  # - Right click on a channel in Discord and select "Edit Channel".
  # - Click on "Integrations" on the left menu.
  # - Click on the "Create Webhook" button.
  # - Click on the box for the new webhook that was created.
  # - Change the name to "GitHub".
  # - Change the image to: https://github.com/IsaacScript/isaacscript/raw/main/misc/github.png
  # - Click on the "Save Changes" button at the bottom.
  # - Click on the "Copy Webhook URL" button.
  # - Go to the main page for your repository on GitHub.
  # - Click on the "Settings" tab near the top.
  # - Click on "Secrets and variables" in the left menu.
  # - Click on "Actions" from the dropdown list.
  # - Click on the "New repository secret" button in the top right.
  # - For the "Name" box, use "DISCORD_WEBHOOK" (without the quotes).
  # - For the "Secret" box, paste in the URL that was copied in the "Copy Webhook URL" step. (The
  #   pasted URL should not have a "/github" suffix.)
  # - Click on the "Add secret" button.
  discord:
    name: Discord Failure Notification
    needs: [foo1, foo2, foo3, foo4, foo5]
    if: always() # This is needed to always run this job, even if the other jobs fail.
    runs-on: ubuntu-latest
    steps:
      - uses: technote-space/workflow-conclusion-action@v3
      - if: env.WORKFLOW_CONCLUSION != 'success' && env.WORKFLOW_CONCLUSION != 'cancelled'
        uses: sarisia/actions-status-discord@v1
        with:
          webhook: ${{ secrets.DISCORD_WEBHOOK }}
          status: ${{ env.WORKFLOW_CONCLUSION }}
          title: ""

JoshuaKGoldberg added bug Something isn't working repo maintenance things to do with maintenance of the repo, and not with code/docs accepting prs Go ahead, send a pull request that resolves this issue labels Dec 25, 2023

JoshuaKGoldberg mentioned this issue Dec 26, 2023

fix(typescript-estree): only create project service from env setting if project is enabled #8136

Merged

3 tasks

auvred mentioned this issue Jan 2, 2024

chore: unpin primary node version in ci #8167

Merged

3 tasks

auvred mentioned this issue Jan 6, 2024

chore: use nx release #8194

Merged

JoshuaKGoldberg closed this as completed in #8136 Jan 7, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jan 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Repo: Failure on main: Run Unit Tests with Experimental TSServer (eslint-plugin) #8131

Repo: Failure on main: Run Unit Tests with Experimental TSServer (eslint-plugin) #8131

JoshuaKGoldberg commented Dec 25, 2023 •

edited

Loading

JoshuaKGoldberg commented Dec 25, 2023

Uh oh!

bradzacher commented Dec 25, 2023

Uh oh!

auvred commented Dec 26, 2023

Uh oh!

JoshuaKGoldberg commented Dec 26, 2023 •

edited

Loading

Uh oh!

Zamiell commented Jan 4, 2024 •

edited

Loading

Uh oh!

Uh oh!

Repo: Failure on main: Run Unit Tests with Experimental TSServer (eslint-plugin) #8131

Repo: Failure on main: Run Unit Tests with Experimental TSServer (eslint-plugin) #8131

Comments

JoshuaKGoldberg commented Dec 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Suggestion

JoshuaKGoldberg commented Dec 25, 2023

Uh oh!

bradzacher commented Dec 25, 2023

Uh oh!

auvred commented Dec 26, 2023

Uh oh!

JoshuaKGoldberg commented Dec 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Zamiell commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JoshuaKGoldberg commented Dec 25, 2023 •

edited

Loading

JoshuaKGoldberg commented Dec 26, 2023 •

edited

Loading

Zamiell commented Jan 4, 2024 •

edited

Loading