Skip to content

feat(rule-tester): allow to create empty tests #7467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Sep 3, 2023
Merged

feat(rule-tester): allow to create empty tests #7467

merged 3 commits into from
Sep 3, 2023

Conversation

azat-io
Copy link
Contributor

@azat-io azat-io commented Aug 13, 2023 •
edited by JoshuaKGoldberg
Loading

PR Checklist

Overview

In some cases I only need to test for valid or invalid scenarios.

For example, if I want to check that a rule for the ESLint plugin works with different code stylings. Or that a rule does not apply to a certain file extension.

Example:

ruleTester.run(
  `${RULE_NAME}: works consistently with an empty array or an array with one element`,
  rule,
  {
    valid: ['[].includes(person)', "['Decim'].includes(bartender)"],
    invalid: [],
  },
)

This test is currently crashing with an error: Error: No test found in suite invalid

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @azat-io!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!

🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

@netlify
Copy link

netlify bot commented Aug 13, 2023 •
edited
Loading

✅ Deploy Preview for typescript-eslint ready!

Name Link
🔨 Latest commit 50adaa0
🔍 Latest deploy log https://app.netlify.com/sites/typescript-eslint/deploys/64f5038e23137900077a6b02
😎 Deploy Preview https://deploy-preview-7467--typescript-eslint.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@nx-cloud
Copy link

nx-cloud bot commented Aug 13, 2023 •
edited
Loading

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 50adaa0. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

✅ Successfully ran 40 targets

Sent with 💌 from NxCloud.

This was referenced Aug 13, 2023
Merged
Merged
JoshuaKGoldberg
JoshuaKGoldberg requested changes Aug 15, 2023
View reviewed changes
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code changes look nifty but let's discuss in a backing issue. The rule tester is an important part of the public API and I'm not sure we want to do this without talking it over first.

@JoshuaKGoldberg JoshuaKGoldberg added the please fill out the template we have the processes for good reasons 😔 label Aug 15, 2023
@azat-io
Copy link
Contributor Author

azat-io commented Aug 15, 2023

I would like to be able to skip valid or invalid scenarios in some tests. In some cases, such examples are simply not needed.

Some simple example:

import noLetRule from '../rule/no-let'

ruleTester.run(
  `disallow to use let`,
  rule,
  {
    valid: [],
    invalid: [`let a = 1`],
  },
)

@azat-io
Copy link
Contributor Author

azat-io commented Aug 15, 2023

Issue #7481

@JoshuaKGoldberg JoshuaKGoldberg added triage Waiting for team members to take a look and removed please fill out the template we have the processes for good reasons 😔 labels Aug 28, 2023
@azat-io
Copy link
Contributor Author

azat-io commented Sep 3, 2023 •
edited
Loading

@JoshuaKGoldberg Could you review this PR?

This pull request is the last step to upgrade to version 6 before the big release in our ESLint plugin: azat-io/eslint-plugin-perfectionist#32

The same PR in ESLint also was merged: eslint/eslint#17475

@JoshuaKGoldberg JoshuaKGoldberg self-requested a review September 3, 2023 17:57
JoshuaKGoldberg
JoshuaKGoldberg requested changes Sep 3, 2023
View reviewed changes
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good, thanks for keeping on this!

Could you please add some unit tests for the new logic though? Conditionally skipping tests is a kind of behavior that's easy to trip up over time.

@JoshuaKGoldberg JoshuaKGoldberg added awaiting response Issues waiting for a reply from the OP or another party and removed triage Waiting for team members to take a look labels Sep 3, 2023
@azat-io
Copy link
Contributor Author

azat-io commented Sep 3, 2023 •
edited
Loading

@JoshuaKGoldberg Hmm. Can you tell me the best way to do that? Since this problem occurs only when using RuleTester with Vitest. Because of this: https://github.com/vitest-dev/vitest/blob/main/packages/runner/src/run.ts#L331-L338

@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Sep 3, 2023 •
edited
Loading

See the tests at the bottom of RuleTester.test.ts in the constructor constraints block for reference:

typescript-eslint/packages/rule-tester/tests/RuleTester.test.ts

Line 762 in 6d3d162

describe('constructor constraints', () => {

They show running ruleTester.run('my-rule", ...) and then asserting on the mockedDescribe.mock. You could add a similar block with tests roughly like:

it('does not call describe with valid if no valid tests are provided', () => {
  const ruleTester = new RuleTester();

  ruleTester.run('my-rule', NOOP_RULE, {
    valid: [],
    invalid: [
      {
        code: 'invalid',
        errors: [{ messageId: 'error' }],
      },
    ],
  });

  expect(mockedDescribe.mock.calls) // ...

@azat-io
Copy link
Contributor Author

azat-io commented Sep 3, 2023

@JoshuaKGoldberg Thanks. Done.

JoshuaKGoldberg
JoshuaKGoldberg approved these changes Sep 3, 2023
View reviewed changes
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! I actually stumbled onto a want for this recently 😄 so it's very exciting to see it come in. Thanks as always @azat-io!

@JoshuaKGoldberg JoshuaKGoldberg removed the awaiting response Issues waiting for a reply from the OP or another party label Sep 3, 2023
@JoshuaKGoldberg JoshuaKGoldberg merged commit 654b35d into typescript-eslint:main Sep 3, 2023
@azat-io azat-io deleted the feat/allow-empty-tests branch September 3, 2023 22:34
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@JoshuaKGoldberg JoshuaKGoldberg JoshuaKGoldberg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Enhancement: Add the ability to skip valid or invalid tests
2 participants
@azat-io @JoshuaKGoldberg