Skip to content

Repo: Lint rule to ban dynamic syntax in generating tests #11052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
JoshuaKGoldberg opened this issue Apr 9, 2025 · 2 comments
Open

Repo: Lint rule to ban dynamic syntax in generating tests #11052

JoshuaKGoldberg opened this issue Apr 9, 2025 · 2 comments
Labels
accepting prs Go ahead, send a pull request that resolves this issue repo maintenance things to do with maintenance of the repo, and not with code/docs

Comments

@JoshuaKGoldberg
Copy link
Member

Suggestion

Following #9691 and its ~dozen linked test cleanup issues, we'd like to statically enforce that folks don't add more auto-generated / dynamic tests. I bet this could be done with a lint rule to ban:

  • Putting function calls or other dynamic syntax inside RuleTester tests
  • Placing RuleTester tests anywhere other than the root of a file (except maybe ifs, for environment/version checks)

Additional Info

I could see this eventually being upstreamed into eslint-plugin-eslint-plugin...

💖
@JoshuaKGoldberg JoshuaKGoldberg added repo maintenance things to do with maintenance of the repo, and not with code/docs triage Waiting for team members to take a look labels Apr 9, 2025
@JoshuaKGoldberg JoshuaKGoldberg added accepting prs Go ahead, send a pull request that resolves this issue and removed triage Waiting for team members to take a look labels May 12, 2025
@Vinccool96
Copy link
Contributor

Could you be more precise with what you define as "dynamic syntax"? Do you mean like the spread syntax?

@JoshuaKGoldberg
Copy link
Member Author

JoshuaKGoldberg commented Jun 9, 2025 •
edited
Loading

Good question. That'd be anything besides direct objects/strings in test cases. no-for-in-array.test.ts is an example of an existing OK test file. All its test cases are "POJOs" (Plain Old Java(Script) Objects).

The intent of this issue is that test cases should all be directly written inline. "Dynamically" generating test cases with spreads, function calls, etc. was used in the past to make it easier to write many of them. But we've seen some downsides with that approach:

  • It becomes a lot harder to tweak individual test cases
  • It encourages writing "catch-all" unit tests -- instead of having each test check for one thing
  • The tests are really hard to statically analyze with test-specific lint rules
  • It's really easy to accidentally make an exponential explosion of number of test cases

consistent-type-assertions.test.ts is an example of a test case with a lot of dynamic generation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
accepting prs Go ahead, send a pull request that resolves this issue repo maintenance things to do with maintenance of the repo, and not with code/docs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JoshuaKGoldberg @Vinccool96