Skip to content

How will we handle tests requiring special environment #7819

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zhw2101024 opened this issue Apr 22, 2025 · 4 comments
Closed

How will we handle tests requiring special environment #7819

zhw2101024 opened this issue Apr 22, 2025 · 4 comments

Comments

@zhw2101024
Copy link
Contributor

There's some tests requiring special system environment, such as one user with digital user, or use one nonexistent userid as parameter.
How will we handle this, should i remove the tests in these PRs? #7788 #7789

@jtracey
Copy link
Contributor

jtracey commented Apr 23, 2025

If the "special environment" is one that can be locally controlled, this can be done in the test. For something like adding a user, this is too intrusive to do on developers' machines. This has been handled in the past by writing tests that should pass in both the normal and special environment, then modifying the environment of one or more of the CICD runners. See, e.g., this additional setup for pinky (if these steps get too complicated, it's better to put them in their own script, like the script to set up an android emulator). That way, even if the test passes in the dev's normal environment, it will still be caught when the PR is filed.

@zhw2101024
Copy link
Contributor Author

If the "special environment" is one that can be locally controlled, this can be done in the test. For something like adding a user, this is too intrusive to do on developers' machines. This has been handled in the past by writing tests that should pass in both the normal and special environment, then modifying the environment of one or more of the CICD runners. See, e.g., this additional setup for pinky (if these steps get too complicated, it's better to put them in their own script, like the script to set up an android emulator). That way, even if the test passes in the dev's normal environment, it will still be caught when the PR is filed.

Thanks, the pinky example is very useful for testing the PR #7788, however the test in PR #7789 need to ensure that a specified userid is not in the system, what should we do if the userid is on the CICD runners?

@jtracey
Copy link
Contributor

jtracey commented Apr 29, 2025

I suspect the runner environments are pretty stable, so if one of them is missing userid 2, just adding a script to fail loudly if that changes would be sufficient; but I suspect that's unlikely, since we presumably would've caught the problem in that case. If you're confident you can safely modify the system to change userid 2 to something else, you could do so in the freebsd VM.

@jtracey
Copy link
Contributor

jtracey commented Apr 29, 2025

Ah, never mind, just read the test in question, I don't think you need a special environment at all (continuing discussion there--if there's nothing actionable here, I think we can close this issue).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants