Open
Description
Suggestion
#8602 (integration tests failing on the main
branch) happened because integration tests install the latest version of upstream dependencies. The benefit of using latest versions is that we see failures as soon as possible. But the downside is that main
can have failing tests unexpectedly.
I think we want to make sure the main
branch is broken as little as possible. A couple strategies come to mind:
- Having the integration tests file a new issue when a new failure is detected: so we get notified via an issue of failures on
main
- Using pinned dependencies for the integration tests, and use Renovate/similar to auto-update them: so failures don't seep into
main
I'd prefer the latter, so that main
doesn't break. Thoughts?