Skip to content

test_ridge_regression_vstacked_X is not stable #23014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jeremiedbb opened this issue Apr 1, 2022 · 6 comments · Fixed by #23017 or #23152
Closed

test_ridge_regression_vstacked_X is not stable #23014

jeremiedbb opened this issue Apr 1, 2022 · 6 comments · Fixed by #23017 or #23152
Labels
module:linear_model module:test-suite everything related to our tests

Comments

@jeremiedbb
Copy link
Member

https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=40346&view=logs&jobId=97641769-79fb-5590-9088-a30ce9b850b9&j=97641769-79fb-5590-9088-a30ce9b850b9&t=4745baa1-36b5-56c8-9a8e-6480742db1a6

introduced in #22910.
@lorentzenchr was it tested on "all" random seeds ? (SKLEARN_TESTS_GLOBAL_RANDOM_SEED=all)
@github-actions github-actions bot added the Needs Triage Issue requires triage label Apr 1, 2022
@jeremiedbb jeremiedbb added Build / CI module:linear_model module:test-suite everything related to our tests and removed Build / CI Needs Triage Issue requires triage labels Apr 1, 2022
@lorentzenchr
Copy link
Member

was it tested on "all" random seeds ?

Yes, locally on my machine.

@lorentzenchr
Copy link
Member

SKLEARN_TESTS_GLOBAL_RANDOM_SEED="all" pytest sklearn/linear_model/tests/test_ridge.py now also has one failing test for me. I'll fix it.

@lorentzenchr
Copy link
Member

But for me, it fails in test_ridge_fit_intercept_sparse[94-False-lbfgs], on the Azure CI link, it fails in test_ridge_regression_vstacked_X[wide-5-True-sag].

@lorentzenchr lorentzenchr mentioned this issue Apr 1, 2022
Merged
This was referenced Apr 1, 2022
Merged
Merged
@jeremiedbb
Copy link
Member Author

Really weird, I restarted the run and now it passes. I didn't find any anything non deterministic in test_ridge_regression_vstacked_X. Is it possible that Ridge(solver="sag") is not completely deterministic even on fixed random_state ?

@TomDLT
Copy link
Member

TomDLT commented Apr 4, 2022

I got an error on test_ridge_regression_vstacked_X[long-16-False-sag], second time on test_ridge_regression_vstacked_X[long-63-True-sag], then no error running the tests 5 times, then error on test_ridge_regression_vstacked_X[wide-75-True-sag], ...

Is it possible that Ridge(solver="sag") is not completely deterministic even on fixed random_state ?

It should not be the case, so there seems to be a bug.

@jeremiedbb
Copy link
Member Author

If sag uses BLAS, it's possible that rounding errors accumulate differently because of the multi-threading, (or maybe it's in the construction of the test).

@ogrisel ogrisel mentioned this issue Apr 13, 2022
Merged
@lorentzenchr lorentzenchr linked a pull request Apr 20, 2022 that will close this issue
make test_ridge_regression_vstacked_X less sensitive #23152
Merged
@lorentzenchr lorentzenchr mentioned this issue Apr 20, 2022
Merged
@lorentzenchr lorentzenchr mentioned this issue Apr 21, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module:linear_model module:test-suite everything related to our tests
Projects
None yet
Milestone
No milestone
3 participants
@TomDLT @lorentzenchr @jeremiedbb