Flaky tests in logistic regression #8879
Comments
|
Which tests, precisely?
…On 14 May 2017 at 22:31, Junhao Li ***@***.***> wrote:
@glemaitre <https://github.com/glemaitre> #8872
<#8872>
In current master branch
1de66a0
<1de66a0>
If you run the following commands several times
nosetests sklearn/linear_model/
Some tests related to comparing results between two methods fail randomly.
Instead of comparing with each other, maybe we can use the following test
case:
x1 x2 y
1 2 0
2 3 0
3 5 1
4 7 0
5 11 1
6 13 1
For this case, I checked in two statistical tools, a professional one, R,
and a casual one, google docs, that the result agrees with each other to
the digits shown below:
,Coefficients,Standard Error
Intercept,-2.942920409,3.47036
X Variable 1,-0.9491680059,3.97308
X Variable 2,0.9755542466 <(975)%20554-2466>,1.89720
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#8879>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AAEz65yFgtf0KySEQ-SIQdhHQCAxP-ryks5r5vQSgaJpZM4NaXSC>
.
|
|
in the file test_logistic, the tests that compares the coef_ from two methods, like saga vs liblinear |
|
And do you have a hint to achieve the failing condition. I tried to run 10 times but I did not succeed. Also, if you can provide you system info: import platform; print(platform.platform())
import sys; print("Python", sys.version)
import numpy; print("NumPy", numpy.__version__)
import scipy; print("SciPy", scipy.__version__)
import sklearn; print("Scikit-Learn", sklearn.__version__) |
|
I see. It could be a dependency issue. Darwin-16.5.0-x86_64-i386-64bit |
|
I just updated some packages and still got fails. Darwin-16.5.0-x86_64-i386-64bit |
|
@jl2922 Confirmed that the same issue occurred on my side as well. test_saga_vs_liblinear fails ~70% of the time. Below are my system info: Darwin-16.5.0-x86_64-i386-64bit I have also extracted out a failed case (data: failed_X and failed_y, fitted model: failed_saga and failed_liblinear) which may help to reproduce the error. I am still looking at it, but at the first glimpse it seems a bit peculiar for saga solver to have coefficients as huge as -5.623e+177. |
|
My neighbor with the following config reproduce the same error. @jnothman @ogrisel @lesteve it seems that there is something specific with Darwin and Anaconda |
|
Maybe some floating point difference? Maybe dependent on the linear algebra library that is being used? @glemaitre can you kindly ask your neighbor to post the nosetests or pytest command to run one of the tests failing + the stacktrace? |
|
@dengemann (he might to be away this week)
I asked for |
|
I'm back tomorrow. I can share the stack traces with you later.
…On Tue, 23 May 2017 at 15:46, Guillaume Lemaitre ***@***.***> wrote:
@dengemann <https://github.com/dengemann> (he might to be away this week)
nosetests or pytest command to run one
I asked for nosetests sklearn/linear_model/tests/test_logistic.py.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8879 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AB0fiobwO88F_5KGvJAsEAxtrP2Ro2zmks5r8uM1gaJpZM4NaXSC>
.
|
|
I just had this happen to me as well. |
@glemaitre #8872
In current master branch
If you run the following commands several times
Some tests related to comparing results between two methods fail randomly.
Instead of comparing with each other, maybe we can use the following test case:
x1 x2 y
1 2 0
2 3 0
3 5 1
4 7 0
5 11 1
6 13 1
For this case, I checked in two statistical tools, a professional one, R, and a casual one, google docs, that the result agrees with each other to the digits shown below:
The text was updated successfully, but these errors were encountered: