Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
1677a64
handle sparse x and intercept in _RidgeGCV with gcv_mode='eigen'
jeromedockes Mar 1, 2019
0c36259
start svd
jeromedockes Mar 8, 2019
b423ea9
iter
jeromedockes Mar 15, 2019
f63516e
iter
jeromedockes Mar 15, 2019
00e6f88
remove warning
jeromedockes Mar 15, 2019
ba8bf8a
iter
jeromedockes Mar 15, 2019
1ef72e7
details
jeromedockes Mar 18, 2019
557261d
only keep first min(n,p) singular values
jeromedockes Mar 18, 2019
027e841
comment
jeromedockes Mar 18, 2019
3bd936a
remove _WIPRidgeCV class
jeromedockes Apr 7, 2019
ea3971c
pep8
jeromedockes Apr 7, 2019
074d139
_check_gcv_mode util
jeromedockes Apr 10, 2019
fbdda41
fall back to 10-fold when sample weights, gcv and sparse design
jeromedockes Apr 13, 2019
a2ffa1c
add test to compare results of GCV with gridsearch leave-one-out
jeromedockes Apr 22, 2019
5ae7c75
test with and without normalize
jeromedockes Apr 22, 2019
784c6f4
Merge branch 'master' into ridge_gcv_sparse_x_intercept
jeromedockes Apr 22, 2019
9facc71
add sparse svd to _test_ridge_loo
jeromedockes Apr 22, 2019
3309421
remove test_ridge_cv_sparse_svd
jeromedockes Apr 22, 2019
9e97a87
smaller test data for ridgecv to speed up test
jeromedockes Apr 22, 2019
5438002
add uninformative features
jeromedockes Apr 22, 2019
06555a7
add tests with sample weights
jeromedockes Apr 22, 2019
824231d
pep8
jeromedockes Apr 22, 2019
a82c8d6
Merge remote-tracking branch 'upstream/master' into ridge_gcv_sparse_…
jeromedockes Apr 22, 2019
d1688ce
pep8
jeromedockes Apr 22, 2019
4d9c372
test sample weights with bigger alpha
jeromedockes Apr 24, 2019
5b78c3b
clip slice for indexing sparse matrices with scipy 0.17
jeromedockes Apr 24, 2019
3df75d7
ignore gridsearch deprecation warning
jeromedockes Apr 24, 2019
d991a9a
don't center in _pre_compute when X is dense
jeromedockes Apr 29, 2019
d0146b0
update whats_new
jeromedockes Apr 29, 2019
5705cd5
use the square roots of sample weights rather than a column of ones
jeromedockes Apr 29, 2019
464e507
compare to singular vector rather than check singular value
jeromedockes Apr 29, 2019
c5a35ce
add some comments + sample weights in sparse primal
jeromedockes Apr 29, 2019
bcf87e9
duplicated line
jeromedockes Apr 29, 2019
4d58bf9
parametrize test_ridge_gcv_vs_k_fold
jeromedockes Apr 30, 2019
f893008
better helper function names
jeromedockes Apr 30, 2019
d99fe7e
handle sample weights with sparse X with gcv_mode='svd'
jeromedockes Apr 30, 2019
d826d32
handle sparse x and sample weights when gcv_mode='eigen'
jeromedockes Apr 30, 2019
84e3a6c
fix test + test sample weights with sparse x
jeromedockes Apr 30, 2019
7b21b3f
refactor gcv mode selection
jeromedockes Apr 30, 2019
b1abf02
parametrize sample_weights test + test for p > n and fit_intercept=0
jeromedockes Apr 30, 2019
5538d4f
pep8
jeromedockes Apr 30, 2019
2beac46
compare gcv+sample weights with groupkfold
jeromedockes Apr 30, 2019
bad249e
address @glemaitre review
jeromedockes Apr 30, 2019
d8ad9b3
test with small noise
jeromedockes Apr 30, 2019
a5b62c8
address comments by @glemaitre and @thomasjpfan
jeromedockes Apr 30, 2019
c99ff06
fix docstring
jeromedockes Apr 30, 2019
e35c648
comments
jeromedockes Apr 30, 2019
58c0cc3
Apply suggestions from code review
glemaitre May 1, 2019
080ddef
fix variable name
jeromedockes May 1, 2019
5a14527
better names for _RidgeGCV private helper functions
jeromedockes May 1, 2019
f28db91
compare singular vectors to +- sqrt sample weights
jeromedockes May 1, 2019
1267d22
docstring
jeromedockes May 1, 2019
c7a56c6
find singular vect with smallest angle instead of comparing values
jeromedockes May 1, 2019
467b8fa
reduce rtol in test
jeromedockes May 1, 2019
6d6ea91
increase rtol to 5e-2 in test
jeromedockes May 1, 2019
b6eb75b
add test for _check_gcv_mode
jeromedockes May 2, 2019
d4fde08
remove temp variables after fitting ridge gcv
jeromedockes May 2, 2019
8450c6f
more tests
jeromedockes May 2, 2019
1dad573
merge tests for gcv store_cv_values
jeromedockes May 2, 2019
8a5712d
Update sklearn/linear_model/tests/test_ridge.py
ogrisel May 2, 2019
bb25458
pep8
jeromedockes May 2, 2019
c4303a0
Update sklearn/linear_model/ridge.py
glemaitre May 2, 2019
b964db0
Apply suggestions from code review
glemaitre May 2, 2019
49fb140
docstring
jeromedockes May 2, 2019
e1b5d4e
doc
jeromedockes May 2, 2019
afee463
_errors_and_values docstrings
jeromedockes May 2, 2019
a0e5284
note on use of sample weights for computing the cv score
jeromedockes May 2, 2019
dc7bec1
parametrize check_gcv_mode test as suggested by @glemaitre
jeromedockes May 2, 2019
73404a9
don't store temp variables computed in ridgegcv fit in private attrib…
jeromedockes May 2, 2019
d7475ca
don't store self._X_offset
jeromedockes May 2, 2019
6beb21a
improve test_check_gcv_mode_error
jeromedockes May 2, 2019
4971b5d
check_scoring earlier in _RidgeGCV fit
jeromedockes May 2, 2019
b0ddc8d
fix case where y.shape == (n_samples, 1)
jeromedockes May 2, 2019
8c48278
more y shapes with sample weights
jeromedockes May 2, 2019
b0a82fd
reduce number of combinations in ridgegcv tests + smaller test data
jeromedockes May 2, 2019
659cb14
pep8
jeromedockes May 2, 2019
44616c8
better parameter name
jeromedockes May 2, 2019
bdd1f8e
changes suggested by @ogrisel
jeromedockes May 2, 2019
3a4e0c7
kernel_size -> nullspace_dim
jeromedockes May 2, 2019
05c56f9
specify why we ignore DeprecationWarning in ridgecv test
jeromedockes May 2, 2019
9a05f60
constant offset in test and x -> X
jeromedockes May 2, 2019
9e67824
increase rtol to 1e-3
jeromedockes May 2, 2019
70a8135
add constant offset to y
jeromedockes May 2, 2019
161ca89
more doc on gcv
jeromedockes May 2, 2019
36d248d
doctest skip
jeromedockes May 2, 2019
c439ccb
perf: use LinearOperator transpose rather than _rmatvec
jeromedockes May 3, 2019
bdea422
don't skip ridge doctest in linear_model.rst
jeromedockes May 3, 2019
0e29d14
docstring
jeromedockes May 3, 2019
3a77e05
improve RidgeCV narrative doc
jeromedockes May 3, 2019
bc3ff7e
Update sklearn/linear_model/tests/test_ridge.py
ogrisel May 3, 2019
34d2c2b
reduce tolerance to 1e-3 in test_ridge_gcv_sample_weights
jeromedockes May 3, 2019
21cf6bf
duplicated line
jeromedockes May 3, 2019
4d4f819
better explaination of column capturing intercept
jeromedockes May 3, 2019
1b8d288
errors_and_values -> solve
jeromedockes May 3, 2019
80244ba
make test data design matrix actually sparse in ridgegcv tests
jeromedockes May 4, 2019
32b6a63
remove _errors_and_values
jeromedockes May 4, 2019
32b5345
pep8
jeromedockes May 4, 2019
afe6ec8
larger noise in tests
jeromedockes May 4, 2019
692e6f0
rename 'out' either squared_errors or predictions
jeromedockes May 4, 2019
6b30b53
Merge remote-tracking branch 'upstream/master' into ridge_gcv_sparse_…
jeromedockes May 5, 2019
3e6e794
address comments by @thomasjpfan
jeromedockes May 6, 2019
6e5ea75
Merge remote-tracking branch 'upstream/master' into ridge_gcv_sparse_…
jeromedockes May 6, 2019
52ef1e9
simplify _compute_gram
jeromedockes May 6, 2019
e38aa8d
add tests for _compute_gram and _compute_covariance
jeromedockes May 6, 2019
aec69bd
assert np.allclose -> assert_allclose
jeromedockes May 6, 2019
26b0efc
keep X in float64
jeromedockes May 7, 2019
b0f7f4a
more explicit decompose / solve private function names
jeromedockes May 7, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions doc/modules/linear_model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -136,17 +136,24 @@ Setting the regularization parameter: generalized Cross-Validation
------------------------------------------------------------------

:class:`RidgeCV` implements ridge regression with built-in
cross-validation of the alpha parameter. The object works in the same way
cross-validation of the alpha parameter. The object works in the same way
as GridSearchCV except that it defaults to Generalized Cross-Validation
(GCV), an efficient form of leave-one-out cross-validation::

>>> import numpy as np
>>> from sklearn import linear_model
>>> reg = linear_model.RidgeCV(alphas=[0.1, 1.0, 10.0], cv=3)
>>> reg.fit([[0, 0], [0, 0], [1, 1]], [0, .1, 1]) # doctest: +SKIP
RidgeCV(alphas=[0.1, 1.0, 10.0], cv=3, fit_intercept=True, scoring=None,
normalize=False)
>>> reg.alpha_ # doctest: +SKIP
0.1
>>> reg = linear_model.RidgeCV(alphas=np.logspace(-6, 6, 13))
>>> reg.fit([[0, 0], [0, 0], [1, 1]], [0, .1, 1]) # doctest: +NORMALIZE_WHITESPACE
RidgeCV(alphas=array([1.e-06, 1.e-05, 1.e-04, 1.e-03, 1.e-02, 1.e-01, 1.e+00, 1.e+01,
1.e+02, 1.e+03, 1.e+04, 1.e+05, 1.e+06]),
cv=None, fit_intercept=True, gcv_mode=None, normalize=False,
scoring=None, store_cv_values=False)
>>> reg.alpha_
0.01

Specifying the value of the `cv` attribute will trigger the use of
cross-validation with `GridSearchCV`, for example `cv=10` for 10-fold
cross-validation, rather than Generalized Cross-Validation.

.. topic:: References

Expand Down
8 changes: 7 additions & 1 deletion doc/whats_new/v0.21.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ random sampling procedures.
seed, including :class:`linear_model.LogisticRegression`,
:class:`linear_model.LogisticRegressionCV`, :class:`linear_model.Ridge`,
and :class:`linear_model.RidgeCV` with 'sag' solver. |Fix|
- :class:`linear_model.ridge.RidgeCV` when using generalized cross-validation
with sparse inputs. |Fix|


Details are listed in the changelog below.
Expand Down Expand Up @@ -384,7 +386,7 @@ Support for Python 3.4 and below has been officially dropped.
:mod:`sklearn.linear_model`
...........................

- |Enhancement| :mod:`linear_model.ridge` now preserves ``float32`` and
- |Enhancement| :class:`linear_model.Ridge` now preserves ``float32`` and
``float64`` dtypes. :issues:`8769` and :issues:`11000` by
:user:`Guillaume Lemaitre <glemaitre>`, and :user:`Joan Massich <massich>`

Expand Down Expand Up @@ -482,6 +484,10 @@ Support for Python 3.4 and below has been officially dropped.
in version 0.21 and will be removed in version 0.23.
:pr:`12821` by :user:`Nicolas Hug <NicolasHug>`.

- |Fix| :class:`linear_model.ridge.RidgeCV` with generalized cross-validation
now correctly fits an intercept when ``fit_intercept=True`` and the design
matrix is sparse. :issue:`13350` by :user:`Jérôme Dockès <jeromedockes>`

:mod:`sklearn.manifold`
.......................

Expand Down
Loading