Common check for sample weight invariance with removed samples #17176
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t-learn#16507) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
|
cc @NicolasHug . We need to first merge #16963 I think. Earlier version of this PR basically implemented a simpler version of that PR. |
|
I guess we need to start with #17134 then :p |
|
@NicolasHug This PR should be good now. The coverage failure doesn't look relevant. |
NicolasHug
approved these changes
May 15, 2020
sklearn/utils/estimator_checks.py
Outdated
| @@ -845,7 +846,7 @@ def check_sample_weights_shape(name, estimator_orig): | |||
|
|
|||
|
|
|||
| @ignore_warnings(category=FutureWarning) | |||
| def check_sample_weights_invariance(name, estimator_orig): | |||
| def check_sample_weights_invariance(name, estimator_orig, kind="ones"): | |||
| # check that the estimators yield same results for | |||
| # unit weights and no weights | |||
| if (has_fit_parameter(estimator_orig, "sample_weight") and | |||
There was a problem hiding this comment.
while we're at it we might as well return early here instead of having the entire function in a if block?
There was a problem hiding this comment.
Actually moved this _yield_all_checks to only run check_sample_weight on estimators that support it. If find that showing a green status for a check when it doesn't apply is not great (and skipping it also produces too much noise).
sklearn/utils/estimator_checks.py
Outdated
| @@ -861,25 +862,45 @@ def check_sample_weights_invariance(name, estimator_orig): | |||
| X = np.array([[1, 3], [1, 3], [1, 3], [1, 3], | |||
| [2, 1], [2, 1], [2, 1], [2, 1], | |||
| [3, 3], [3, 3], [3, 3], [3, 3], | |||
| [4, 1], [4, 1], [4, 1], [4, 1]], dtype=np.dtype('float')) | |||
| [4, 1], [4, 1], [4, 1], [4, 1]], dtype=np.float64) | |||
There was a problem hiding this comment.
for consistency these could be X1 and y1
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
thomasjpfan
approved these changes
May 16, 2020
sklearn/utils/estimator_checks.py
Outdated
| if sparse.issparse(X_pred1): | ||
| X_pred1 = X_pred1.toarray() | ||
| X_pred2 = X_pred2.toarray() | ||
| assert_allclose(X_pred1, X_pred2, err_msg=err_msg) |
There was a problem hiding this comment.
We can use sklearn.utils._testing.assert_allclose_dense_sparse here.
viclafargue
pushed a commit
to viclafargue/scikit-learn
that referenced
this pull request
Jun 26, 2020
…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Jul 5, 2020
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Jul 5, 2020
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Aug 5, 2020
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Oct 15, 2020
…ple is never a support vector
jayzed82
pushed a commit
to jayzed82/scikit-learn
that referenced
this pull request
Oct 22, 2020
…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Nov 28, 2020
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Dec 12, 2020
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Feb 25, 2021
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Feb 25, 2021
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Mar 27, 2021
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Jun 15, 2021
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Jul 23, 2021
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Nov 10, 2021
…ple is never a support vector
This was referenced Dec 9, 2021
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
May 15, 2022
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Jun 14, 2022
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Aug 29, 2022
…ple is never a support vector
ivannz
added a commit
to ivannz/scikit-learn
that referenced
this pull request
Sep 5, 2022
…ple is never a support vector
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as #16507 but taking into account latest updates in check_estimator and handling of xfail estimators.
I initially tried to fix this quickly in #17175, but in the end decided to revert the inital commit to fix CI and do a less rushed solution in this PR.