Skip to content

add common test that zero sample weight means samples are ignored #15015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

add common test that zero sample weight means samples are ignored #15015

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
85dc97e
add test that zero sample weight means samples are ignored
amueller Sep 18, 2019
95a2d5b
add predict_proba and decision_function to tests
amueller Sep 18, 2019
bc285fa
Merge remote-tracking branch 'origin/master' into sample_weight_real_…
ogrisel Oct 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion sklearn/utils/estimator_checks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -750,31 +750,51 @@ def check_sample_weights_invariance(name, estimator_orig):

estimator1 = clone(estimator_orig)
estimator2 = clone(estimator_orig)
estimator3 = clone(estimator_orig)
set_random_state(estimator1, random_state=0)
set_random_state(estimator2, random_state=0)
set_random_state(estimator3, random_state=0)

X = np.array([[1, 3], [1, 3], [1, 3], [1, 3],
[2, 1], [2, 1], [2, 1], [2, 1],
[3, 3], [3, 3], [3, 3], [3, 3],
[4, 1], [4, 1], [4, 1], [4, 1]], dtype=np.dtype('float'))
y = np.array([1, 1, 1, 1, 2, 2, 2, 2,
1, 1, 1, 1, 2, 2, 2, 2], dtype=np.dtype('int'))

X2 = np.vstack([X, X])
y2 = np.hstack([y, 3 - y])
y = _enforce_estimator_tags_y(estimator1, y)
y2 = _enforce_estimator_tags_y(estimator3, y2)
weights = np.ones(shape=len(y) * 2)
weights[len(y):] = 0
X2, y2, weights = shuffle(X2, y2, weights, random_state=0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shuffling the data might have an impact, isn't it.
I was checking and OneClassSVM will give the same result if you are not shuffling the data.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed. Your test is better wrt to that, I think. I didn't want things to be sorted by weight (CV estimators will fail) but I guess I would need to ensure the order of the non-zero-weight samples stays the same (which is I think what you did?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the default tol is too high to compare the decision function. I just tried with SVC with a tol=1e-12 then the decision function will be really close.


estimator1.fit(X, y=y, sample_weight=np.ones(shape=len(y)))
estimator2.fit(X, y=y, sample_weight=None)
estimator3.fit(X2, y=y2, sample_weight=weights)

for method in ["predict", "transform"]:
for method in ["predict", "predict_proba",
"decision_function", "transform"]:
if hasattr(estimator_orig, method):
X_pred1 = getattr(estimator1, method)(X)
X_pred2 = getattr(estimator2, method)(X)
X_pred3 = getattr(estimator3, method)(X)
if sparse.issparse(X_pred1):
X_pred1 = X_pred1.toarray()
X_pred2 = X_pred2.toarray()
X_pred3 = X_pred3.toarray()
assert_allclose(X_pred1, X_pred2,
err_msg="For %s sample_weight=None is not"
" equivalent to sample_weight=ones"
% name)
assert_allclose(
X_pred1, X_pred3,
err_msg="For %s sample_weight is not"
" equivalent to removing samples"
% name)




@ignore_warnings(category=(DeprecationWarning, FutureWarning, UserWarning))
Expand Down