Skip to content

[MRG+1] Ovr/OVO classifier decision_function shape fixes #9100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 10, 2017
Merged

[MRG+1] Ovr/OVO classifier decision_function shape fixes #9100

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5ef4b2e
fix OVR classifier edgecase bugs
amueller Jun 10, 2017
3b675d1
add regression tests for OVO and OVR decision function shapes
amueller Jun 10, 2017
346aa4b
add whatsnew entry
amueller Jun 10, 2017
f031817
make test of decision_function conditional on whether there's a decis…
amueller Jun 10, 2017
4998777
gah fix indentation
amueller Jun 10, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/whats_new.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,11 @@ API changes summary
:func:`model_selection.cross_val_predict`.
:issue:`2879` by :user:`Stephen Hoover <stephen-hoover>`.

- The ``decision_function`` output shape for binary classification in
:class:`multi_class.OneVsRestClassifier` and
:class:`multi_class.OneVsOneClassifier` is now ``(n_samples,)`` to conform
to scikit-learn conventions. :issue:`9100` by `Andreas Müller`_.

- Gradient boosting base models are no longer estimators. By `Andreas Müller`_.

- :class:`feature_selection.SelectFromModel` now validates the ``threshold``
Expand Down
7 changes: 6 additions & 1 deletion sklearn/multiclass.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,8 @@ def decision_function(self, X):
T : array-like, shape = [n_samples, n_classes]
"""
check_is_fitted(self, 'estimators_')
if len(self.estimators_) == 1:
return self.estimators_[0].decision_function(X)
return np.array([est.decision_function(X).ravel()
for est in self.estimators_]).T

Expand Down Expand Up @@ -574,6 +576,8 @@ def predict(self, X):
Predicted multi-class targets.
"""
Y = self.decision_function(X)
if self.n_classes_ == 2:
return self.classes_[(Y > 0).astype(np.int)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes that estimators_[0].decision_function correctly returns a vector. Are we relying on check_estimator to validate this, or should we check it here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no we should assume that base_estimator actually works. Otherwise everything in sklearn is broken ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, lgtm then

return self.classes_[Y.argmax(axis=1)]

def decision_function(self, X):
Expand Down Expand Up @@ -606,7 +610,8 @@ def decision_function(self, X):
for est, Xi in zip(self.estimators_, Xs)]).T
Y = _ovr_decision_function(predictions,
confidences, len(self.classes_))

if self.n_classes_ == 2:
return Y[:, 1]
return Y

@property
Expand Down
9 changes: 9 additions & 0 deletions sklearn/tests/test_multiclass.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,9 @@ def conduct_test(base_clf, test_predict_proba=False):
assert_equal(set(clf.classes_), classes)
y_pred = clf.predict(np.array([[0, 0, 4]]))[0]
assert_equal(set(y_pred), set("eggs"))
if hasattr(base_clf, 'decision_function'):
dec = clf.decision_function(X)
assert_equal(dec.shape, (5,))

if test_predict_proba:
X_test = np.array([[0, 0, 4]])
Expand Down Expand Up @@ -524,6 +527,12 @@ def test_ovo_decision_function():
n_samples = iris.data.shape[0]

ovo_clf = OneVsOneClassifier(LinearSVC(random_state=0))
# first binary
ovo_clf.fit(iris.data, iris.target == 0)
decisions = ovo_clf.decision_function(iris.data)
assert_equal(decisions.shape, (n_samples,))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP8 here too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is pep8....


# then multi-class
ovo_clf.fit(iris.data, iris.target)
decisions = ovo_clf.decision_function(iris.data)

Expand Down