Skip to content

[MRG+1] more explicit error message when multiple scores passed #11008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 22, 2018
Merged

[MRG+1] more explicit error message when multiple scores passed #11008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d20cb6d
more explicit error message when mulitple scores passed to cross_val_…
minggli Apr 22, 2018
91f19da
update error message in test_search.py
minggli Apr 22, 2018
015557d
use collections.Iterable since string type already checked.
minggli Apr 22, 2018
d311945
whatsnew entry
minggli Apr 22, 2018
8118baf
moved to bugfix section
minggli Apr 22, 2018
deb5925
add subsection
minggli Apr 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions doc/whats_new/v0.20.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,11 +91,11 @@ Preprocessing

- Added :class:`MICEImputer`, which is a strategy for imputing missing
values by modeling each feature with missing values as a function of
other features in a round-robin fashion. :issue:`8478` by
other features in a round-robin fashion. :issue:`8478` by
:user:`Sergey Feldman <sergeyf>`.

- Updated :class:`preprocessing.MinMaxScaler` to pass through NaN values. :issue:`10404`
by :user:`Lucija Gregov <LucijaGregov>`.
- Updated :class:`preprocessing.MinMaxScaler` to pass through NaN values. :issue:`10404`
by :user:`Lucija Gregov <LucijaGregov>`.

Model evaluation

Expand Down Expand Up @@ -414,6 +414,12 @@ Preprocessing
``inverse_transform`` on unseen labels. :issue:`9816` by :user:`Charlie Newey
<newey01c>`.

Model evaluation and meta-estimators

- Add improved error message in :func:`model_selection.cross_val_score` when
multiple metrics are passed in ``scoring`` keyword.
:issue:`11006` by :user:`Ming Li <minggli>`.

Datasets

- Fixed a bug in :func:`dataset.load_boston` which had a wrong data point.
Expand Down
5 changes: 5 additions & 0 deletions sklearn/metrics/scorer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
# License: Simplified BSD

from abc import ABCMeta, abstractmethod
from collections import Iterable
import warnings

import numpy as np
Expand Down Expand Up @@ -300,6 +301,10 @@ def check_scoring(estimator, scoring=None, allow_none=False):
"If no scoring is specified, the estimator passed should "
"have a 'score' method. The estimator %r does not."
% estimator)
elif isinstance(scoring, Iterable):
raise ValueError("For evaluating multiple scores, use "
"sklearn.model_selection.cross_validate instead. "
"{0} was passed.".format(scoring))
else:
raise ValueError("scoring value should either be a callable, string or"
" None. %r was passed" % scoring)
Expand Down
8 changes: 4 additions & 4 deletions sklearn/model_selection/tests/test_search.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1217,10 +1217,10 @@ def test_fit_grid_point():
assert_equal(n_test_samples, test.size)

# Should raise an error upon multimetric scorer
assert_raise_message(ValueError, "scoring value should either be a "
"callable, string or None.", fit_grid_point, X, y,
svc, params, train, test, {'score': scorer},
verbose=True)
assert_raise_message(ValueError, "For evaluating multiple scores, use "
"sklearn.model_selection.cross_validate instead.",
fit_grid_point, X, y, svc, params, train, test,
{'score': scorer}, verbose=True)


def test_pickle():
Expand Down