Skip to content

FIX Extract estimator objects before aggregating dict of scores #17745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 27, 2020
Merged

FIX Extract estimator objects before aggregating dict of scores #17745

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2481c11
FIX Extract estimator objects before aggregating dict of scores [doc …
alfaro96 Jun 26, 2020
9c0d38d
Minor changes [doc build]
alfaro96 Jun 26, 2020
9a0c3ac
CLN Apply suggested changes [doc build]
alfaro96 Jun 26, 2020
f7e5dfb
MNT Link PR to the test [doc build]
alfaro96 Jun 26, 2020
615e619
CLN Minor adjustments
thomasjpfan Jun 26, 2020
aee62b0
BLD [doc build]
thomasjpfan Jun 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions sklearn/model_selection/_validation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1539,7 +1539,7 @@ def validation_curve(estimator, X, y, *, param_name, param_range, groups=None,
def _aggregate_score_dicts(scores):
"""Aggregate the list of dict to dict of np ndarray

The aggregated output of _fit_and_score will be a list of dict
The aggregated output of _aggregate_score_dicts will be a list of dict
of form [{'prec': 0.1, 'acc':1.0}, {'prec': 0.1, 'acc':1.0}, ...]
Convert it to a dict of array {'prec': np.array([0.1 ...]), ...}

Expand All @@ -1559,5 +1559,9 @@ def _aggregate_score_dicts(scores):
{'a': array([1, 2, 3, 10]),
'b': array([10, 2, 3, 10])}
"""
return {key: np.asarray([score[key] for score in scores])
for key in scores[0]}
return {
key: np.asarray([score[key] for score in scores])
if isinstance(scores[0][key], numbers.Number)
else [score[key] for score in scores]
for key in scores[0]
}
17 changes: 17 additions & 0 deletions sklearn/model_selection/tests/test_validation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,23 @@ def test_cross_validate_invalid_scoring_param():
cross_validate, SVC(), X, y, scoring="mse")


def test_cross_validate_nested_estimator():
# Non-regression test to ensure that nested
# estimators are properly returned in a list
# https://github.com/scikit-learn/scikit-learn/pull/17745
(X, y) = load_iris(return_X_y=True)
pipeline = Pipeline([
("imputer", SimpleImputer()),
("classifier", MockClassifier()),
])

results = cross_validate(pipeline, X, y, return_estimator=True)
estimators = results["estimator"]

assert isinstance(estimators, list)
assert all(isinstance(estimator, Pipeline) for estimator in estimators)


def test_cross_validate():
# Compute train and test mse/r2 scores
cv = KFold()
Expand Down