FIX Fix error when using Calibrated with Voting #20087
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bc43de1 to
502b8a7
Compare
thomasjpfan
reviewed
May 17, 2021
sklearn/calibration.py
Outdated
Comment on lines
511
to
512
| elif method_name == 'predict_proba' or method_name == '_predict_proba': | ||
| # The `_predict_proba` option is needed for `VotingClassifier` |
There was a problem hiding this comment.
I do not think special casing VotingClassifier is great. I think it would be better to get _get_prediction_method to return a tuple (callable, method_name) and then use the method_name here.
(_get_prediction_method is always called before _compute_predictions)
There was a problem hiding this comment.
I agree. I have updated the code.
502b8a7 to
34322ee
Compare
ogrisel
approved these changes
Jun 1, 2021
34322ee to
8eef25f
Compare
thomasjpfan
added a commit
to thomasjpfan/scikit-learn
that referenced
this pull request
Jun 8, 2021
* TST enable test docstring params for feature extraction module (scikit-learn#20188) * DOC fix a reference in sklearn.ensemble.GradientBoostingRegressor (scikit-learn#20198) * FIX mcc zero divsion (scikit-learn#19977) * TST Add TransformedTargetRegressor to test_meta_estimators_delegate_data_validation (scikit-learn#20175) Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com> * TST enable n_feature_in_ test for feature_extraction module * FIX Uses points instead of pixels in plot_tree (scikit-learn#20023) * MNT n_features_in through the multiclass module (scikit-learn#20193) * CI Removes python 3.6 builds from wheel building (scikit-learn#20184) * FIX Fix typo in error message in `fetch_openml` (scikit-learn#20201) * FIX Fix error when using Calibrated with Voting (scikit-learn#20087) * FIX Fix RandomForestRegressor doesn't accept max_samples=1.0 (scikit-learn#20159) Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com> * ENH Adds Poisson criterion in RandomForestRegressor (scikit-learn#19836) Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com> Co-authored-by: Alihan Zihna <alihanz@gmail.com> Co-authored-by: Alihan Zihna <a.zihna@ckhgbdp.onmicrosoft.com> Co-authored-by: Chiara Marmo <cmarmo@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com> Co-authored-by: naozin555 <37050583+naozin555@users.noreply.github.com> Co-authored-by: Venkatachalam N <venky.yuvy@gmail.com> Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com> * TST Replace assert_warns from decomposition/tests (scikit-learn#20214) * TST check n_features_in_ in pipeline module (scikit-learn#20192) Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com> * Allow `n_knots=None` if knots are explicitly specified in `SplineTransformer` (scikit-learn#20191) Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> * FIX make check_complex_data deterministic (scikit-learn#20221) * TST test_fit_docstring_attributes include properties (scikit-learn#20190) * FIX Uses the color max for colormap in ConfusionMatrixDisplay (scikit-learn#19784) * STY Changing .format method to f-string formatting (scikit-learn#20215) * CI Adds permissions for label action Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com> Co-authored-by: tsuga <2888173+tsuga@users.noreply.github.com> Co-authored-by: Conner Shen <connershen98@hotmail.com> Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com> Co-authored-by: mlondschien <61679398+mlondschien@users.noreply.github.com> Co-authored-by: Clément Fauchereau <clement.fauchereau@ensta-bretagne.org> Co-authored-by: murata-yu <67666318+murata-yu@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> Co-authored-by: Brian Sun <52805678+bsun94@users.noreply.github.com> Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com> Co-authored-by: Alihan Zihna <alihanz@gmail.com> Co-authored-by: Alihan Zihna <a.zihna@ckhgbdp.onmicrosoft.com> Co-authored-by: Chiara Marmo <cmarmo@users.noreply.github.com> Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com> Co-authored-by: naozin555 <37050583+naozin555@users.noreply.github.com> Co-authored-by: Venkatachalam N <venky.yuvy@gmail.com> Co-authored-by: Nanshan Li <nanshanli@dsaid.gov.sg> Co-authored-by: solosilence <abhishekkr23rs@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Fixes #20053
What does this implement/fix? Explain your changes.
The commit #17856 changed the way
CalibratedClassifierCVinternally works. Due to weird implementation ofVotingClassifier.predict_proba, it broke the compatibility whenVotingClassifieris used asbase_estimatorinCalibratedClassifier.This is a simple fix to restore compatibility.
Any other comments?
The real issue is the way
VotingClassifier.predict_probais implemented. However, it seems to me that it can't be resolved without breaking changes.The issue is that
predict_probais not a method but an attribute which holds another method. The goal of this trick was to do polymorphism and implementpredict_probaonly whenvoting="soft"using a getter.I think
VotingClassifiershould be an abstract class and we should implementSoftVotingClassifierandHardVotingClassifiersince they don't implement the same methods. It is however a big API change.It would be simpler to only raise an error if
voting="hard"but if I understand correctly it is assumed that ifpredict_probaexists then it must work. It would therefore leads to other incompatibilities.I am not familiar enough with the code base to know how it is usually dealt with. Anyway, I think it needs further discussion.