Skip to content

FIX mcc zero divsion #19977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 3, 2021
Merged

Conversation

Connerrrrr
Copy link
Contributor

@Connerrrrr Connerrrrr commented Apr 25, 2021

Reference Issues/PRs

Fixes #16924

What does this implement/fix? Explain your changes.

The original MCC function included the case of confusion matrix has all zeros but for one column. However, the original solution did not expect the warning message would pop up, even the return value is correct.

According to Jurman, Riccadonna, Furlanello, (2012). A Comparison of MCC and CEN Error Measures in MultiClass Prediction

> MCC is equal to 0 when C is all zeros but for one column (all samples have been classified to be of a class k)

and Wikipedia of Matthews Correlation Coefficient,

> If any of the four sums in the denominator is zero, the denominator can be arbitrarily set to one

the fix is checking whenever there is only a column or a row in the confusion matrix that is non-zero, the MCC automatically set to 0.

Copy link
Member

@jeremiedbb jeremiedbb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR @Connerrrrr. Here are a few comments.

(also synchronizing with main will fix the failing CI job)

@Connerrrrr
Copy link
Contributor Author

@jeremiedbb Thanks for the comment! They are all resolved now.

Copy link
Member

@jeremiedbb jeremiedbb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @Connerrrrr !

Copy link
Member

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well. Thanks for the fix!

@ogrisel ogrisel merged commit 64bafa3 into scikit-learn:main Jun 3, 2021
thomasjpfan added a commit to thomasjpfan/scikit-learn that referenced this pull request Jun 8, 2021
* TST enable test docstring params for feature extraction module (scikit-learn#20188)

* DOC fix a reference in sklearn.ensemble.GradientBoostingRegressor (scikit-learn#20198)

* FIX mcc zero divsion  (scikit-learn#19977)

* TST Add TransformedTargetRegressor to test_meta_estimators_delegate_data_validation (scikit-learn#20175)

Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>

* TST enable n_feature_in_ test for feature_extraction module

* FIX Uses points instead of pixels in plot_tree (scikit-learn#20023)

* MNT n_features_in through the multiclass module (scikit-learn#20193)

* CI Removes python 3.6 builds from wheel building (scikit-learn#20184)

* FIX Fix typo in error message in `fetch_openml` (scikit-learn#20201)

* FIX Fix error when using Calibrated with Voting (scikit-learn#20087)

* FIX Fix RandomForestRegressor doesn't accept max_samples=1.0 (scikit-learn#20159)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* ENH Adds Poisson criterion in RandomForestRegressor (scikit-learn#19836)

Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>
Co-authored-by: Alihan Zihna <alihanz@gmail.com>
Co-authored-by: Alihan Zihna <a.zihna@ckhgbdp.onmicrosoft.com>
Co-authored-by: Chiara Marmo <cmarmo@users.noreply.github.com>
Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com>
Co-authored-by: naozin555 <37050583+naozin555@users.noreply.github.com>
Co-authored-by: Venkatachalam N <venky.yuvy@gmail.com>
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>

* TST Replace assert_warns from decomposition/tests (scikit-learn#20214)

* TST check n_features_in_ in pipeline module (scikit-learn#20192)

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com>

* Allow `n_knots=None` if knots are explicitly specified in `SplineTransformer` (scikit-learn#20191)


Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

* FIX make check_complex_data deterministic (scikit-learn#20221)

* TST test_fit_docstring_attributes include properties (scikit-learn#20190)

* FIX Uses the color max for colormap in ConfusionMatrixDisplay (scikit-learn#19784)

* STY Changing .format method to f-string formatting (scikit-learn#20215)

* CI Adds permissions for label action

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Co-authored-by: tsuga <2888173+tsuga@users.noreply.github.com>
Co-authored-by: Conner Shen <connershen98@hotmail.com>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Co-authored-by: mlondschien <61679398+mlondschien@users.noreply.github.com>
Co-authored-by: Clément Fauchereau <clement.fauchereau@ensta-bretagne.org>
Co-authored-by: murata-yu <67666318+murata-yu@users.noreply.github.com>
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
Co-authored-by: Brian Sun <52805678+bsun94@users.noreply.github.com>
Co-authored-by: Christian Lorentzen <lorentzen.ch@gmail.com>
Co-authored-by: Alihan Zihna <alihanz@gmail.com>
Co-authored-by: Alihan Zihna <a.zihna@ckhgbdp.onmicrosoft.com>
Co-authored-by: Chiara Marmo <cmarmo@users.noreply.github.com>
Co-authored-by: Olivier Grisel <olivier.grisel@gmail.com>
Co-authored-by: naozin555 <37050583+naozin555@users.noreply.github.com>
Co-authored-by: Venkatachalam N <venky.yuvy@gmail.com>
Co-authored-by: Nanshan Li <nanshanli@dsaid.gov.sg>
Co-authored-by: solosilence <abhishekkr23rs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Matthews correlation coefficient metric throws misleading division by zero RuntimeWarning
4 participants