FEA confusion matrix derived metrics #17265
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: samskruthi padigepati <https://github.com/ddhar1> Co-authored-by: Divya Dhar<https://github.com/samskruthireddy>
Co-authored-by: samskruthi padigepati <https://github.com/ddhar1> Co-authored-by: Divya Dhar<https://github.com/samskruthireddy>
Co-authored-by: Divya Dhar <https://github.com/ddhar1> Co-authored-by: samskruthi padigepati <https://github.com/samskruthireddy>
Co-authored-by: samskruthi padigepati <https://github.com/samskruthireddy>
Modify doc and add deprecation to position arg.
Add test for binary classification. (Modify some lines to pass flake8)
cmarmo
reviewed
Sep 27, 2020
There was a problem hiding this comment.
Thanks @haochunchang for your pull request. Sorry for the late answer. If you could find sometime to fix conflicts this will be very helpful. Thanks!
sklearn/metrics/_classification.py
Outdated
Comment on lines
1540
to
1605
| labels : list, optional | ||
| The set of labels to include when ``average != 'binary'``, and their | ||
| order if ``average is None``. Labels present in the data can be | ||
| excluded, for example to calculate a multiclass average ignoring a | ||
| majority negative class, while labels not present in the data will | ||
| result in 0 components in a macro average. For multilabel targets, | ||
| labels are column indices. By default, all labels in ``y_true`` and | ||
| ``y_pred`` are used in sorted order. | ||
|
|
||
| pos_label : str or int, 1 by default | ||
| The class to report if ``average='binary'`` and the data is binary. | ||
| If the data are multiclass or multilabel, this will be ignored; | ||
| setting ``labels=[pos_label]`` and ``average != 'binary'`` will report | ||
| scores for that label only. | ||
|
|
||
| average : string, [None (default), 'binary', 'micro', 'macro', 'samples', \ | ||
| 'weighted'] | ||
| If ``None``, the scores for each class are returned. Otherwise, this | ||
| determines the type of averaging performed on the data: | ||
|
|
||
| ``'binary'``: | ||
| Only report results for the class specified by ``pos_label``. | ||
| This is applicable only if targets (``y_{true,pred}``) are binary. | ||
| ``'micro'``: | ||
| Calculate metrics globally by counting the total true positives, | ||
| false negatives and false positives. | ||
| ``'macro'``: | ||
| Calculate metrics for each label, and find their unweighted | ||
| mean. This does not take label imbalance into account. | ||
| ``'weighted'``: | ||
| Calculate metrics for each label, and find their average weighted | ||
| by support (the number of true instances for each label). This | ||
| alters 'macro' to account for label imbalance. | ||
| ``'samples'``: | ||
| Calculate metrics for each instance, and find their average (only | ||
| meaningful for multilabel classification where this differs from | ||
| :func:`accuracy_score`). | ||
|
|
||
| warn_for : tuple or set, for internal use | ||
| This determines which warnings will be made in the case that this | ||
| function is being used to return only one of its metrics. | ||
|
|
||
| sample_weight : array-like of shape (n_samples,), default=None | ||
| Sample weights. | ||
|
|
||
| zero_division : "warn", 0 or 1, default="warn" | ||
| Sets the value to return when there is a zero division: | ||
| - tpr, fnr: when there are no positive labels | ||
| - fpr, tnr: when there are no negative labels | ||
|
|
||
| If set to "warn", this acts as 0, but warnings are also raised. | ||
|
|
||
| Returns | ||
| ------- | ||
| tpr : float (if average is not None) or array of float, shape =\ | ||
| [n_unique_labels] | ||
|
|
||
| fpr : float (if average is not None) or array of float, shape =\ | ||
| [n_unique_labels] | ||
|
|
||
| tnr : float (if average is not None) or array of float, shape =\ | ||
| [n_unique_labels] | ||
|
|
||
| fnr : float (if average is not None) or array of float, shape =\ | ||
| [n_unique_labels] | ||
| The number of occurrences of each label in ``y_true``. |
There was a problem hiding this comment.
Do you mind checking scikit-learn guidelines for writing documentation and homogenize parameter and attribute descriptions? Thanks.
…into confusion-matrix-derived-metrics
|
@haochunchang I see you are online right now: do you mind fixing the description, referring to #15522. Thanks a lot! And thanks for coming back at this! |
a4652ca to
f74fc10
Compare
|
Hi! |
vaibhavmehrotraml
approved these changes
Dec 26, 2020
| warn_for=('tpr', 'fpr', 'tnr', 'fnr'), sample_weight=None,zero_division="warn"): | ||
| """Compute TPR, FPR, TNR, FNR for each class | ||
|
|
||
| The TPR is the ratio ``tp / (tp + fn)`` where ``tp`` is the number of |
There was a problem hiding this comment.
The TPR, also called sensitivity or recall, is the ratio ...
Might be more informative
| The FPR is the ratio ``fp / (tn + fp)`` where ``tn`` is the number of | ||
| true negatives and ``fp`` the number of false positives. | ||
|
|
||
| The TNR is the ratio ``tn / (tn + fp)`` where ``tn`` is the number of |
There was a problem hiding this comment.
The TNR, also called specificity or selectivity, is the ratio
Might be more informative
sklearn/metrics/_classification.py
Outdated
| fp_sum = MCM[:, 0, 1] | ||
| fn_sum = MCM[:, 1, 0] | ||
| tp_sum = MCM[:, 1, 1] | ||
| pred_sum = tp_sum + MCM[:, 0, 1] |
There was a problem hiding this comment.
Any specific reason to not use fp_sum instead of MCM[:, 0, 1]?
|
closing in favor of #19556 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Take over PR #15532
Adding Fall-out, Miss rate, specificity as metrics #5516
What does this implement/fix? Explain your changes.
Implemented a function which returns fpr, tpr, fnr, tnr.
Any other comments?
As this comment mentioned, tn, fp, fn, tp can also be calculated from confusion matrix.
Is this function provide a more flexible way for calculating the rates?
Any advices and help are very appreciated.
Co-authored by @ddhar1 @samskruthireddy