Ambiguity in brier score doc fixed #10969
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Please refer to the log https://travis-ci.org/scikit-learn/scikit-learn/jobs/365980305 to resolve the flake8 errors
sklearn/metrics/classification.py
Outdated
| @@ -1929,7 +1929,7 @@ def brier_score_loss(y_true, y_prob, sample_weight=None, pos_label=None): | |||
| takes on a value between zero and one, since this is the largest | |||
| possible difference between a predicted probability (which must be | |||
| between zero and one) and the actual outcome (which can take on values | |||
| of only 0 and 1). | |||
| of only 0 and 1). The Brier loss is decomposed of refinement loss and calibration loss. | |||
There was a problem hiding this comment.
E501 line too long (91 > 79 characters)
jnothman
reviewed
Apr 14, 2018
| is evaluated with Brier score :func:`brier_score_loss`, reported in the legend | ||
| (the smaller the better). | ||
| both isotonic calibration and sigmoid calibration. | ||
| The Brier score is a metric which is a combination of calibration loss and refinement loss, |
There was a problem hiding this comment.
This is no longer grammatical. Try "Performance is evaluated with ...." then adding a new sentence summarising the intention of the metric.
There was a problem hiding this comment.
you're not explaining calibration and refinement loss, right?
sklearn/metrics/classification.py
Outdated
| @@ -1929,7 +1929,8 @@ def brier_score_loss(y_true, y_prob, sample_weight=None, pos_label=None): | |||
| takes on a value between zero and one, since this is the largest | |||
| possible difference between a predicted probability (which must be | |||
| between zero and one) and the actual outcome (which can take on values | |||
| of only 0 and 1). | |||
| of only 0 and 1). The Brier loss is decomposed of refinement loss | |||
jnothman
approved these changes
May 7, 2018
doc/modules/calibration.rst
Outdated
| The Brier score is a metric which is a combination of calibration loss and refinement loss, | ||
| :func:`brier_score_loss`, reported in the legend (the smaller the better). | ||
| Calibration loss is defined as the mean squared deviation from empirical probabilities | ||
| derived from slope of ROC segments. Refinement loss can be defined as the expected |
fix grammar as suggested by jnothman.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Fix #10883
What does this implement/fix? Explain your changes.
Ambiguity in brier score doc fixed. Details about calibration loss added.
Any other comments?