[MRG+1] Add Brier_score_loss to model validation doc #6841
Conversation
2e1e031 to
20ff083
Compare
|
LGTM |
| >>> y_prob = np.array([0.1, 0.9, 0.8, 0.4]) | ||
| >>> y_pred = np.array([0, 1, 1, 0]) | ||
| >>> brier_score_loss(y_true, y_prob) # doctest: +ELLIPSIS | ||
| 0.055 |
There was a problem hiding this comment.
You don't need # doctest: +ELLIPSIS if you don't use the ... placeholder in the string representation of the result.
|
Please also add a reference to the Wikipedia article that gives more interesting details. |
…x typo in doctest result
|
@ogrisel thanks for the feedback, added the explanation of the why of the metrics, reference and the fix to the doctest. Let me me know what you think. thanks! |
| (the mean square difference is smaller), the more accurate the prediction is. | ||
| It can be thought of as a measure of the "calibration" of a set of probabilistic predictions. | ||
| `Here <ttp://docs.lib.noaa.gov/rescue/mwr/078/mwr-078-01-0001.pdf>`_ is | ||
| the paper describing it. |
There was a problem hiding this comment.
Instead of putting an inline link with "Here", please use the standard "Reference" section format used consistently throughout the scikit-learn documentation.
|
Did you understand why those doctests were not run? |
Apparently the travis tests did not run at all on your first commit for some reason, only the appveyor tests did pass but those do not run the doctests under windows intentionally. Anyway travis ran the tests in |
…fix references section and pep8 formatting
|
@ogrisel |
| BS = \frac{1}{N} \sum_{t=1}^{N}(f_t - o_t)^2 | ||
|
|
||
| where : :math:`N` is the total number of predictions, :math:`f_t` is the predicted | ||
| probablity of the actual outcome :math:`o_t` |
There was a problem hiding this comment.
It's a detail, but I prefer if, before a code block, you add "::", which is the explicit way of defining a code-block, rather than relying on indentation, which is fragile.
There was a problem hiding this comment.
I agree. I will do that change and squash/merge.
Reference Issue
Add Brier_score_loss to model validation doc #4804
What does this implement/fix? Explain your changes.
Trying to fix the travis build with ellipsis in the docs