Skip to content

precision_recall_curve should raise UndefinedMetricWarning when there are no true samples #13043

@zjpoh

Description

@zjpoh

Description

When precision_recall_score is calculated with no true samples, a RuntimeWarning due to division by 0 is raised and nan is returned. However, when recall_score is calculated with no true labels, a UndefinedMetricWarning due to no true samples is raised.

For consistency, I think precision_recall_score should raise the same error and return 0.

Steps/Code to Reproduce

from sklearn.metrics import precision_recall_curve, recall_score
print(precision_recall_curve([0, 0], [0, 1]))
print(recall_score([0, 0], [0, 1]))

Expected Results

/Users/.../scikit-learn/sklearn/metrics/classification.py:1293: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 due to no true samples.
(array([0., 1.]), array([0.,  0.]), array([1]))
/Users/.../scikit-learn/sklearn/metrics/classification.py:1293: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 due to no true samples.
  'recall', 'true', average, warn_for)
0.0

Actual Results

/Users/.../scikit-learn/sklearn/metrics/ranking.py:601: RuntimeWarning: invalid value encountered in true_divide
  recall = tps / tps[-1]
(array([0., 1.]), array([nan,  0.]), array([1]))
/Users/.../scikit-learn/sklearn/metrics/classification.py:1293: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 due to no true samples.
  'recall', 'true', average, warn_for)
0.0

Versions

System:
    python: 3.7.2 (default, Dec 29 2018, 00:00:04)  [Clang 4.0.1 (tags/RELEASE_401/final)]
executable: /Users/.../anaconda3/envs/sklearn-py37/bin/python
   machine: Darwin-18.2.0-x86_64-i386-64bit

BLAS:
    macros: SCIPY_MKL_H=None, HAVE_CBLAS=None
  lib_dirs: /Users/.../anaconda3/envs/sklearn-py37/lib
cblas_libs: mkl_rt, pthread

Python deps:
       pip: 18.1
setuptools: 40.6.3
   sklearn: 0.21.dev0
     numpy: 1.15.4
     scipy: 1.1.0
    Cython: 0.29.2
    pandas: 0.23.4

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions