From 6cf009e6dbcfc353f83cd3af4000da5789f73b8e Mon Sep 17 00:00:00 2001 From: Greg Gandenberger Date: Wed, 5 Dec 2018 14:03:19 -0600 Subject: [PATCH 1/2] Replace undefined symbols with defined symbols Replace undefined "`A`" and "`B`" in definitions of precision, recall, and F-beta with defined symbols "`y_l`" and "`\hat{y}_l`" and correct the resulting definitions. --- doc/modules/model_evaluation.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/modules/model_evaluation.rst b/doc/modules/model_evaluation.rst index cde6174239094..d650b56bb9159 100644 --- a/doc/modules/model_evaluation.rst +++ b/doc/modules/model_evaluation.rst @@ -887,11 +887,11 @@ To make this more explicit, consider the following notation: * :math:`y_l` the subset of :math:`y` with label :math:`l` * similarly, :math:`\hat{y}_s` and :math:`\hat{y}_l` are subsets of :math:`\hat{y}` -* :math:`P(A, B) := \frac{\left| A \cap B \right|}{\left|A\right|}` -* :math:`R(A, B) := \frac{\left| A \cap B \right|}{\left|B\right|}` - (Conventions vary on handling :math:`B = \emptyset`; this implementation uses - :math:`R(A, B):=0`, and similar for :math:`P`.) -* :math:`F_\beta(A, B) := \left(1 + \beta^2\right) \frac{P(A, B) \times R(A, B)}{\beta^2 P(A, B) + R(A, B)}` +* :math:`P(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|\hat{y}_l\right|}` +* :math:`R(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|y_l\right|}` + (Conventions vary on handling :math:`\hat{y}_l = \emptyset`; this implementation uses + :math:`R(y_l, \hat{y}_l):=0`, and similar for :math:`P`.) +* :math:`F_\beta(y_l, \hat{y}_l) := \left(1 + \beta^2\right) \frac{P(y_l, \hat{y}_l) \times R(y_l, \hat{y}_l)}{\beta^2 P(y_l, \hat{y}_l) + R(y_l, \hat{y}_l)}` Then the metrics are defined as: From 0af5c1ee092c3b05108201ee926dcdb862dd6d3a Mon Sep 17 00:00:00 2001 From: Greg Gandenberger Date: Wed, 5 Dec 2018 15:14:57 -0600 Subject: [PATCH 2/2] Swap `y_l` and `\hat{y}_l` in definitions I missed that the definitions use `\hat{y}` for true labels and `y` for predicted labels, so I had precision and recall reversed. --- doc/modules/model_evaluation.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/modules/model_evaluation.rst b/doc/modules/model_evaluation.rst index d650b56bb9159..f4177c61b5025 100644 --- a/doc/modules/model_evaluation.rst +++ b/doc/modules/model_evaluation.rst @@ -887,8 +887,8 @@ To make this more explicit, consider the following notation: * :math:`y_l` the subset of :math:`y` with label :math:`l` * similarly, :math:`\hat{y}_s` and :math:`\hat{y}_l` are subsets of :math:`\hat{y}` -* :math:`P(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|\hat{y}_l\right|}` -* :math:`R(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|y_l\right|}` +* :math:`P(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|y_l\right|}` +* :math:`R(y_l, \hat{y}_l) := \frac{\left| y_l \cap \hat{y}_l \right|}{\left|\hat{y}_l\right|}` (Conventions vary on handling :math:`\hat{y}_l = \emptyset`; this implementation uses :math:`R(y_l, \hat{y}_l):=0`, and similar for :math:`P`.) * :math:`F_\beta(y_l, \hat{y}_l) := \left(1 + \beta^2\right) \frac{P(y_l, \hat{y}_l) \times R(y_l, \hat{y}_l)}{\beta^2 P(y_l, \hat{y}_l) + R(y_l, \hat{y}_l)}`