Description
Currently, in scikit-learn, we have multiple definitions of pos_label=None and the doc seems not clear, which might confuse users.
(1) precision/recall/f1/fbeta: do not support pos_label=None
(2) brier_score_loss: default pos_label=None, means the greater of two values (the implementation currently under fix by me in #9562)
(3) precision_recall_curve/roc_auc_curve: default pos_label=None, means pos_label=1
According to @jnothman
Actually, I suspect None should mean "the greater of two values" (unless there is only one value present).
possible solutions:
(1)deprecate pos_label=None and change the default value to 1
(2)correct the behaviour of pos_label=None in precision_recall_curve/roc_auc_curve
(3)ignore the difference and simply update the doc
Personally, I advocate for (1), at least in precision_recall_curve/roc_auc_curve, because
(1) It will not influence user's code too much, unless you are using something like y_true = ['a', 'b'] and don't provide a pos_label. In this case, it seems not reasonable to let scikit-learn set pos_label='b' for you.
(2) We don't have a correct implementation of pos_label=None in scikit-learn currently. The meaning of pos_label=None is hard to understand by user and hard to define. If there are two labels, ok, we pick the greater one. How about the situation if there is only one label? Pick that one? I'm afraid it is not always the case ,e.g., when there's only negative samples.
cc @jnothman