float?

Is this treatment from logloss? it seems a bit magical.

@amueller Thanks.
Currently, log_loss don't support pos_label so the code is taken from _binary_clf_curve. But in the test we have to support situations when y_pred = ['egg', 'spam'] without a pos_label (see test_invariance_string_vs_numbers_labels). In order not to remove test, I give this awkward solution.
Btw, it might be OK not to use float like _binary_clf_curve because np.array_equal([1.0], [1])=True

I don't understand. That test uses pos_label right?

Why instead don't you put this logic in _check_binary_probabilistic_predictions? Surely it (if not pos_label, at least the handling of a single class) applies there too.

I meant that this logic could move to the helper, so that other binary probabilistic metrics would similarly identify -1 alone as the negative class, and to avoid rubbing unique multiple times.

@jnothman Sorry but I don't quite understand. This part is about giving pos_label the right value and raise an error when needed. Currently, _check_binary_probabilistic_predictions don't accept pos_label as a parameter. So it might be hard to move the logic into _check_binary_probabilistic_predictions .

After determining pos_label's appropriate value, it is used once: to binarise y_true into {0,1}. This is also what _check_binary_probabilistic_predictions does, only without the ability to configure pos_label. If _check_binary_probabilistic_predictions took an optional pos_label you could handle all cases.

@jnothman Sorry to disturb again but I have some questions:
(1)It seems that _check_binary_probabilistic_predictions is only used in brier_score_loss(metrics.classification) and calibration_curve(calibration).
The latter do not support pos_label.
(2)I am wondering how we should took the optional pos_label since even if we support pos_label, there are cases when pos_label is not provided by users(default to None).
It seems that I cannot use def(..., pos_label=None) and it seems ugly to use something like def(..., **dict) .
(3)From my perspective, since _check_binary_probabilistic_predictions is only used in two functions and in each of these two functions, some part of it is meaningless(e.g., label_binarize in brier_score_loss and the check of y_prob in calibration_curve), how about remove the function(no test directly use the function)?

label_binarize isn't meaningless in brier: it's basically identical to what's happening in https://github.com/scikit-learn/scikit-learn/pull/9562/files/adfbbec50f326fe0a3678ea29c630269ef258fe1#diff-b04acd877dd793f28ae7be13a999ed88R1931

And why can't you use def _check_binary_probabilistic_predictions(y_true, y_prob, pos_label=None)?

@jnothman For example, in the function, if pos_label=None, there are two situations
(1)I don't pass pos_label(used in calibration)
(2)I pass pos_label=None(used in brier_score_loss when pos_label is not provided)
How can I distinguish them? Thanks. :)

Why do you need to distinguish them? Should they not be treated the same?

I think these two lines are unnecessary if you just set y_true to the output of _check_binary_probabilistic_predictions

@jnothman Sorry but I don't understand your meaning. Which two lines? The core issue I want to solve here is that previous implementation use the function after y_true is binarized. Kindly provide more details please. Thanks.

Right, I realise again that you're not using _check_binary_probabilistic_predictions because it does not support pos_label. But it probably should.

My issue is that _check_binary_probabilistic_predictions does processing and returns something. It's weird not to use that return value.

I think I would add to _check_binary_probabilistic_predictions:

out = label_binarize...
if pos_label is not None:
    if pos_label != labels[-1]:
        if pos_label != labels[0]:
            raise ValueError...
        out = 1 - out