[MRG] FIX solve consistency between predict and predict_proba in AdaBoost #14114
Conversation
|
@NicolasHug @amueller I am playing with something which is not my strong suit. Could you have a look at the PR and if the fix proposed seems theoretically reasonable? |
There was a problem hiding this comment.
As far as I can tell the changes are correct.
What is strange to me is that the previous implementation is also a softmax of the decision function, so other than the fact that the new version is much clearer, I don't understand where the fix is.
(to be honest this isn't really my specialty either)
sklearn/ensemble/weight_boosting.py
Outdated
| 2009. | ||
| """ | ||
| if n_classes == 2: | ||
| decision = np.vstack([-decision, decision]).T |
There was a problem hiding this comment.
can you explain this? is it in the paper too?
There was a problem hiding this comment.
My original thought was that we were keeping one of the 2 columns when computing the decision function. Actually, we do something a bit different:
if n_classes == 2:
pred[:, 0] *= -1
return pred.sum(axis=1)You originally have symmetry between both classes. However, I should divide decision by 2 since that we are summing.
It uses the |
|
@NicolasHug do you have any other comments? Any second reviewer? @rth @thomasjpfan |
|
@thomasjpfan I addressed the comments. |
|
thx @glemaitre |
closes #14084
closes #2974
compute the probabilities in AdaBoostClassifier as specified in "Multiclass AdaBoost"