The negative_gradient method of ExponentialLoss returns the positive gradient #9666
Labels
Comments
|
I would like to submit a patch for this issue. |
|
I'm not sure that anyone from the core team has confirmed the issue yet,
but if you can identify it and test for correct behaviour, go ahead and
submit a patch.
…On 2 Sep 2017 3:46 pm, "theuchiha" ***@***.***> wrote:
I would like to submit a patch for this issue.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#9666 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz6wrTRvE0ZhQNEMFPrEtrmD4Lzd2Uks5seOvDgaJpZM4PJszr>
.
|
|
Indeed, this seems fishy. We used the following as a reference: https://pbil.univ-lyon1.fr/CRAN/web/packages/gbm/vignettes/gbm.pdf The loss for a given point is given by: u = -(2.0 * y_true - 1.0)
np.exp(u * y_pred)Thus taking the derivative respect to u = -(2.0 * y_true - 1.0)
u * np.exp(u * y_pred)Thus, the negative gradient is indeed: u = -(2.0 * y_true - 1.0)
- u * np.exp(u * y_pred)All gradient reported in the document seems to be the negative gradient but not for the AdaBoost loss. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've been experimenting with gradient boosting and noticed that the
ExponentialLossloss function has a minor bug. Thenegative_gradientmethod actually returns the positive gradient of the loss function. As far as I can tell, this doesn't affect fitting at all for the sklearnGradientBoostingClassifierbecause of the way_update_terminal_regionworks. However, when using the loss functions for a custom gradient boosting implementation that uses a line search, it caused the algorithm to fail to progress. Adding a minus sign to this line fixed the problem.The text was updated successfully, but these errors were encountered: