A common private module for differentiable loss functions used as objective functions in estimators #15123
Description
Currently each model that needs it defines their own losses, it might be useful to put them all in one place to see if anything could be re-used in the future.
In particular, losses are defined in,
- HistGradientBoosting:
sklearn/ensemble/_hist_gradient_boosting/loss.py - GradientBoosting:
sklearn/ensemble/_gb_losses.py - SGD:
sklearn/linear_model/sgd_fast.pyx - GLM: Minimal Generalized linear models implementation (L2 + lbfgs) #14300 would add
sklearn/linear_model/_glm/distributions.py - MLP:
sklearn/neural_network/_base.py:LOSS_FUNCTIONS - losses implementations in
sklearn.metrics - and somewhat related RandomForest:
sklearn/tree/_criterion.pyx
This issues proposed to progressively put most of them (except for RF criterions) in a single private folder sklearn/_loss to see what could be made less redundant there.
In particular, for the GLM PR in #1430 this would allow breaking circular dependency between the sklearn.linear_model and sklearn.metrics modules #14300 (comment)
@NicolasHug I don't know if you already had some plans about unifying new and legacy gradient boosting losses and if this would go in the right direction. cc @ogrisel