Currently each model that needs it defines their own losses, it might be useful to put them all in one place to see if anything could be re-used in the future.

In particular, losses are defined in,

• HistGradientBoosting: sklearn/ensemble/_hist_gradient_boosting/loss.py
• GradientBoosting: sklearn/ensemble/_gb_losses.py
• SGD: sklearn/linear_model/sgd_fast.pyx
• GLM: Minimal Generalized linear models implementation (L2 + lbfgs) #14300 would add sklearn/linear_model/_glm/distributions.py
• MLP: sklearn/neural_network/_base.py:LOSS_FUNCTIONS
• losses implementations in sklearn.metrics
• and somewhat related RandomForest: sklearn/tree/_criterion.pyx

This issues proposed to progressively put most of them (except for RF criterions) in a single private folder sklearn/_loss to see what could be made less redundant there.

In particular, for the GLM PR in #1430 this would allow breaking circular dependency between the sklearn.linear_model and sklearn.metrics modules #14300 (comment)

@NicolasHug I don't know if you already had some plans about unifying new and legacy gradient boosting losses and if this would go in the right direction. cc @ogrisel

Weakly related to #5044 #3481 #5368