Skip to content

DOC Add link to Early Stopping example in Gradient Boosting #27025

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Sep 29, 2023
6 changes: 4 additions & 2 deletions doc/modules/ensemble.rst
Original file line number Diff line number Diff line change
Expand Up @@ -740,8 +740,10 @@ of ``learning_rate`` require larger numbers of weak learners to maintain
a constant training error. Empirical evidence suggests that small
values of ``learning_rate`` favor better test error. [HTF]_
recommend to set the learning rate to a small constant
(e.g. ``learning_rate <= 0.1``) and choose ``n_estimators`` by early
stopping. For a more detailed discussion of the interaction between
(e.g. ``learning_rate <= 0.1``) and choose ``n_estimators`` large enough
that early stopping applies,
see :ref:`sphx_glr_auto_examples_ensemble_plot_gradient_boosting_early_stopping.py`
for a more detailed discussion of the interaction between
``learning_rate`` and ``n_estimators`` see [R2007]_.

Subsampling
Expand Down
2 changes: 1 addition & 1 deletion examples/ensemble/plot_gradient_boosting_early_stopping.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""
===================================
Early stopping of Gradient Boosting
Early stopping in Gradient Boosting
===================================

Gradient boosting is an ensembling technique where several weak learners
Expand Down
4 changes: 4 additions & 0 deletions sklearn/ensemble/_gb.py
Original file line number Diff line number Diff line change
Expand Up @@ -1282,6 +1282,8 @@ class GradientBoostingClassifier(ClassifierMixin, BaseGradientBoosting):
improving in all of the previous ``n_iter_no_change`` numbers of
iterations. The split is stratified.
Values must be in the range `[1, inf)`.
See
:ref:`sphx_glr_auto_examples_ensemble_plot_gradient_boosting_early_stopping.py`.

.. versionadded:: 0.20

Expand Down Expand Up @@ -1891,6 +1893,8 @@ class GradientBoostingRegressor(RegressorMixin, BaseGradientBoosting):
improving in all of the previous ``n_iter_no_change`` numbers of
iterations.
Values must be in the range `[1, inf)`.
See
:ref:`sphx_glr_auto_examples_ensemble_plot_gradient_boosting_early_stopping.py`.

.. versionadded:: 0.20

Expand Down
6 changes: 3 additions & 3 deletions sklearn/linear_model/_passive_aggressive.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,11 @@ class PassiveAggressiveClassifier(BaseSGDClassifier):
.. versionadded:: 0.19

early_stopping : bool, default=False
Whether to use early stopping to terminate training when validation.
Whether to use early stopping to terminate training when validation
score is not improving. If set to True, it will automatically set aside
a stratified fraction of training data as validation and terminate
training when validation score is not improving by at least tol for
n_iter_no_change consecutive epochs.
training when validation score is not improving by at least `tol` for
`n_iter_no_change` consecutive epochs.

.. versionadded:: 0.20

Expand Down
6 changes: 3 additions & 3 deletions sklearn/linear_model/_perceptron.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,11 @@ class Perceptron(BaseSGDClassifier):
See :term:`Glossary <random_state>`.

early_stopping : bool, default=False
Whether to use early stopping to terminate training when validation.
Whether to use early stopping to terminate training when validation
score is not improving. If set to True, it will automatically set aside
a stratified fraction of training data as validation and terminate
training when validation score is not improving by at least tol for
n_iter_no_change consecutive epochs.
training when validation score is not improving by at least `tol` for
`n_iter_no_change` consecutive epochs.

.. versionadded:: 0.20

Expand Down
2 changes: 1 addition & 1 deletion sklearn/neural_network/_multilayer_perceptron.py
Original file line number Diff line number Diff line change
Expand Up @@ -887,7 +887,7 @@ class MLPClassifier(ClassifierMixin, BaseMultilayerPerceptron):
Whether to use early stopping to terminate training when validation
score is not improving. If set to true, it will automatically set
aside 10% of training data as validation and terminate training when
validation score is not improving by at least tol for
validation score is not improving by at least ``tol`` for
``n_iter_no_change`` consecutive epochs. The split is stratified,
except in a multilabel setting.
If early stopping is False, then the training stops when the training
Expand Down