Skip to content

DOC Add note on overlapping test sets in GroupShuffleSplit #29676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 23, 2024
Merged

DOC Add note on overlapping test sets in GroupShuffleSplit #29676

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4f27090
improve
lucyleeow Aug 15, 2024
e13035f
Merge branch 'main' into doc_split_shuffle
lucyleeow Aug 22, 2024
9ec90ad
review
lucyleeow Aug 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions sklearn/model_selection/_split.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1932,8 +1932,9 @@ class ShuffleSplit(_UnsupportedGroupCVMixin, BaseShuffleSplit):
Yields indices to split data into training and test sets.

Note: contrary to other cross-validation strategies, random splits
do not guarantee that all folds will be different, although this is
still very likely for sizeable datasets.
do not guarantee that test sets across all folds will be mutually exclusive,
and might include overlapping samples. However, this is still very likely for
sizeable datasets.

Read more in the :ref:`User Guide <ShuffleSplit>`.

Expand Down Expand Up @@ -2049,6 +2050,11 @@ class GroupShuffleSplit(GroupsConsumerMixin, BaseShuffleSplit):
``LeavePGroupsOut(p=10)`` would be
``GroupShuffleSplit(test_size=10, n_splits=100)``.

Contrary to other cross-validation strategies, the random splits
do not guarantee that test sets across all folds will be mutually exclusive,
and might include overlapping samples. However, this is still very likely for
sizeable datasets.

Note: The parameters ``test_size`` and ``train_size`` refer to groups, and
not to samples as in :class:`ShuffleSplit`.

Expand Down Expand Up @@ -2176,13 +2182,14 @@ class StratifiedShuffleSplit(BaseShuffleSplit):

Provides train/test indices to split data in train/test sets.

This cross-validation object is a merge of StratifiedKFold and
ShuffleSplit, which returns stratified randomized folds. The folds
This cross-validation object is a merge of :class:`StratifiedKFold` and
:class:`ShuffleSplit`, which returns stratified randomized folds. The folds
are made by preserving the percentage of samples for each class.

Note: like the ShuffleSplit strategy, stratified random splits
do not guarantee that all folds will be different, although this is
still very likely for sizeable datasets.
Note: like the :class:`ShuffleSplit` strategy, stratified random splits
do not guarantee that test sets across all folds will be mutually exclusive,
and might include overlapping samples. However, this is still very likely for
sizeable datasets.

Read more in the :ref:`User Guide <stratified_shuffle_split>`.

Expand Down