Skip to content

[MRG] Adds documentation for parallelisation of custom scorer #12813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jan 15, 2019
Merged

[MRG] Adds documentation for parallelisation of custom scorer #12813

Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
43980ff
Adds doc, for custom scorer parallelisation
fx86 Dec 15, 2018
c21a283
updated documentation for parallel custom scorer
fx86 Dec 19, 2018
de3a2e2
MAINT: Fixed formatting in model_selection.rst
fx86 Dec 27, 2018
9ded716
adds # doctest: +SKIP to circumvent ModuleNotFoundError
fx86 Dec 31, 2018
edfa3cf
adds another # doctest: +SKIP to circumvent ModuleNotFoundError
fx86 Dec 31, 2018
bf14c0e
adds newline and indentation for the new code block
fx86 Jan 2, 2019
34e4113
adds context of other joblib backends in custom scorer functionality
fx86 Jan 2, 2019
720d011
updated documentation on custom scorer's parallelisation
fx86 Jan 13, 2019
a5e7e57
fixes custom scorer documentation for clarity
fx86 Jan 14, 2019
176dfc2
fixes custom scorer documentation for clarity
fx86 Jan 14, 2019
b0780de
fixes custom scorer documentation for clarity
fx86 Jan 14, 2019
b18c442
remove backticks from note-section title
fx86 Jan 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions doc/modules/model_evaluation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,25 @@ the following two rules:
Again, by convention higher numbers are better, so if your scorer
returns loss, that value should be negated.

.. note:: **Using custom scorers in functions where n_jobs > 1**

While defining the custom scoring function alongside the calling function
should work out of the box with the default joblib backend (loky),
importing it from another module will be a more robust approach and work
independently of the joblib backend.

For example, to use, ``n_jobs`` greater than 1 in the example below,
``custom_scoring_function`` function is saved in a user-created module
(``custom_scorer_module.py``) and imported::

>>> from custom_scorer_module import custom_scoring_function # doctest: +SKIP
>>> cross_val_score(model,
... X_train,
... y_train,
... scoring=make_scorer(custom_scoring_function, greater_is_better=False),
... cv=5,
... n_jobs=-1) # doctest: +SKIP

.. _multimetric_scoring:

Using multiple metric evaluation
Expand Down