CI Uses pytest-xdist to parallelize tests #13041
Conversation
|
The same LGTM error happening on master as well: https://lgtm.com/projects/g/scikit-learn/scikit-learn/rev/3cb0b3bc22b0a6f99f80c025fd378660896a2c68 |
|
I merge #13044 PR. Could merge master. It seems that Travis did not launch last time. |
|
Not being able to speed up the travis build is concerning. I suspect running coverage slows it down. I am not content with how On my system, |
|
BTW This PR only enables |
|
The error in lgtm is a unrelated. I will fix it in another PR. |
| @@ -31,10 +31,15 @@ test-doc: | |||
| ifeq ($(BITS),64) | |||
| $(PYTEST) $(shell find doc -name '*.rst' | sort) | |||
| endif | |||
| test-code-parallel: in | |||
| $(PYTEST) -n auto --showlocals -v sklearn --durations=20 | |||
There was a problem hiding this comment.
Maybe add a comment that this requires pytest-xdist. Else pytest fails with
pytest: error: unrecognized arguments: -n
| IF "%PYTHON_ARCH%"=="64" ( | ||
| call activate %VIRTUALENV% | ||
| set PYTEST_ARGS=%PYTEST_ARGS% -n2 |
There was a problem hiding this comment.
Briefly describe -n2 for future us?
| @@ -123,11 +123,14 @@ def test_normalized_output(metric_name): | |||
| assert not (score < 0).any() | |||
|
|
|||
|
|
|||
| SUPERVISED_UNSUPERVISED_METRICS = {**SUPERVISED_METRICS, | |||
There was a problem hiding this comment.
has it been reverted @thomasjpfan ?
sklearn/utils/tests/test_testing.py
Outdated
| @@ -446,6 +446,7 @@ def fit(self, X, y): | |||
| """Incorrect docstring but should not be tested""" | |||
|
|
|||
|
|
|||
| @ignore_warnings(category=DeprecationWarning) | |||
There was a problem hiding this comment.
why is this needed? add a comment?
|
Looks like some of the sorted parameters are still dicts. Since xdist will only work on python >= 3.6 this is not needed. |
|
If you think this is finished, feel free to merge, @thomasjpfan |
|
I get 3.1x speed-up for the test suite on 4 cores (using |
Reference Issues/PRs
Fixes #13016
What does this implement/fix? Explain your changes.
Overall the changes are made to address how dictionaries are not ordered in Python <= 3.5. The major changes are as follows:
OrderedDictin test to make sure the values passedpytest.mark.parametrizehas a deterministic order.sortedon sets for a deterministic order.sklearn/linear_model/coordinate_descent.pywere to ensure that staticmethodpathhas a deterministic order in their respective classes. Only the two classes that useslasso_pathas a staticmethod are affected.pytest-xdist.Any other comments?
When
pytest-xdistspawns processes, the processes independently look through all files insklearnto test. If the order of discovered test functions are not the same for all processes,pytest-xdistwill return an error.Travis-CI does not look like it gets that much faster. Locally, this PR does speed up running our tests suite.