Description
I'm currently working on adding support for the Python 3.13 free-threaded build to projects in the scientific python ecosystem. We are tracking this work at https://github.com/Quansight-Labs/free-threaded-compatibility. Right now we're focusing on projects relatively low in the stack. scikit-learn isn't the lowest in the stack but it has a lot of tests that perform multithreaded workflows so running the scikit-learn tests is a productive way to elucidate threading bugs in scikit-learn and its dependencies.
Currently, scikit-learn builds fine and almost all the tests pass with the GIL disabled 🎉
The following tests have failures:
FAILED sklearn/ensemble/tests/test_voting.py::test_sample_weight[42] - AssertionError:
FAILED sklearn/model_selection/tests/test_search.py::test_random_search_cv_results_multimetric - AssertionError:
FAILED sklearn/semi_supervised/tests/test_self_training.py::test_classification[threshold-base_estimator1] - AssertionError:
FAILED sklearn/semi_supervised/tests/test_self_training.py::test_classification[k_best-base_estimator1] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_consistent_proba - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_libsvm_iris - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_svc_clone_with_callable_kernel - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_consistent_proba - AssertionError:
FAILED sklearn/tests/test_build.py::test_openmp_parallelism_enabled - AssertionError:
FAILED sklearn/tests/test_pipeline.py::test_pipeline_memory - AssertionError:
Note that some of these failures are flaky and you may not get the same list if you run the scikit-learn tests. I also saw one deadlock in test_calibration.py::test_parallel_execution, so I'd also suggest running pytest with e.g. -o fauthandler_timeout=60 set so you will get python tracebacks for all threads if any test hangs and runs longer than 60 seconds. Your mileage may vary on how fast your computer is, so 60 seconds may not be sufficient if any tests take that long to run on your system.
My plan is to go through these failures and figure out what's going wrong. Some of these may not be specific to the free-threaded build.
To run the tests with the GIL disabled, you'll need to install 3.13b1 or the latest version of the CPython 3.13 branch, build cython, numpy, and scipy from the latest version of their respective git repositories. You'll need to e.g. pip install pybind11 and pythran after installing numpy and before building scipy. Then build scikit-learn as an editable install and run:
PYTHON_GIL=0 python -m pytest sklearn