Closed
Description
As discussed in #11950 and #14265, we need a way to control at runtime the number of threads OpenMP or BLAS are allowed to use. This is to prevent oversubscription in nested parallelism of the form
joblib(backend = 'threading')
BLAS or OpenMP
or
OpenMP
BLAS
(Note the backend='loky' case should/will be handled by loky itself)
With @ogrisel and @tomMoral, we've been working on threadpoolctl to solve this issue. There might be a plan to include it in numpy so at some point we'll be able to import it from numpy. In the meantime I propose to vendor it in scikit-learn (I openend #14980).
The first place where it's needed is in #11950, but there might be other places as well.