Skip to content

Vendor threadpoolctl #14979

Closed
Closed
@jeremiedbb

Description

@jeremiedbb

As discussed in #11950 and #14265, we need a way to control at runtime the number of threads OpenMP or BLAS are allowed to use. This is to prevent oversubscription in nested parallelism of the form

joblib(backend = 'threading')
    BLAS or OpenMP

or

OpenMP
    BLAS

(Note the backend='loky' case should/will be handled by loky itself)

With @ogrisel and @tomMoral, we've been working on threadpoolctl to solve this issue. There might be a plan to include it in numpy so at some point we'll be able to import it from numpy. In the meantime I propose to vendor it in scikit-learn (I openend #14980).

The first place where it's needed is in #11950, but there might be other places as well.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions