Thanks for the report, this is indeed quite unexpected.

I am not sure I can reproduce on my local machine. EDIT: I had not read "On my laptop, both work as expected.".

Can you please try the following:

import torch
from joblib import Parallel, delayed
import os


def heavy_func():
    for _ in range(10000):
        [i for i in range(10000)]
    return os.getpid()


n_jobs = 40
results = Parallel(n_jobs=n_jobs)(delayed(heavy_func)() for _ in range(2*n_jobs))
print(len(set(results)))

This gave me different numbers each time I ran the script.

When torch is imported first: on 5 runs I got: 27, 23, 20, 17, 16.

When joblib is imported first: on 5 runs I got: 4, 2, 4, 3, 4.

I am not sure I can reproduce on my local machine

The example of my first message was on the Margaret cluster. Maybe you can try to reproduce the result on a Margaret node?

Hi, I encounter the same issue in an ubuntu 20.04 machine:

• python 3.9.16 h2782a2a_0_cpython conda-forge
• pytorch 1.13.1 py3.9_cuda11.7_cudnn8.5.0_0 pytorch
• joblib 1.3.0 pyhd8ed1ab_1 conda-forge

The issue occurs with my own code, and I can also reproduce the issue with @aperezlebel 's code.

Hi, same here.

• python 3.8.17
• torch 1.12.1
• joblib 1.3.2

Importing joblib before torch can resolve it.

@tomMoral Thank you for this ref. It is useful.

The llvm-openmp version is 16.0.6, and I can set KMP_AFFINITY=disabled to make it run well as well.

pytorch/pytorch#99625 (comment)

Let's close it looks like there is a work-around for this.