I've been using Joblib for a while now and it's been incredibly useful! I've recently switched to an M1 Macbook and have found some issues and wondering if anyone has seen.

Running the following with Loky backend (default) results in hanging processes, and eventually uses all available RAM on my machine, requiring a force reboot:

from math import sqrt

from joblib import Parallel, delayed

out = Parallel(n_jobs=2)(delayed(sqrt)(i**2) for i in range(10))
print(out)

I have found that switching to backend="multiprocessing", and moving all Parallel calls to an if __name__ == "__main__" block resolves, but was wondering if anyone has seen this! These problems seem exclusive to the M1 Macbooks and I cannot reproduce on Linux.

from math import sqrt

from joblib import Parallel, delayed

if __name__ == "__main__":
    out = Parallel(n_jobs=2, backend="multiprocessing")(
        delayed(sqrt)(i**2) for i in range(10)
    )
    print(out)