@pv wrote on 2012-08-14

I cannot reproduce this (Ubuntu x86, OpenSUSE x86_64, both with Numpy 1.6.1).

trac user agchang wrote on 2012-08-14

Sorry, I forgot to mention something very relevant. This bug seems to occur only on specific hardware, i.e, Macbook Pro(Mid-2010). I ran this also on a non Macintosh machine and it was able to work. Also, I'm not sure how relevant this ticket it is, but it seems to have similar problems as well.

http://projects.scipy.org/numpy/ticket/2091

However, this person uses OSX so it is presumably on Macintosh hardware as well.

trac user agchang wrote on 2012-08-14

The machine that it works on(the non-Macintosh) has a kernel version of:

Linux byrd 3.2.0-27-generic #43-Ubuntu SMP Fri Jul 6 14:25:57 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

trac user agchang wrote on 2012-08-15

This also seems to run on Mac OSX Snow Leopard my machine as well.

@pv wrote on 2012-08-15

This is probably relevant:
http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html

If this is indeed the origin of this crash, then it's an Apple issue with multiprocessing, and not fixable by us (except by shipping binaries with Accelerate disabled, but that's a workaround).

trac user agchang wrote on 2012-08-22

Hm. Sounds like it. Would it still make sense that it works on Mac OSX Snow Leopard on my machine, but not Ubuntu? Maybe Ubuntu does utilize the Accelerate framework while Mac OSX does not, by default?

@pv wrote on 2012-08-22

I must have misread this bug report originally. Accelerate is Apple software that comes with OSX, so I don't think running Ubuntu on the hardware uses it. So this seems to be a different issue than #2683

I'm experiencing the same bug on Ubuntu 12.04 with NumPy 1.6.1 and Python 2.7.

I just upgraded to NumPy 1.6.2 and the problem has gone away.

I get the same issue on :

Ubuntu 13.04 running kernel version 3.8.0-30-generic
Python version = 2.7.4
numpy version = 1.7.1
multiprocessing version = 0.70a1

In an experiment very similar to the one reported in the bug description by @numpy-gitbot, the code hangs when executing the np.linalg.eig function (the same happens with the np.linalg.inv)

from multiprocessing        import Process as process

def eigen_values():
    import numpy as np
    print 'Test eig with multiprocessing...'
    data = np.random.normal(0, 1, (2,2))
    cov = np.cov(data, rowvar=0)

    # Execution hangs when executing the following
    l, w = np.linalg.eig(cov)

p = process(target=eigen_values)
p.start()
p.join()

Update : This seems to be an issue with OpenBLAS and multiprocessing. Setting an environment variable OPENBLAS_NUM_THREADS=1 is a workaround to this issue.

I got this from :
http://numpy-discussion.10968.n7.nabble.com/svd-multiprocessing-hangs-tc34252.html#none

I'm having the very same issue on Ubuntu 12.04 with NumPy 1.8.0, Python 2.7. and multiprocessing 0.70a1.

The following snippet (from above) does not terminate:

import numpy as np
import multiprocessing as mp

def foo(x):
    print np.linalg.inv([[2,3],[2,2]]) #this causes the crash
    #print np.dot([[1,2],[3,4]],[[1,2],[3,4]]) # this works fine

def test():
    print "running..."
    print np.__version__
    print mp.__version__
    vals = [1,2,3,4]
    pool = mp.Pool(1) #This has an issue
    #pool = mp.Pool(mp.cpu_count()) #this has an issue
    pool.map(foo, vals)

if __name__ == "__main__":
    test()
    #foo(1) #this works fine

EDIT:
In fact, setting OPENBLAS_NUM_THREADS=1 did help to solve the problem!

This should really be documented somewhere; I don't think it can be fixed by NumPy. The thing is that multiprocessing on a POSIX system is unreliable in the face of multithreaded libraries such as OpenBLAS, because POSIX does not allow fork without exec in a multithreaded context. Python 3.4 promises to solve this by a new feature in multiprocessing called the forkserver, but the problem is only solved when that is enabled explicitly at program startup.