@matthew-brett - i can check this against mkl later tonight, but if you download miniconda and then conda install accelerate, you should be able to as well. it will give you a 30 day trial.

Alexis - I changed the priority of the setup file and env var - OK now?

Satra - thanks - please do get back to me if you do get time to test.

@matthew-brett - current master and this branch still break with mkl. the problem is that numpy's get lapack_info picks up the mkl link library that is only available for developing with mkl. i've posted an issue with continuum.

ContinuumIO/anaconda-issues#431

i'll probably still update my pull-request/branch for those who want to install nipy in mkl environments.

Satra - do you mean that you get a build error when you set external lapack linking on, with the environment variable or the setup.cfg file? I assume nipy builds correctly in the default state (no env var or edits to setup.cfg)?

@matthew-brett
I see that at line 74 of nipy/labs/setup.py and onwards, you are calling numpy's lapack info routine, while it's not needed if the user does not want system lapack linking. This might cause the crash reported by @satra . I think you want to embed that call in the loop further down. Sorry for not spotting that yesterday.

Surely calling get_info should not cause a crash if lapack is not found?

Satra - what error are you getting?

@matthew-brett

in the mkl setting there is an external lapack available as far as numpy is concerned, i.e. mkl. and then it picks up libraries as shown in the continuum issue that cannot be linked against (see error below).

i believe @alexis-roche is correct in that if we are to rely on internal lapack, numpy's lapack-info routine should only be called when external lapack or site.cfg says so.

gcc -pthread -shared -L/om/user/satra/miniconda/lib -Wl,-rpath=/om/user/satra/miniconda/lib,--no-as-needed build/temp.linux-x86_64-2.7/nipy/labs/bindings/linalg.o -L/om/user/satra/miniconda/lib -L/om/user/satra/miniconda/lib -Lbuild/temp.linux-x86_64-2.7 -lcstat -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lpython2.7 -o build/lib.linux-x86_64-2.7/nipy/labs/bindings/linalg.so
/usr/bin/ld: cannot find -lmkl_lapack95_lp64
collect2: error: ld returned 1 exit status
/usr/bin/ld: cannot find -lmkl_lapack95_lp64
collect2: error: ld returned 1 exit status
error: Command "gcc -pthread -shared -L/om/user/satra/miniconda/lib -Wl,-rpath=/om/user/satra/miniconda/lib,--no-as-needed build/temp.linux-x86_64-2.7/nipy/labs/bindings/linalg.o -L/om/user/satra/miniconda/lib -L/om/user/satra/miniconda/lib -Lbuild/temp.linux-x86_64-2.7 -lcstat -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lpython2.7 -o build/lib.linux-x86_64-2.7/nipy/labs/bindings/linalg.so" failed with exit status 1

My impression is that, whenever numpy's get lapack info is called, the external library linking options are passed on to gcc, resulting in a crash at compilation time. It may not be a problem of python not finding the libs, but rather one of passing undesired options to gcc. Am I correct?

Satra - are you explicitly asking to link against external lapack here, with the environment variable or nipy setup.cfg file? As far as I can see, the lapack libraries and includes do not get added to the link step if you don't do that. If you do do that, it seems reasonable that this will fail at the link step, if the libraries are not available.

@matthew-brett - i was using default master or this branch. both had the exact same effect. the default site.cfg was used.

[lapack]
# Value of 0 or False implies compile of, link to lapack_lite
# Value of 1 or True will cause setup to try and link to external BLAS /
# LAPACK as identified with the numpy configuration.  Default is False.
# The value in this file overrides the equivalent setting in the environment
# variable NIPY_EXTERNAL_LAPACK.
#external = False

these are all the places get_info is being called and that would have added those libraries.

./algorithms/clustering/setup.py:
   12 :     from numpy.distutils.system_info import get_info
   13 :     lapack_info = get_info('lapack_opt',0)
   17 :         lapack_info = get_info('lapack',0)
./labs/bindings/setup.py:
   12 :     from numpy.distutils.system_info import get_info
   13 :     lapack_info = get_info('lapack_opt',0)
   17 :         lapack_info = get_info('lapack',0)
./labs/glm/setup.py:
    8 :     from numpy.distutils.system_info import get_info
    9 :     lapack_info = get_info('lapack_opt', 0)
   13 :         lapack_info = get_info('lapack', 0)
./labs/group/setup.py:
    6 :     from numpy.distutils.system_info import get_info
    9 :     lapack_info = get_info('lapack_opt', 0)
   13 :         lapack_info = get_info('lapack',0)
./labs/setup.py:
   47 :     from numpy.distutils.system_info import get_info, system_info
   61 :     # XXX: We need to better sort out the use of get_info() for Lapack, because
   64 :     # get_info('lapack_opt') does not return the keys: 'libraries' and
   65 :     # 'library_dirs', but get_info('lapack') does.
   74 :     lapack_info = get_info('lapack_opt', 0)
   78 :         lapack_info = get_info('lapack', 0)
./labs/utils/setup.py:
    8 :     from numpy.distutils.system_info import get_info
    9 :     lapack_info = get_info('lapack_opt', 0)
   13 :         lapack_info = get_info('lapack', 0)

My memory is failing me, but reading again the discussion about #326, I see that I decided to remove any call to get_info to make it compile with MKL, which led to #328. Unfortunately, in the later PR, I also removed the option to compile via lapack_lite.

I'm just trying, with stripping all lapack references from the setup files (apart from the root nipy/labs one). Compiles / runs OK on my OSX machine, trying now on travis : https://travis-ci.org/matthew-brett/nipy/builds/78460738

Those compiles also look OK, so I'm adding these edits to this PR.

Log for non-external compile : https://s3.amazonaws.com/archive.travis-ci.org/jobs/78460742/log.txt

Log for external lapack compile : https://s3.amazonaws.com/archive.travis-ci.org/jobs/78460745/log.txt

Satra - does it work now? If not, can you post the full log somewhere?

@matthew-brett - this branch works for me now even with MKL, with the default site.cfg and no environmental variable specification - thank you.