Remove lapack linking #328

alexis-roche · 2014-11-11T12:02:11Z

This PR implements an alternate strategy to bind BLAS/LAPACK routines based on importing functions at run time via C-pointers provided by scipy. I removed any attempt to link BLAS/Lapack libraries at build time, even using source code (the lapack-lite distribution therefore becomes obsolete).

Because not all the BLAS/LAPACK functions are available through scipy, I had to slightly rewrite some parts of the fff library and the labs package. The tests are however unchanged and all pass on my box.

matthew-brett · 2014-11-11T18:36:42Z

Alexis - would you mind rebasing this one on master?

satra · 2014-11-11T18:55:39Z

@matthew-brett - unfortunately, i don't have any machines to offer. i live between my laptop and our compute clusters. but the following steps would setup anaconda with mkl (accelerate, which provides mkl, does require a license that is free for academic use, but also provides a 30 day trial period which allows using things like travis for unit-testing).

wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh
chmod +x miniconda.sh
./miniconda.sh -b
echo "export PATH=$HOME/miniconda/bin:\\$PATH" >> .bashrc
$HOME/miniconda/bin/conda update --yes conda
$HOME/miniconda/bin/conda update --yes accelerate # installs scipy, numpy, etc.,. with mkl support

(there are other equivalent steps for os x and windows).

mwaskom · 2014-11-11T19:57:25Z

This appears to install correctly and nipy is now importable on my system. But the tests fail with 22 errors and 9 failures (though it looks like Travis is seeing those too).

bthirion · 2015-03-26T22:52:23Z

Removing the lapack linking is really great. However, I also get the errors on my box.

matthew-brett · 2015-03-26T22:54:21Z

Sorry guys - I have dropped the ball on this one. I will get to it tomorrow.

matthew-brett · 2015-03-28T00:23:41Z

Alexis - I can't get this to compile on OSX:

37 warnings generated.
clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o -Lbuild/temp.macosx-10.6-intel-2.7 -lcstat -o nipy/labs/bindings/linalg.so
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_array.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_base.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_blas.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_matrix.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_vector.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fffpy.o)
ld: 6 duplicate symbols for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_array.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_base.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_blas.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_matrix.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fff_vector.o)
duplicate symbol _FFF_EXTERNAL_FUNC in:
    build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o
    build/temp.macosx-10.6-intel-2.7/libcstat.a(fffpy.o)
ld: 6 duplicate symbols for architecture i386
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: Command "clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-2.7/nipy/labs/bindings/linalg.o -Lbuild/temp.macosx-10.6-intel-2.7 -lcstat -o nipy/labs/bindings/linalg.so" failed with exit status 1

Shall I give you a login on my desktop so you can debug?

matthew-brett · 2015-03-28T00:30:25Z

On Linux I am getting this:

In file included from nipy/labs/utils/routines.c:359:0:
/home/mb312/dev_trees/nipy/lib/fff/fff_gen_stats.h:34:17: note: expected ‘struct fff_vector *’ but argument is of type ‘struct fff_matrix *’
   extern double fff_mahalanobis(fff_vector* x, fff_matrix* S, fff_vector* xaux, fff_vector* Saux);
                 ^
nipy/labs/utils/routines.c:2603:29: error: too few arguments to function ‘fff_mahalanobis’
     (__pyx_v_d2->data[0]) = fff_mahalanobis(__pyx_v_x_tmp, (&__pyx_v_Sx), __pyx_v_Sx_tmp);
                             ^
In file included from nipy/labs/utils/routines.c:359:0:
/home/mb312/dev_trees/nipy/lib/fff/fff_gen_stats.h:34:17: note: declared here
   extern double fff_mahalanobis(fff_vector* x, fff_matrix* S, fff_vector* xaux, fff_vector* Saux);
                 ^
In file included from /usr/lib/python2.7/dist-packages/numpy/core/include/numpy/ufuncobject.h:327:0,
                 from nipy/labs/utils/routines.c:353:
nipy/labs/utils/routines.c: At top level:
/usr/lib/python2.7/dist-packages/numpy/core/include/numpy/__ufunc_api.h:241:1: warning: ‘_import_umath’ defined but not used [-Wunused-function]
 _import_umath(void)
 ^
error: Command "x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -Inipy/algorithms/registration -Inipy/algorithms/slicetiming -Inipy/algorithms/segmentation -Inipy/algorithms/statistics -I/home/mb312/dev_trees/nipy/lib/fff -I/home/mb312/dev_trees/nipy/lib/fff_python_wrapper -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -c nipy/labs/utils/routines.c -o build/temp.linux-x86_64-2.7/nipy/labs/utils/routines.o" failed with exit status 1

matthew-brett · 2015-03-28T00:32:53Z

The Linux errors are also on Travis : https://travis-ci.org/nipy/nipy/jobs/40747586

alexis-roche · 2015-03-28T06:59:18Z

Hi,
I will check this out on Monday. It's no longer fresh in my mind, so the issue is not obvious from your error messages.
Alexis

-------- Message d'origine --------
De : Matthew Brett notifications@github.com
Date : 28/03/2015 01:32 (GMT+01:00)
À : nipy/nipy nipy@noreply.github.com
Cc : Alexis Roche alexis.roche@gmail.com
Objet : Re: [nipy] Remove lapack linking (#328)

The Linux errors are also on Travis : https://travis-ci.org/nipy/nipy/jobs/40747586

—
Reply to this email directly or view it on GitHub.

alexis-roche · 2015-03-30T13:34:50Z

@matthew-brett @bthirion Let me know if that works for you now...

matthew-brett · 2015-03-30T16:51:21Z

Yes, working for me on OSX now, and I see it's building again on travis.

Would you mind rebasing on master to get rid of the merge commits?

…y import of lapack functions using scipy

alexis-roche · 2015-05-21T10:11:37Z

Can I merge this?

bthirion · 2015-05-21T21:07:27Z

As far as I can see this is OK (current travis errors do not seem related to this PR).

MRG: Remove lapack linking This PR implements an alternate strategy to bind BLAS/LAPACK routines based on importing functions at run time via C-pointers provided by scipy. I removed any attempt to link BLAS/Lapack libraries at build time, even using source code (the lapack-lite distribution therefore becomes obsolete). Because not all the BLAS/LAPACK functions are available through scipy, I had to slightly rewrite some parts of the fff library and the labs package. The tests are however unchanged and all pass on my box.

matthew-brett · 2015-06-14T00:27:12Z

Alexis - lots of errors on the buildbots - see http://nipy.bic.berkeley.edu/builders/nipy-py2.6-32/builds/269/steps/shell_6/logs/stdio - any thoughts?

alexis-roche · 2015-06-15T07:16:14Z

Hey Matthew, I just did a full re-install on my box and can't reproduce any of these bugs. I am running python 2.7.6, numpy 1.8.1 and scipy 0.13.3. The problem seems to be related with the failing of the following import on the buildbots:

import scipy.linalg._fblas

Don't know if this is a scipy compatibility issue, or a deeper installation/library linking problem.

…inking" This reverts commit d6b2e86, reversing changes made to 74b9529.

* reverting-lapack-refactor: MAINT: update .c files from modified .pyx files RF: refactor import_array as documented BF: recythonize routines.pyx to fix import error MAINT: fix cythonize Makefile target Revert "Merge pull request nipy#328 from alexis-roche/remove-lapack-linking"

matthew-brett · 2015-08-29T12:38:08Z

Some notes for my own understanding:

http://mail.scipy.org/pipermail/nipy-devel/2014-July/010066.html

http://mail.scipy.org/pipermail/scipy-user/2014-March/035561.html

matthew-brett · 2015-08-29T12:47:57Z

Another relevant note:

http://numpy-discussion.10968.n7.nabble.com/2012-Accessing-LAPACK-and-BLAS-from-the-numpy-C-API-td3480.html

Alexis - sorry to overwhelm you - but do you think there is any way of using the trick above in older Scipy versions? I think this is what Pauli is describing here:

http://mail.scipy.org/pipermail/scipy-user/2014-March/035562.html

matthew-brett · 2015-08-29T17:45:57Z

I think this will also work for earlier versions:

In [9]: import scipy.linalg.blas as slb
In [10]: func = getattr(slb.fblas, 'dgemm')

at least it works in scipy 0.9.0 ... But 'dsymm' is still missing before 0.13. Possible to work round?

MRG: revert lapack refactor for compatibility Lapack refactor increases our scipy dependency to 0.13 (see gh-344) and causes some build errors (see gh-342). Revert for now, so we can reconsider how best to fix up gh-328. As discussed in gh-344.

matthew-brett · 2015-08-31T09:23:39Z

Some explorations : https://gist.github.com/0f79c432b6f77d478d98

Output for scipy 0.13:

In [1]: import scipy

In [2]: scipy.__version__
Out[2]: '0.13.0'

In [3]: run ~/tmp/try_lapack.py
{'dasum': <fortran dasum>,
 'daxpy': <fortran object>,
 'dcopy': <fortran object>,
 'ddot': <fortran ddot>,
 'dgemm': <fortran object>,
 'dgemv': <fortran object>,
 'dger': <fortran object>,
 'dnrm2': <fortran dnrm2>,
 'drot': <fortran object>,
 'drotg': <fortran object>,
 'drotm': <fortran object>,
 'drotmg': <fortran object>,
 'dscal': <fortran object>,
 'dswap': <fortran object>,
 'dsymm': <fortran object>,
 'dsymv': <fortran object>,
 'dsyr': 'missing',
 'dsyr2': 'missing',
 'dsyr2k': <fortran object>,
 'dsyrk': <fortran object>,
 'dtrmm': 'missing',
 'dtrmv': <fortran object>,
 'dtrsm': 'missing',
 'dtrsv': 'missing',
 'idamax': <fortran object>}
{'dgeqrf': <fortran object>,
 'dgesdd': <fortran object>,
 'dgetrf': <fortran object>,
 'dpotrf': <fortran object>,
 'dpotrs': <fortran object>}

My Scipy 0.16 only has these missing:

 'dtrsm': 'missing',
 'dtrsv': 'missing',

I think this means that we'll run into trouble with missing routines on different scipy versions.

alexis-roche · 2015-08-31T10:00:15Z

Yes, that's one of the reasons why the most robust approach is probably to
use the lapack-lite C code, at least until the scipy API stabilizes... The
only real downside I see is a potential loss in performance (compared to
linking with Fortran-compiled system blas/lapack libs), but the few
comparison tests I ran in the past did not reveal significant computation
time or numerical differences in practice.

On Mon, Aug 31, 2015 at 11:23 AM, Matthew Brett notifications@github.com
wrote:

Some explorations : https://gist.github.com/0f79c432b6f77d478d98

Output for scipy 0.13:

In [1]: import scipy

In [2]: scipy.version
Out[2]: '0.13.0'

In [3]: run ~/tmp/try_lapack.py
{'dasum': ,
'daxpy': ,
'dcopy': ,
'ddot': ,
'dgemm': ,
'dgemv': ,
'dger': ,
'dnrm2': ,
'drot': ,
'drotg': ,
'drotm': ,
'drotmg': ,
'dscal': ,
'dswap': ,
'dsymm': ,
'dsymv': ,
'dsyr': 'missing',
'dsyr2': 'missing',
'dsyr2k': ,
'dsyrk': ,
'dtrmm': 'missing',
'dtrmv': ,
'dtrsm': 'missing',
'dtrsv': 'missing',
'idamax': }
{'dgeqrf': ,
'dgesdd': ,
'dgetrf': ,
'dpotrf': ,
'dpotrs': }

My Scipy 0.16 only has these missing:

'dtrsm': 'missing',
'dtrsv': 'missing',

I think this means that we'll run into trouble with missing routines on
different scipy versions.

—
Reply to this email directly or view it on GitHub
#328 (comment).

Lead Clinical Research
Advanced Clinical Imaging Technology
Siemens/CHUV/EPFL
1015 Lausanne, Switzerland
Phone: +41 21 545 9972
https://sites.google.com/site/alexisroche

alexis-roche mentioned this pull request Nov 11, 2014

Attempt to fix Anaconda compilation bug #326

Closed

alexis-roche force-pushed the remove-lapack-linking branch 3 times, most recently from d1c8fc0 to e185231 Compare November 12, 2014 08:03

alexis-roche force-pushed the remove-lapack-linking branch 4 times, most recently from 0ae078c to 67242ae Compare March 30, 2015 12:53

alexis-roche force-pushed the remove-lapack-linking branch from 67242ae to fae677e Compare March 31, 2015 07:02

alexis-roche added 3 commits May 21, 2015 12:09

remove linking with system lapack or lapack lite, implement on-the-fl…

8d26df7

…y import of lapack functions using scipy

fix wrong declaration in cython file

cbb35e5

attempt to fix OSX compilation error

02cf23b

alexis-roche force-pushed the remove-lapack-linking branch from fae677e to 02cf23b Compare May 21, 2015 10:09

matthew-brett merged commit d6b2e86 into nipy:master May 26, 2015

alexis-roche deleted the remove-lapack-linking branch June 8, 2015 13:19

matthew-brett mentioned this pull request Aug 25, 2015

Linking errors, probably from "remove lapack linking" branch #342

Open

matthew-brett added a commit to matthew-brett/nipy that referenced this pull request Aug 27, 2015

Revert "Merge pull request nipy#328 from alexis-roche/remove-lapack-l…

1af7466

…inking" This reverts commit d6b2e86, reversing changes made to 74b9529.

matthew-brett added a commit to matthew-brett/nipy that referenced this pull request Aug 27, 2015

Revert "Merge pull request nipy#328 from alexis-roche/remove-lapack-l…

5e64fcb

…inking" This reverts commit d6b2e86, reversing changes made to 74b9529.

matthew-brett mentioned this pull request Aug 27, 2015

MRG: temporarily revert lapack refactor for testing #346

Merged

matthew-brett added a commit to matthew-brett/nipy that referenced this pull request Aug 28, 2015

Revert "Merge pull request nipy#328 from alexis-roche/remove-lapack-l…

cf0f9ed

…inking" This reverts commit d6b2e86, reversing changes made to 74b9529.

matthew-brett added a commit to matthew-brett/nipy that referenced this pull request Aug 28, 2015

Revert "Merge pull request nipy#328 from alexis-roche/remove-lapack-l…

9d83fc3

…inking" This reverts commit d6b2e86, reversing changes made to 74b9529.

alexis-roche mentioned this pull request Sep 2, 2015

CONFIG: add config option for LAPACK compile #352

Merged

Remove lapack linking #328

Remove lapack linking #328

Conversation

alexis-roche commented Nov 11, 2014

Uh oh!

matthew-brett commented Nov 11, 2014

Uh oh!

satra commented Nov 11, 2014

Uh oh!

mwaskom commented Nov 11, 2014

Uh oh!

bthirion commented Mar 26, 2015

Uh oh!

matthew-brett commented Mar 26, 2015

Uh oh!

matthew-brett commented Mar 28, 2015

Uh oh!

matthew-brett commented Mar 28, 2015

Uh oh!

matthew-brett commented Mar 28, 2015

Uh oh!

alexis-roche commented Mar 28, 2015

Uh oh!

alexis-roche commented Mar 30, 2015

Uh oh!

matthew-brett commented Mar 30, 2015

Uh oh!

alexis-roche commented May 21, 2015

Uh oh!

bthirion commented May 21, 2015

Uh oh!

matthew-brett commented Jun 14, 2015

Uh oh!

alexis-roche commented Jun 15, 2015

Uh oh!

matthew-brett commented Aug 29, 2015

Uh oh!

matthew-brett commented Aug 29, 2015

Uh oh!

matthew-brett commented Aug 29, 2015

Uh oh!

matthew-brett commented Aug 31, 2015

Uh oh!

alexis-roche commented Aug 31, 2015

Uh oh!