ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15069

pv · 2019-12-07T13:03:59Z

Generalize the ILP64 BLAS/LAPACK symbol name handling to deal with
arbitrary prefix/suffix.

The build-time behavior is changed so that HAVE_BLAS_ILP64 and
BLAS_SYMBOL_SUFFIX/PREFIX defines are added to compile options
as appropriate.

Mainly to make autodetection of BLAS/LAPACK easier for downstream
numpy.distutils users, add get_info aliases 'blas_ilp64_opt',
'blas_ilp64_plain_opt', and 'blas64__opt' for any/no/""&"64_"
prefix&suffix, and the same for lapack. (Due to the way system_info
works, each also gets a separate class.)

In addition to openblas64_ which has a fixed suffix, add the (by default
suffixless) openblas_ilp64, which correspond to the two most likely cases to
be present.

Moreover: add ilp64 blas support to *_dot, *_vdot, *_matmul, which
were missed in the previous pass (sorry...)

Similar thing could be easily done for other BLAS libraries in
system_info.py. Users can also set CFLAGS="-DHAVE_BLAS_ILP64"
to turn the 64-bit ABI on for any BLAS/LAPACK (however, I'd not advertise
that as the "public" way to turn this stuff on).

pv · 2019-12-07T21:27:50Z

Ok, not ready to merge yet --- needs still to think whether this should be get_info('blas_ilp64_opt') and not get_info('blas_opt'). Doesn't matter for numpy itself, but downstream might want to link to both 64-bit and 32-bit blas in different extension modules and stick to use cases where this works.

pv · 2019-12-11T20:13:10Z

Ok, I think this is now converged, ready for review.

tylerjereddy

I tested this locally on a machine with 128 GB phys mem and it did work after an initial failure, and I needed to change something in the feature branch so I will at least note that in my review---whether you think it warrants an additional comment is up to you I guess:

Install 64-bit OpenBLAS with no symbol adjustment using spack with gcc compiler: spack install openblas@0.3.7+ilp64 %gcc@7.4.0
Adjust site.cfg to point to that particular OpenBLAS:

[openblas_ilp64]
libraries = openblas64
library_dirs = /home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/openblas-0.3.7-arju45l32hlynt7mpqlnwmcgthh2fm5u/lib
include_dirs = /home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/openblas-0.3.7-arju45l32hlynt7mpqlnwmcgthh2fm5u/include
runtime_library_dirs = /home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/openblas-0.3.7-arju45l32hlynt7mpqlnwmcgthh2fm5u/lib
symbol_prefix =
symbol_suffix =

Run the test for 64-bit integer support: NPY_USE_BLAS_ILP64=1 NPY_BLAS_ILP64_ORDER=openblas_ilp64 NPY_LAPACK_ILP64_ORDER=openblas_ilp64 python runtests.py --mode=full -t "numpy/linalg/tests/test_linalg.py::test_blas64_dot" -- -rsx
Fails with libraries openblas64 not found in ['/home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/openblas-0.3.7-arju45l32hlynt7mpqlnwmcgthh2fm5u/lib']
Inspect contents of that path and see these files: cmake libopenblas.a libopenblas_haswell-r0.3.7.a libopenblas_haswell-r0.3.7.so libopenblas.so libopenblas.so.0 pkgconfig
Modify site.cfg to look for openblas shared library without the 64 suffix (I'm talking about the library name, not the symbols).
Success:

Building, see build.log...
Build OK
NumPy version 1.19.0.dev0+761e690
NumPy relaxed strides checking option: True
.                                                                                                                                                                                                                                                                               [100%]
1 passed in 17.16s

site.cfg.example

tylerjereddy · 2019-12-13T23:53:22Z

Oh, one more comment here---I realized that the CI doesn't currently have access to ILP64 OpenBLAS (until we merge your upstream PR on openblas-libs). I checked locally, and there was 1 error in the full test suite using this feature branch with ILP64 OpenBLAS---I'll place the large traceback for that below the fold for reviewers to inspect. Note that I could not reproduce the error on latest master when using "normal" OpenBLAS, so it is perhaps a little suspicious.

The test invocation was: NPY_USE_BLAS_ILP64=1 NPY_BLAS_ILP64_ORDER=openblas_ilp64 NPY_LAPACK_ILP64_ORDER=openblas_ilp64 python runtests.py --mode=full -- -n 20 -rsx

[gw13] linux -- Python 3.8.0 /home/tyler/python38_venv/bin/python

self = <numpy.f2py.tests.test_crackfortran.TestNoSpace object at 0x7f99ddfd2ac0>

    def setup(self):
        if sys.platform == 'win32':
            pytest.skip('Fails with MinGW64 Gfortran (Issue #9673)')
    
        if self.module is not None:
            return
    
        # Check compiler availability first
        if not has_c_compiler():
            pytest.skip("No C compiler available")
    
        codes = []
        if self.sources:
            codes.extend(self.sources)
        if self.code is not None:
            codes.append(self.suffix)
    
        needs_f77 = False
        needs_f90 = False
        for fn in codes:
            if fn.endswith('.f'):
                needs_f77 = True
            elif fn.endswith('.f90'):
                needs_f90 = True
        if needs_f77 and not has_f77_compiler():
            pytest.skip("No Fortran 77 compiler available")
        if needs_f90 and not has_f90_compiler():
            pytest.skip("No Fortran 90 compiler available")
    
        # Build the module
        if self.code is not None:
>           self.module = build_code(self.code, options=self.options,
                                     skip=self.skip, only=self.only,
                                     suffix=self.suffix,
                                     module_name=self.module_name)

codes      = ['.f']
fn         = '.f'
needs_f77  = True
needs_f90  = False
self       = <numpy.f2py.tests.test_crackfortran.TestNoSpace object at 0x7f99ddfd2ac0>

numpy/f2py/tests/util.py:354: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
numpy/f2py/tests/util.py:80: in wrapper
    memo[key] = func(*a, **kw)
numpy/f2py/tests/util.py:166: in build_code
    return build_module([path], options=options, skip=skip, only=only,
numpy/f2py/tests/util.py:80: in wrapper
    memo[key] = func(*a, **kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

source_files = ['/tmp/tmpzh78e64b.f'], options = [], skip = [], only = [], module_name = '_test_ext_module_5407'

    @_memoize
    def build_module(source_files, options=[], skip=[], only=[], module_name=None):
        """
        Compile and import a f2py module, built from the given files.
    
        """
    
        code = ("import sys; sys.path = %s; import numpy.f2py as f2py2e; "
                "f2py2e.main()" % repr(sys.path))
    
        d = get_module_dir()
    
        # Copy files
        dst_sources = []
        for fn in source_files:
            if not os.path.isfile(fn):
                raise RuntimeError("%s is not a file" % fn)
            dst = os.path.join(d, os.path.basename(fn))
            shutil.copyfile(fn, dst)
            dst_sources.append(dst)
    
            fn = os.path.join(os.path.dirname(fn), '.f2py_f2cmap')
            if os.path.isfile(fn):
                dst = os.path.join(d, os.path.basename(fn))
                if not os.path.isfile(dst):
                    shutil.copyfile(fn, dst)
    
        # Prepare options
        if module_name is None:
            module_name = get_temp_module_name()
        f2py_opts = ['-c', '-m', module_name] + options + dst_sources
        if skip:
            f2py_opts += ['skip:'] + skip
        if only:
            f2py_opts += ['only:'] + only
    
        # Build
        cwd = os.getcwd()
        try:
            os.chdir(d)
            cmd = [sys.executable, '-c', code] + f2py_opts
            p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
                                 stderr=subprocess.STDOUT)
            out, err = p.communicate()
            if p.returncode != 0:
>               raise RuntimeError("Running f2py failed: %s\n%s"
                                   % (cmd[4:], asstr(out)))
E                                  RuntimeError: Running f2py failed: ['-m', '_test_ext_module_5407', '/tmp/tmpxuyaxe68/tmpzh78e64b.f']
E                                  running build
E                                  running config_cc
E                                  unifing config_cc, config, build_clib, build_ext, build commands --compiler options
E                                  running config_fc
E                                  unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
E                                  running build_src
E                                  build_src
E                                  building extension "_test_ext_module_5407" sources
E                                  f2py options: []
E                                  f2py:> /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407module.c
E                                  creating /tmp/tmpsvy77mcz/src.linux-x86_64-3.8
E                                  Reading fortran codes...
E                                  	Reading file '/tmp/tmpxuyaxe68/tmpzh78e64b.f' (format:fix,strict)
E                                  crackline: groupcounter=5 groupname={0: '', 1: 'module', 2: 'interface', 3: 'subroutine', 4: 'subroutine', 5: 'function'}
E                                  crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.
E                                  crackline: groupcounter=4 groupname={0: '', 1: 'module', 2: 'interface', 3: 'subroutine', 4: 'subroutine', 5: 'function'}
E                                  crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.
E                                  crackline: groupcounter=3 groupname={0: '', 1: 'module', 2: 'interface', 3: 'subroutine', 4: 'subroutine', 5: 'function'}
E                                  crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.
E                                  crackline: groupcounter=2 groupname={0: '', 1: 'module', 2: 'interface', 3: 'subroutine', 4: 'subroutine', 5: 'function'}
E                                  crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.
E                                  crackline: groupcounter=1 groupname={0: '', 1: 'module', 2: 'interface', 3: 'subroutine', 4: 'subroutine', 5: 'function'}
E                                  crackline: Mismatch of blocks encountered. Trying to fix it by assuming "end" statement.
E                                  Post-processing...
E                                  	Block: _test_ext_module_5407
E                                  			Block: subb
E                                  Post-processing (stage 2)...
E                                  Building modules...
E                                  	Building module "_test_ext_module_5407"...
E                                  		Creating wrapper for Fortran subroutine "subb"("subb")...
E                                  		Constructing wrapper function "subb"...
E                                  		  subb(k)
E                                  	Wrote C/API module "_test_ext_module_5407" to file "/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407module.c"
E                                  	Fortran 77 wrappers are saved to "/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f"
E                                    adding '/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/fortranobject.c' to sources.
E                                    adding '/tmp/tmpsvy77mcz/src.linux-x86_64-3.8' to include_dirs.
E                                  copying /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c -> /tmp/tmpsvy77mcz/src.linux-x86_64-3.8
E                                  copying /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.h -> /tmp/tmpsvy77mcz/src.linux-x86_64-3.8
E                                    adding '/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f' to sources.
E                                  build_src: building npy-pkg config files
E                                  running build_ext
E                                  customize UnixCCompiler
E                                  customize UnixCCompiler using build_ext
E                                  get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']'
E                                  customize Gnu95FCompiler
E                                  Found executable /usr/bin/gfortran
E                                  customize Gnu95FCompiler
E                                  customize Gnu95FCompiler using build_ext
E                                  building '_test_ext_module_5407' extension
E                                  compiling C sources
E                                  C compiler: /usr/bin/gcc-7 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -fPIC -std=c99
E                                  
E                                  creating /tmp/tmpsvy77mcz/tmp
E                                  creating /tmp/tmpsvy77mcz/tmp/tmpsvy77mcz
E                                  creating /tmp/tmpsvy77mcz/tmp/tmpsvy77mcz/src.linux-x86_64-3.8
E                                  compile options: '-I/tmp/tmpsvy77mcz/src.linux-x86_64-3.8 -I/home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include -I/home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/python-3.8.0-fx5rz4ap2eson7qduk5y5mknaenon2e4/include/python3.8 -c'
E                                  gcc-7: /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407module.c
E                                  gcc-7: /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/fortranobject.c
E                                  In file included from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832:0,
E                                                   from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
E                                                   from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
E                                                   from /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/fortranobject.h:13,
E                                                   from /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/fortranobject.c:2:
E                                  /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
E                                   #warning "Using deprecated NumPy API, disable it with " \
E                                    ^~~~~~~
E                                  In file included from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832:0,
E                                                   from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
E                                                   from /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
E                                                   from /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/fortranobject.h:13,
E                                                   from /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407module.c:16:
E                                  /home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
E                                   #warning "Using deprecated NumPy API, disable it with " \
E                                    ^~~~~~~
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407module.c:137:12: warning: â��f2py_sizeâ�� defined but not used [-Wunused-function]
E                                   static int f2py_size(PyArrayObject* var, ...)
E                                              ^~~~~~~~~
E                                  compiling Fortran sources
E                                  Fortran f77 compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
E                                  Fortran f90 compiler: /usr/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
E                                  Fortran fix compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
E                                  creating /tmp/tmpsvy77mcz/tmp/tmpxuyaxe68
E                                  compile options: '-I/tmp/tmpsvy77mcz/src.linux-x86_64-3.8 -I/home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include -I/home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/python-3.8.0-fx5rz4ap2eson7qduk5y5mknaenon2e4/include/python3.8 -c'
E                                  gfortran:f77: /tmp/tmpxuyaxe68/tmpzh78e64b.f
E                                  gfortran:f77: /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:11:10:
E                                  
E                                             subroutine subc(w,k) ! in :_test_ext_module_5407:/tmp/tm
E                                            1
E                                  Error: Unclassifiable statement at (1)
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:14:63:
E                                  
E                                                 real(kind=8), intent(out),dimension(size(w)) :: k
E                                                                                                 1
E                                  Error: Symbol â��kâ�� at (1) already has basic type of REAL
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:15:14:
E                                  
E                                                 function t0(value) ! in :_test_ext_module_5407:/tmp/
E                                                1
E                                  Error: Unclassifiable statement at (1)
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:19:17:
E                                  
E                                                 end function t0
E                                                   1
E                                  Error: Expecting END SUBROUTINE statement at (1)
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:20:29:
E                                  
E                                             end subroutine subc
E                                                               1
E                                  Error: Expected label â��subbâ�� for END SUBROUTINE statement at (1)
E                                  /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f:13:56:
E                                  
E                                                 real(kind=8), intent(in),dimension(:) :: w
E                                                                                          1
E                                  Error: Symbol at (1) is not a DUMMY variable
E                                  error: Command "/usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops -I/tmp/tmpsvy77mcz/src.linux-x86_64-3.8 -I/home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages/numpy/core/include -I/home/tyler/github_projects/spack/opt/spack/linux-ubuntu18.04-skylake_avx512/gcc-7.4.0/python-3.8.0-fx5rz4ap2eson7qduk5y5mknaenon2e4/include/python3.8 -c -c /tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.f -o /tmp/tmpsvy77mcz/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.o" failed with exit status 1

cmd        = ['/home/tyler/python38_venv/bin/python', '-c', "import sys; sys.path = ['/tmp/tmpxuyaxe68', '/home/tyler/github_projec...v/lib/python3.8/site-packages']; import numpy.f2py as f2py2e; f2py2e.main()", '-c', '-m', '_test_ext_module_5407', ...]
code       = "import sys; sys.path = ['/tmp/tmpxuyaxe68', '/home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packa...inux-x86_64.egg', '/home/tyler/python38_venv/lib/python3.8/site-packages']; import numpy.f2py as f2py2e; f2py2e.main()"
cwd        = '/home/tyler/github_projects/numpy/build/testenv/lib/python3.8/site-packages'
d          = '/tmp/tmpxuyaxe68'
dst        = '/tmp/tmpxuyaxe68/tmpzh78e64b.f'
dst_sources = ['/tmp/tmpxuyaxe68/tmpzh78e64b.f']
err        = None
f2py_opts  = ['-c', '-m', '_test_ext_module_5407', '/tmp/tmpxuyaxe68/tmpzh78e64b.f']
fn         = '/tmp/tmpxuyaxe68/tmpzh78e64b.f'
module_name = '_test_ext_module_5407'
only       = []
options    = []
out        = b'running build\nrunning config_cc\nunifing config_cc, config, build_clib, build_ext, build commands --compiler option...tmp/tmpsvy77mcz/tmp/tmpsvy77mcz/src.linux-x86_64-3.8/_test_ext_module_5407-f2pywrappers.o" failed with exit status 1\n'
p          = <subprocess.Popen object at 0x7f99ddfd23a0>
skip       = []
source_files = ['/tmp/tmpzh78e64b.f']

numpy/f2py/tests/util.py:141: RuntimeError

pv · 2019-12-14T11:09:18Z

Quite hard to see how that could be connected to the changes here --- since this is not touching f2py at all, and that f2py test is not using lapack. It also doesn't repro here.

Generalize the ILP64 BLAS/LAPACK symbol name handling to deal with arbitrary prefix/suffix. The build-time behavior is changed so that HAVE_BLAS_ILP64 and BLAS_SYMBOL_SUFFIX/PREFIX defines are added to compile options as appropriate. Mainly to make autodetection of BLAS/LAPACK easier for downstream numpy.distutils users, add get_info aliases 'blas_ilp64_opt', 'blas_ilp64_plain_opt', and 'blas64__opt' for any/no/""&"64_" prefix&suffix, and the same for lapack. (Due to the way system_info works, each also gets a separate class.) In addition to openblas64_ which has a fixed suffix, add the (by default suffixless) openblas_ilp64, which correspond to the most likely cases to be present.

Revise the BLAS name mangling to support the general scheme.

Changing these to support ILP64 blas was missed in numpygh-15012

pv · 2019-12-14T12:15:04Z

The test_crackfortran.py where the failing test is doesn't actually exist in this branch (it was added in a later PR, together with an associated f2py bugfix). So maybe the problem was some leftover file? Anyway, rebased on top of master.

tylerjereddy

Ok, the 64-bit test is passing on two different machines for me now (one WSL, the other native Linux) with this feature branch.

The full test suite is now also okay for me on both machines (including the one previously complaining about f2py) with the ILP64 OpenBLAS linked in.

mattip · 2019-12-14T15:48:04Z

I think the 1.18 release note now needs a tweak to mention support for non-suffixed 64-bit OpenBLAS.

mattip · 2019-12-14T15:50:47Z

doc/source/user/building.rst

+
+The order in which they are preferred is determined by
+``NPY_BLAS_ILP64_ORDER`` and ``NPY_LAPACK_ILP64_ORDER`` environment
+variables, similarly as above.


nit: maybe "similarly as above" -> "which is openblas64_,openblas_ilp64 by default"

Fixed. (also the release note).

mattip · 2019-12-14T21:45:00Z

tests are failing because pip cannot find cython==0.29.14, but I see it on PyPI and the same tests passed earlier.

charris · 2019-12-15T20:25:34Z

Thanks Pauli.

eric-wieser · 2020-08-20T09:39:09Z

numpy/core/src/multiarray/common.h

        if (stride <= INT_MAX) {
+#else
+        if (stride <= NPY_MAX_INT64) {
+#endif
            return stride;


xref gh-17111, the return type here is int, but we're comparing against the bounds for int64

tylerjereddy added 01 - Enhancement component: numpy.distutils labels Dec 7, 2019

pv added component: numpy.linalg component: numpy._core labels Dec 7, 2019

pv changed the title ~~ENH: add support for ILP64 OpenBLAS (without symbol suffix)~~ WIP: ENH: add support for ILP64 OpenBLAS (without symbol suffix) Dec 7, 2019

pv force-pushed the blas-ilp64 branch 5 times, most recently from cf64ab2 to 761e690 Compare December 9, 2019 19:28

pv mentioned this pull request Dec 9, 2019

ENH: linalg: 64-bit BLAS/LAPACK scipy/scipy#11193

Draft

7 tasks

pv changed the title ~~WIP: ENH: add support for ILP64 OpenBLAS (without symbol suffix)~~ ENH: add support for ILP64 OpenBLAS (without symbol suffix) Dec 11, 2019

pv mentioned this pull request Dec 12, 2019

Matlab/NumPy BLAS incompatibility #15049

Closed

tylerjereddy reviewed Dec 13, 2019

View reviewed changes

site.cfg.example Show resolved Hide resolved

pv added 5 commits December 14, 2019 14:14

ENH: update BLAS symbol suffix/prefix handling in cblasfuncs & linalg

669cd13

Revise the BLAS name mangling to support the general scheme.

DOC: document 64-bit BLAS/LAPACK build-time variables

b7f42ea

BUG: core: use blas_ilp64 also for *_matmul, *_dot, and *_vdot

de8a10d

Changing these to support ILP64 blas was missed in numpygh-15012

DOC: site.cfg: add note about 64-bit openblas library names

d43645a

pv force-pushed the blas-ilp64 branch from 761e690 to d43645a Compare December 14, 2019 12:14

tylerjereddy approved these changes Dec 14, 2019

View reviewed changes

charris added the 09 - Backport-Candidate PRs tagged should be backported label Dec 14, 2019

charris added this to the 1.18.0 release milestone Dec 14, 2019

mattip reviewed Dec 14, 2019

View reviewed changes

pv added 2 commits December 14, 2019 18:46

DOC: adjust ILP64 openblas release note

3814538

DOC: explain NPY_*_ILP64_ORDER default values

85447b1

charris merged commit de06954 into numpy:master Dec 15, 2019

charris mentioned this pull request Dec 15, 2019

ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15114

Merged

charris removed the 09 - Backport-Candidate PRs tagged should be backported label Dec 15, 2019

charris removed this from the 1.18.0 release milestone Dec 15, 2019

eric-wieser mentioned this pull request Aug 20, 2020

blas_stride has the wrong return type with HAVE_BLAS_ILP64 #17111

Closed

eric-wieser reviewed Aug 20, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15069

ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15069

Uh oh!

pv commented Dec 7, 2019 •

edited

Loading

Uh oh!

pv commented Dec 7, 2019 •

edited

Loading

Uh oh!

pv commented Dec 11, 2019

Uh oh!

tylerjereddy left a comment

Uh oh!

Uh oh!

tylerjereddy commented Dec 13, 2019

Uh oh!

pv commented Dec 14, 2019 •

edited

Loading

Uh oh!

pv commented Dec 14, 2019 •

edited

Loading

Uh oh!

tylerjereddy left a comment

Uh oh!

mattip commented Dec 14, 2019

Uh oh!

mattip Dec 14, 2019

Uh oh!

pv Dec 14, 2019 •

edited

Loading

Uh oh!

mattip commented Dec 14, 2019

Uh oh!

charris commented Dec 15, 2019

Uh oh!

eric-wieser Aug 20, 2020

Uh oh!

Uh oh!

Uh oh!

ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15069

ENH: add support for ILP64 OpenBLAS (without symbol suffix) #15069

Uh oh!

Conversation

pv commented Dec 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pv commented Dec 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pv commented Dec 11, 2019

Uh oh!

tylerjereddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tylerjereddy commented Dec 13, 2019

Uh oh!

pv commented Dec 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pv commented Dec 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tylerjereddy left a comment

Choose a reason for hiding this comment

Uh oh!

mattip commented Dec 14, 2019

Uh oh!

mattip Dec 14, 2019

Choose a reason for hiding this comment

Uh oh!

pv Dec 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattip commented Dec 14, 2019

Uh oh!

charris commented Dec 15, 2019

Uh oh!

eric-wieser Aug 20, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pv commented Dec 7, 2019 •

edited

Loading

pv commented Dec 7, 2019 •

edited

Loading

pv commented Dec 14, 2019 •

edited

Loading

pv commented Dec 14, 2019 •

edited

Loading

pv Dec 14, 2019 •

edited

Loading