Skip to content

Issue with f2py in Numpy 1.13 when character arrays appear in a callback function #10027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
swryan opened this issue Nov 15, 2017 · 4 comments
Closed

Issue with f2py in Numpy 1.13 when character arrays appear in a callback function #10027

swryan opened this issue Nov 15, 2017 · 4 comments
Labels
06 - Regression component: numpy.f2py

Comments

@swryan
Copy link

swryan commented Nov 15, 2017

There is an issue with f2py in Numpy 1.13 when character arrays appear in a callback function. The following example works with Numpy 1.12 but not in 1.13:

#! /usr/bin/env python

from __future__ import print_function
import numpy
from subprocess import call

#
# build extension from FORTRAN source
#
source = """
       subroutine func_with_callback(callBack, cu, lencu)

       implicit none

       integer :: lencu
       character*8 cu(lencu)
       external callBack

       print *,'Lencu:', lencu

       print *,'Calling callback...'
       call callBack(cu, lencu)
       print *,'Back in fortran...'

       end subroutine func_with_callback
"""

with open("callback.f", "w") as src_file:
    src_file.write(source)

rc = call("f2py callback.f -m callback -h callback.pyf".split())
if rc: quit()

rc = call("f2py -c callback.pyf callback.f".split())
if rc: quit()


#
# define callback function
#
def callBack(cu, lencu):
    print ('In callback:', cu)

#
# call library function with callback
#
from callback import func_with_callback
print(func_with_callback.__doc__)

print("\nNumpy Version:", numpy.__version__, "\n")

cu = numpy.zeros((1, 8), 'c')

func_with_callback(callBack, cu)

================

Numpy Version: 1.12.1

Lencu: 1
Calling callback...
In callback: [['' '' '' '' '' '' '' '']]
Back in fortran...

================

Numpy Version: 1.13.3

Lencu: 1
Calling callback...
Call-back cb_callback_in_func_with_callback__user__routines failed.
Traceback (most recent call last):
File "test_callback.py", line 57, in
func_with_callback(callBack, cu)
ValueError: data type must provide an itemsize
@charris
Copy link
Member

charris commented Nov 15, 2017 •
edited
Loading

The problem came in with a618b4e, #8948.

@charris charris mentioned this issue Nov 15, 2017
Merged
@charris charris added this to the 1.14.0 release milestone Nov 15, 2017
@charris
Copy link
Member

charris commented Nov 15, 2017

The proximate cause is this code in the generated source:

  if (cb_callBack_in_func_with_callback__user__routines_nofargs>capi_i) {
    PyArrayObject *tmp_arr = (PyArrayObject *)PyArray_New(&PyArray_Type,2,cu_Dims,NPY_STRING,NULL,(char*)cu,0,NPY_ARRAY_CARRAY,NULL); /*XXX: Hmm, what will destroy this array??? */

@charris charris mentioned this issue Nov 15, 2017
Merged
charris added a commit to charris/numpy that referenced this issue Nov 15, 2017
@charris
BUG: Fix f2py string variables in callbacks.
0a1069a 
When NPY_CHAR was deprecated and replaced by NPY_STRING in f2py, calls
to PyArray_New that previously relied on the type to get the itemsize
needed the size explicitly specified, but that modification was missed
in some of the code. Because the strings that replacee the 'c' type are
always 'S1', we use an itemsize of 1 for the string types and pass it
explicitly.

Closes numpy#10027.
charris added a commit to charris/numpy that referenced this issue Nov 16, 2017
@charris
TST: Test f2py passing string array to callback.
012ad9f 
See numpy#10027.
charris added a commit to charris/numpy that referenced this issue Nov 18, 2017
@charris
BUG: Fix f2py string variables in callbacks.
9a36ead 
When NPY_CHAR was deprecated and replaced by NPY_STRING in f2py, calls
to PyArray_New that previously relied on the type to get the itemsize
needed the size explicitly specified, but that modification was missed
in some of the code. Because the strings that replacee the 'c' type are
always 'S1', we use an itemsize of 1 for the string types and pass it
explicitly.

Closes numpy#10027.
@charris charris mentioned this issue Nov 18, 2017
Merged
charris added a commit to charris/numpy that referenced this issue Nov 18, 2017
@charris
TST: Test f2py passing string array to callback.
8825fac 
See numpy#10027.
@aelanman
Copy link

aelanman commented Dec 8, 2021

I've recently come across this issue while using f2py in numpy 1.20.3. As a minimum working example, I have the following simple fortran module which declares two allocatable arrays (charge is real type and names is character). The subroutine "main" will allocate them:

! -*- f90 -*-

      module inputs
      implicit none
              real*8, dimension(:,:), target, allocatable :: charge
              integer is_alloc
              character(len=10), allocatable, dimension(:) :: names
      end module inputs

      subroutine main(N)
      use inputs
      implicit none

      integer ii, jj, N
      
      IF (allocated(charge)) THEN
         deallocate(charge)
      end if
      allocate(charge(N, N))
      allocate(character(len=10) :: names(N)) !, source="TEST")

      END subroutine

I can build this by running python setup.py build_ext --inplace with the following setup.py file (having put the above code in test.f90):

from numpy.distutils.core import Extension
from numpy.distutils.core import setup
from numpy.distutils.misc_util import Configuration


F90_COMBINED = "test.f90"

calc_ext = Extension(
    name="alloc",
    sources=[F90_COMBINED],
    include_dirs=['.'],
    libraries=['gsl', 'gslcblas'],
    extra_f90_compile_args=['-Wargument-mismatch'],
)

def configuration(parent_package='', top_path=None):

    config = Configuration('', parent_package, top_path,
                           ext_modules=[calc_ext])
    return config

if __name__ == "__main__":
    setup(configuration=configuration)

This builds a module called alloc. Here's a screencap of what happens if I try to access each allocated array.
image

Any suggestions would be greatly appreciated!

@melissawm melissawm reopened this Dec 8, 2021
This was referenced Jun 5, 2022
Merged
Open
@HaoZeke
Copy link
Member

HaoZeke commented Jun 5, 2022

Note that the original issue of callbacks and character arrays is fixed by #19388. The issue raised for allocations is still relevant and tracked separately.

@HaoZeke HaoZeke closed this as completed Jun 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
06 - Regression component: numpy.f2py
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@charris @swryan @melissawm @HaoZeke @aelanman