Skip to content

gradient differences between v1.11.0 and v1.13.0 #9401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
evanmason opened this issue Jul 12, 2017 · 10 comments
Open

gradient differences between v1.11.0 and v1.13.0 #9401

evanmason opened this issue Jul 12, 2017 · 10 comments

Comments

@evanmason
Copy link

With np.gradient v1.11.0 it was possible to use 2d irregular dx and dy arguments, for example:

from pyproj import Proj
lon, lat = meshgrid(arange(-4, 5, 1.5), arange(3, 6, 1.5))
proj = Proj('+proj=aeqd +lat_0=%s +lon_0=%s' % (lon.mean(), lat.mean()))
dx, dy = proj(lon, lat)
data = lon + lat
ddx, ddy = gradient(data, dx, dy)

However with v.1.13.0 (and also in v.1.12.0) there is now an error:
ValueError: distances must be either scalars or match the length of the corresponding dimension

I know that gradient was recently updated #8446. I wonder if this functionality was overlooked, or expressly omitted. Either way it is useful when working with geophysical model data, and it would be nice to have it back again.

@eric-wieser
Copy link
Member

eric-wieser commented Jul 12, 2017

Can you print out dx and dy, for those of us without pyproj?

@dopplershift
Copy link

👍 to putting dx back to an array.

@eric-wieser
Copy link
Member

eric-wieser commented Jul 12, 2017

This runs without error for me, so you need to show us the type and shape of dx and dy:

>>> x, y = np.meshgrid(np.arange(3), np.arange(3))
>>> np.gradient(x+y, x, y)
[array([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]]), array([[ inf,  nan,  inf],
        [ inf,  nan,  inf],
        [ inf,  nan,  inf]])]

Results look pretty meaningless though

@chrisb13
Copy link

chrisb13 commented Jul 13, 2017

@eric-wieser, extending your example, is the following expected behaviour? Best I can tell the shapes/types match in this toy...

In [119]: print np.__version__
1.13.0

In [106]: x, y = np.meshgrid([4.,6,25], np.arange(3));np.gradient(x+y, x, axis=0)
Out[106]: 
array([[ 0.5       ,  0.5       ,  0.5       ],
       [ 0.45739348,  0.45739348,  0.45739348],
       [ 0.05263158,  0.05263158,  0.05263158]])

In [107]: type(x+y)
Out[107]: numpy.ndarray

In [108]: type(np.random.randint(0,4,size=(3,3)).astype('float'))
Out[108]: numpy.ndarray

In [109]: z=x+y

In [110]: type(z[0][0])
Out[110]: numpy.float64

In [111]: z=np.random.randint(0,4,size=(3,3)).astype('float')

In [112]: type(z[0][0])
Out[112]: numpy.float64

In [113]: np.shape(np.random.randint(0,4,size=(3,3)).astype('float'))
Out[113]: (3, 3)

In [114]: np.shape(x+y)
Out[114]: (3, 3)

In [115]: np.shape(x)
Out[115]: (3, 3)

In [116]: np.gradient(x+y, np.random.randint(0,4,size=(3,3)).astype('float'), axis=0)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-116-92d522988b0a> in <module>()
----> 1 np.gradient(x+y, np.random.randint(0,4,size=(3,3)).astype('float'), axis=0)

/home/chris/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.pyc in gradient(f, *varargs, **kwargs)
   1773             a.shape = b.shape = c.shape = shape
   1774             # 1D equivalent -- out[1:-1] = a * f[:-2] + b * f[1:-1] + c * f[2:]
-> 1775             out[slice1] = a * f[slice2] + b * f[slice3] + c * f[slice4]
   1776 
   1777         # Numerical differentiation: 1st order edges

ValueError: could not broadcast input array from shape (4,3) into shape (1,3)

I'm with @evanmason really useful to be able to pass a dx/dy/dz as an array for geophysical datasets. I use it a lot with version numpy 1.11.2. And when running on real data I get the same error:

ValueError: distances must be either scalars or match the length of the corresponding dimension

Here's an example (unzip and run on the attached):

import numpy as np
var=np.load('./toy.npz')
print np.__version__
print np.shape(var['dx'])
print np.shape(var['z'])
print type(var['z'][0][0])
print type(var['dx'][0][0])
np.gradient(var['z'],var['dx'],axis=1)

Here's the output..

% python loadtoy.py 
1.13.0
(289, 431)
(289, 431)
<type 'numpy.float32'>
<type 'numpy.float64'>
Traceback (most recent call last):
  File "loadtoy.py", line 12, in <module>
    np.gradient(var['z'],var['dx'],axis=1)
  File "/home/chris/anaconda/lib/python2.7/site-packages/numpy/lib/function_base.py", line 1695, in gradient
    raise ValueError("distances must be either scalars or match "
ValueError: distances must be either scalars or match the length of the corresponding dimension

Here's the output using np 1.11.2

% python loadtoy.py
1.11.2
(289, 431)
(289, 431)
<type 'numpy.float32'>
<type 'numpy.float64'>

toy.zip

@evanmason
Copy link
Author

Sorry, yes I should have included that, here are dx and dy from pyproj, @eric-wieser

In [50]: dx, dy
Out[50]: 
(array([[-862009.04816654, -695169.05606593, -528328.76211994,
         -361488.23931367, -194647.56018792,  -27806.79698057],
        [-861059.7691241 , -694404.14874909, -527747.81428583,
         -361090.93840543, -194433.69272203,  -27776.24812907]]),
 array([[ 360219.57862864,  359921.88896914,  359688.33036752,
          359518.65742001,  359412.69206566,  359370.32319323],
        [ 526597.07122525,  526121.21826211,  525747.8611421 ,
          525476.61970445,  525307.21810707,  525239.48422114]]))

@eric-wieser
Copy link
Member

The problem in your toy example is that axis=1. This is probably a bug in numpy, but...

Your first example doesn't have axis=1. Can you show the value of data too, so that we can reproduce that original example?

@eric-wieser
Copy link
Member

Ok, here's a simple repro:

>>> x, y = np.meshgrid(np.arange(3), np.arange(4))
>>> np.gradient(x + y, x, y)
ValueError: distances must be either scalars or match the length of the corresponding dimension

I don't think this ever did the right thing though. What is the output of the above for you on 1.10, @evanmason?

@evanmason
Copy link
Author

Here's what I get, @eric-wieser

In [1]: from pyproj import Proj
   ...: lon, lat = meshgrid(arange(-4, 5, 1.5), arange(3, 6, 1.5))
   ...: proj = Proj('+proj=aeqd +lat_0=%s +lon_0=%s' % (lon.mean(), lat.mean()))
   ...: dx, dy = proj(lon, lat)
   ...: data = lon + lat
   ...: ddx, ddy = gradient(data, dx, dy)
   ...: 

In [2]: ddx, ddy
Out[2]: 
(array([[ -1.74012095e-06,  -2.15774852e-06,  -2.83914128e-06,
          -4.14951259e-06,  -7.70623582e-06,  -5.39436455e-05],
        [ -1.74203935e-06,  -2.16012534e-06,  -2.84226663e-06,
          -4.15407821e-06,  -7.71471230e-06,  -5.40029738e-05]]),
 array([[  4.16412680e-06,   4.16757093e-06,   4.17027708e-06,
           4.17224522e-06,   4.17347532e-06,   4.17396736e-06],
        [  2.84847767e-06,   2.85105399e-06,   2.85307865e-06,
           2.85455136e-06,   2.85547190e-06,   2.85584014e-06]]))

In [3]: np.__version__
Out[3]: '1.11.0'

eric-wieser added a commit to eric-wieser/numpy that referenced this issue Jul 13, 2017
2d arrays would work, but in unpredictable and undocumented ways.

This at least makes numpygh-9401 give a better error message.
theodoregoetz pushed a commit to theodoregoetz/numpy that referenced this issue Oct 23, 2017
2d arrays would work, but in unpredictable and undocumented ways.

This at least makes numpygh-9401 give a better error message.
@eric-wieser
Copy link
Member

@evanmason: Sorry, I wasn't clear there, Can you run the code that I included in this answer, in your old version of numpy?

@evanmason
Copy link
Author

@eric-wieser: Hi, here's what I get:

In [1]: np.__version__
Out[1]: '1.10.0'

In [2]: x, y = np.meshgrid(np.arange(3), np.arange(4))

In [3]: np.gradient(x + y, x, y)
/home/emason/VENV_NP1.9.3/lib/python2.7/site-packages/numpy/lib/function_base.py:1101: RuntimeWarning: divide by zero encountered in true_divide
  out /= dx[axis]
Out[3]: 
[array([[ inf,  1. ,  0.5],
        [ inf,  1. ,  0.5],
        [ inf,  1. ,  0.5],
        [ inf,  1. ,  0.5]]), array([[        inf,         inf,         inf],
        [ 1.        ,  1.        ,  1.        ],
        [ 0.5       ,  0.5       ,  0.5       ],
        [ 0.33333333,  0.33333333,  0.33333333]])]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants