Skip to content

BUG: np.asarray does not create new array view when different but compatible type code is passed in the dtype argument (Trac #870) #1468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
numpy-gitbot opened this issue Oct 19, 2012 · 6 comments · Fixed by #21995
Labels
00 - Bug component: numpy._core sprintable Issue fits the time-frame and setting of a sprint

Comments

@numpy-gitbot
Copy link

numpy-gitbot commented Oct 19, 2012

Original ticket http://projects.scipy.org/numpy/ticket/870 on 2008-08-01 by trac user damian.eads, assigned to @stefanv.

Examining ndarray type codes is common in C extensions before passing array contents to functions expecting int* or long*. The type codes are ideal ensure the contents can be safely casted to int* or long* on the host machine. In some cases, the int and long types are of the same size but their type codes are different. np.asarray(..., dtype='i') should be used to create a new array view with the type code the C type code expected by the C extension; however, it does not appear to work when using it for this purpose.

Problem
I construct X as a C-long array and then I cast it to a C-int array Y (using np.asarray) but the type code does not change. However, when constructing the array from scratch as a C-int (using np.zeros), I get the right type code (ie 5).

Expectation of Behavior
I assumed that when X gets casted to a C-int, no copying should occur but a new array view should be constructed with the C-int type code.

Workaround
Casting from a C-long (type code 7) to a double to a C-int returns an array with the right type code but an ugly double copy occurs.

Recreation

# Construct X as a C-long.

In [16]: X=numpy.zeros((10,10),dtype='l')

# Now cast X to a C-int.

In [17]: Y=numpy.asarray(X, dtype='i')

# Check X and Y's data type; they are the same.

In [18]: X.dtype
Out[18]: dtype('int32')

In [19]: Y.dtype
Out[19]: dtype('int32')

# Their type codes are the same.

In [20]: X.dtype.num
Out[20]: 7

In [21]: Y.dtype.num
Out[21]: 7

# Constructing with dtype='i', gives the right type code.

In [22]: Z=numpy.zeros((10,10),dtype='i')

In [23]: Z.dtype
Out[23]: dtype('int32')

In [24]: Z.dtype.num
Out[24]: 5

Stefan van der Walt observes that the dtype objects are compared for equality but not the type codes. This explains why the asarray function does not create a new view.

In [4]: np.dtype('i') == np.dtype('l')
Out[4]: True

In [5]: np.dtype('i').num == np.dtype('l').num
Out[5]: False

Fix

Check the type codes for equality, not the dtype objects, to decide whether to create a new array view in np.asarray.


summary @seberg:

This is still true, the array fast-paths to do the no-copy behaviour is correct to not copy, but apparently not aggressive enough with creating views, on 64bit linux:

arr = np.array([1, 2, 3, 4], "q")
np.asarray(arr, dtype="l") is arr  # True
np.dtype("l").num == np.dtype("q").num  # not True

The good news: This really should be a simple fix!

@numpy-gitbot
Copy link
Author

trac user tihocan wrote on 2010-01-20

Another workaround
I'm having the same problem and found that using .view instead of giving a dtype to asarray may do the trick. Example:

Y=numpy.asarray(X).view(numpy.intc)

@numpy-gitbot
Copy link
Author

trac user tihocan wrote on 2010-01-21

Replying to [comment:2 tihocan]:

Another workaround
I'm having the same problem and found that using .view instead of giving a dtype to asarray may do the trick. Example:

{{{
Y=numpy.asarray(X).view(numpy.intc)
}}}

Sorry, the code above is not a good idea, you need to give a dtype to asarray as well in case a conversion is needed. Correct version:

Y=numpy.asarray(X, dtype=numpy.inc).view(numpy.intc)

@numpy-gitbot
Copy link
Author

@mwiebe wrote on 2011-03-23

This is yet another issue that would go away by switching to implementing based on the sized types instead of the C types.

@numpy-gitbot
Copy link
Author

Milestone changed to Unscheduled by @mwiebe on 2011-03-23

@NeilGirdhar
Copy link
Contributor

@mwiebe Was the implementation ultimately fixed?

@seberg seberg added the sprintable Issue fits the time-frame and setting of a sprint label May 12, 2022
@eirrgang
Copy link
Contributor

PR is in preparation.

Intended behavior:

  • numpy.asarray(..., dtype=X) always produces an object with type as specified to dtype kwarg, even if existing object type is equivalent.
  • np.dtype('i') == np.dtype('l') continues to evaluate equivalence. It does not imply identical character codes or other (possibly aliased) symbolic type.

eirrgang added a commit to eirrgang/numpy that referenced this issue Jul 16, 2022
For `asarray` and for the `dtype` equality operator,
equivalent dtype aliases are considered exact matches.

Note: Intended behavior of `np.dtype('i') == np.dtype('l')`
is to test compatibility, not identity. This change does not
affect the behavior of `PyArray_EquivTypes()`, and the
`__eq__` operator for `dtype` continues to map to `PyArray_EquivTypes()`.

Fixes numpy#1468.
eirrgang added a commit to eirrgang/numpy that referenced this issue Jul 16, 2022
For `asarray` and for the `dtype` equality operator,
equivalent dtype aliases are considered exact matches.

Note: Intended behavior of `np.dtype('i') == np.dtype('l')`
is to test compatibility, not identity. This change does not
affect the behavior of `PyArray_EquivTypes()`, and the
`__eq__` operator for `dtype` continues to map to
`PyArray_EquivTypes()`.

Fixes numpy#1468.
eirrgang added a commit to eirrgang/numpy that referenced this issue Jul 16, 2022
For `asarray` and for the `dtype` equality operator,
equivalent dtype aliases were considered exact matches.
This change ensures that the returned array has a descriptor
that exactly matches the requested dtype.

Note: Intended behavior of `np.dtype('i') == np.dtype('l')`
is to test compatibility, not identity. This change does not
affect the behavior of `PyArray_EquivTypes()`, and the
`__eq__` operator for `dtype` continues to map to
`PyArray_EquivTypes()`.

Fixes numpy#1468.
@InessaPawson InessaPawson changed the title np.asarray does not create new array view when different but compatible type code is passed in the dtype argument (Trac #870) BUG: np.asarray does not create new array view when different but compatible type code is passed in the dtype argument (Trac #870) Aug 24, 2022
eirrgang added a commit to eirrgang/numpy that referenced this issue Sep 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug component: numpy._core sprintable Issue fits the time-frame and setting of a sprint
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants