BUG: Numpy scalar types sometimes have the same name #10151

eric-wieser · 2017-12-02T21:19:29Z

~~This breaks np.core.numerictypes.bitname on integer types, but perhaps it's private enough that we don't care.~~ (no longer true)

Tested on 64-bit windows locally.

Before:

>>> np.dtype(np.int_), np.int_
(dtype('int32'), numpy.int32)

>>> np.dtype(np.intc), np.intc
(dtype('int32'), numpy.int32)

>>> np.dtype(np.double), np.double
(dtype('float64'), numpy.float64)

>>> np.dtype(np.longdouble), np.longdouble
(dtype('float64'), numpy.float64)

After:

>>> np.dtype(np.int_), np.int_
(dtype('int32'), numpy.int32)

>>> np.dtype(np.intc), np.intc
(dtype('int32'), numpy.intc)

>>> np.dtype(np.double), np.double
(dtype('float64'), numpy.float64)

>>> np.dtype(np.longdouble), np.longdouble
(dtype('float64'), numpy.longdouble)

If we want to "fix" the dtype output too, then it probably makes sense to wait for #10602

numpy/core/numerictypes.py

eric-wieser · 2018-01-10T07:13:36Z

Finally passing. There were a bunch of places that actually relied upon the name being what it was.

numpy/core/tests/test_scalarmath.py

eric-wieser · 2018-06-26T04:57:09Z

Rebased on #11328, now a very contained diff

mhvk · 2018-06-26T13:20:52Z

This looks like a nice solution; one question, though: is the definition such that if one does

np.array(1, dtype=int).dtype

one always gets a dtype with a repr that is dtype('int32') or dtype('int64')? I ask because for the "default" python int one should definitely get the form that has the number of bits.

Indeed, could you add a test that explicitly checks this?

eric-wieser · 2018-06-26T15:38:07Z

@mhvk: That's not the right constraint. The requirement is that np.int64.__name__ == 'int64'. We can't change what np.int64 points to now without risking breaking downstream users

I think that what you ask for happens anyway, but i don't think testing it is in scope for this patch.

mhvk · 2018-06-26T16:38:38Z

My comment was only that this particular aspect should not change; as I don't have easy access to a 32-bit machine, though, I cannot test it...

eric-wieser · 2018-06-26T16:46:17Z

I don't know if I agree with that comment.

There are two questions here:

Is np.array(1).dtype.type is np.int64?
Is np.array(1).dtype.type.__name__ == 'int64'?

My claim is that this patch does not change the answer to the first question, and that its goal is to reduce confusion by making the answer to the second question match the first one. If those answers are False, that's unfortunate, but changing (1) has compatibility ramifications, and changing (2) does not.

As it happens, I think that the first answer is always true anyway, since the default integer type is a C long, which gets first choice of name.

mattip · 2018-10-16T07:14:20Z

xref #12179 which is also about cleanups around dtype.__repr__

eric-wieser · 2018-10-16T12:35:55Z

Yep, I keep running into problems that need solving before I can get this one working

charris · 2018-11-13T19:22:17Z

@eric-wieser What do you want to do with this?

charris · 2018-11-14T16:29:23Z

Needs rebase

charris · 2018-11-14T19:53:34Z

This seems a WIP, so pushing off to 1.17.

A Numpy string formatting bug causes this doctest to fail on certain 32-bit platforms (e.g. armhf). The bug is evident in the following example: >>> np.arange(8) array([0, 1, 2, 3, 4, 5, 6, 7]) >>> 1 << np.arange(8) array([ 1, 2, 4, 8, 16, 32, 64, 128], dtype=int32) The `dtype=int32` should be supressed in both cases, but it is not. This might get fixed upstream in Numpy 1.17. See numpy/numpy#9799, numpy/numpy#10151.

eric-wieser · 2019-09-12T00:09:53Z

Tests passing

seberg

I am good with giving this a shot, I cannot really think of a reasonable way to write code that is broken by this...

numpy/core/src/multiarray/scalartypes.c.src

numpy/core/tests/test_numerictypes.py

This removes a test that enforced the opposite - dtype.name is documented as being a bitname, but it is exactly this property that causes confusion when applied to __name__ - so we should not expect them to be equal. Fixes numpygh-9799

eric-wieser · 2019-09-12T05:42:24Z

Updated with a tweaked comment and better tests

mattip · 2019-09-12T11:31:22Z

doc/release/upcoming_changes/10151.improvement.rst

+------------------------------------------------------------
+On any given platform, two of ``np.intc``, ``np.int_``, and ``np.longlong``
+would previously appear indistinguishable through their ``repr``, despite
+having different properties when wrapped into ``dtype``s.


nit: the s right after the double back-ticks sometimes causes problems when rendering, but I am not sure what the actual logic is. Hopefully can be caught when rendering the release note

You're right, needs to be ``dtype``\ s

mattip · 2019-09-12T11:33:21Z

Thanks, let's give this a shot. I hope there are no work-arounds in user code that this breaks.

eric-wieser added the 00 - Bug label Dec 2, 2017

eric-wieser mentioned this pull request Dec 3, 2017

MAINT: Use a StructSequence in place of the typeinfo tuples #10154

Merged

eric-wieser added 55 - Needs work component: numpy._core labels Dec 3, 2017

eric-wieser mentioned this pull request Dec 12, 2017

arr.dtype.type has different hashes #4779

Closed

eric-wieser force-pushed the integer-type-__name__ branch from aaa43af to cec5248 Compare January 5, 2018 16:03

eric-wieser commented Jan 5, 2018

View reviewed changes

numpy/core/numerictypes.py Outdated Show resolved Hide resolved

eric-wieser force-pushed the integer-type-__name__ branch from cec5248 to 077ac4f Compare January 5, 2018 16:15

eric-wieser commented Jan 5, 2018

View reviewed changes

numpy/core/numerictypes.py Outdated Show resolved Hide resolved

eric-wieser force-pushed the integer-type-__name__ branch from 077ac4f to 1658397 Compare January 7, 2018 00:28

eric-wieser removed the 55 - Needs work label Jan 7, 2018

eric-wieser force-pushed the integer-type-__name__ branch 4 times, most recently from 110e934 to 2e57b32 Compare January 10, 2018 03:46

eric-wieser commented Jan 10, 2018

View reviewed changes

numpy/core/tests/test_scalarmath.py Outdated Show resolved Hide resolved

eric-wieser mentioned this pull request Feb 13, 2018

bitwise_and corrupting dtype.type on windows #10579

Open

eric-wieser mentioned this pull request Mar 20, 2018

Add NumPy scalar hierarchy numpy/numpy-stubs#14

Merged

This was referenced Apr 30, 2018

.dtype.type class object is not preserved over some operations #11020

Closed

Image inversion of uint64 scikit-image/scikit-image#3043

Closed

eric-wieser force-pushed the integer-type-__name__ branch from 2e57b32 to 8e15d05 Compare June 13, 2018 08:42

This was referenced Jun 13, 2018

MAINT: Don't use dtype strings when the dtypes themselves can be used #11324

Merged

MAINT: Misc numeric cleanup #11328

Merged

eric-wieser force-pushed the integer-type-__name__ branch from 8e15d05 to ce1a079 Compare June 26, 2018 04:55

eric-wieser mentioned this pull request Sep 14, 2018

MAINT: Small tidy-ups to np.core._dtype #11949

Merged

YannickJadoul mentioned this pull request Sep 14, 2018

DOC: add docstrings for numeric types #11858

Merged

eric-wieser mentioned this pull request Oct 4, 2018

BUG: Floor division returns messed up dtype on windows (python 3) #12069

Open

eric-wieser mentioned this pull request Oct 18, 2018

MAINT: avoid relying on np.generic.__name__ in np.dtype.name #12205

Merged

eric-wieser force-pushed the integer-type-__name__ branch from 4e127e6 to c848326 Compare October 18, 2018 05:20

charris modified the milestones: 1.16.0 release, 1.17.0 release Nov 14, 2018

lpsinger mentioned this pull request Jan 17, 2019

Work around Numpy formatting glitch in doctest healpy/healpy#524

Merged

charris removed this from the 1.17.0 release milestone May 11, 2019

0x0L mentioned this pull request Jun 11, 2019

ENH: ufunc helper for variance #13263

Closed

effigies mentioned this pull request Jun 12, 2019

Apparent is/equality failures in Python 3.4, 3.5 on Windows #12096

Closed

eric-wieser mentioned this pull request Aug 18, 2019

intp on 32bit architectures not same as int32 #6038

Closed

eric-wieser force-pushed the integer-type-__name__ branch from c848326 to 703cb31 Compare September 11, 2019 17:13

seberg approved these changes Sep 12, 2019

View reviewed changes

numpy/core/src/multiarray/scalartypes.c.src Outdated Show resolved Hide resolved

numpy/core/tests/test_numerictypes.py Show resolved Hide resolved

BUG: Ensure scalar types have unique __name__s

fa09f5e

This removes a test that enforced the opposite - dtype.name is documented as being a bitname, but it is exactly this property that causes confusion when applied to __name__ - so we should not expect them to be equal. Fixes numpygh-9799

eric-wieser force-pushed the integer-type-__name__ branch from 703cb31 to fa09f5e Compare September 12, 2019 05:41

mattip reviewed Sep 12, 2019

View reviewed changes

mattip merged commit 2f81858 into numpy:master Sep 12, 2019

eric-wieser deleted the integer-type-__name__ branch September 13, 2019 01:47

eric-wieser mentioned this pull request Oct 10, 2019

types compare as not-equal (e.g. np.int32 != np.int32) #14667

Closed

effigies mentioned this pull request Jan 7, 2020

TEST: Numpy changed longdouble str representations in 1.18 nipy/nibabel#858

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: Numpy scalar types sometimes have the same name #10151

BUG: Numpy scalar types sometimes have the same name #10151

eric-wieser commented Dec 2, 2017 •

edited

Loading

eric-wieser commented Jan 10, 2018

eric-wieser commented Jun 26, 2018

mhvk commented Jun 26, 2018

eric-wieser commented Jun 26, 2018 •

edited

Loading

mhvk commented Jun 26, 2018

eric-wieser commented Jun 26, 2018 •

edited

Loading

mattip commented Oct 16, 2018

eric-wieser commented Oct 16, 2018

charris commented Nov 13, 2018

charris commented Nov 14, 2018

charris commented Nov 14, 2018

eric-wieser commented Sep 12, 2019

seberg left a comment

eric-wieser commented Sep 12, 2019

mattip Sep 12, 2019

eric-wieser Sep 13, 2019

mattip commented Sep 12, 2019

BUG: Numpy scalar types sometimes have the same name #10151

BUG: Numpy scalar types sometimes have the same name #10151

Conversation

eric-wieser commented Dec 2, 2017 • edited Loading

eric-wieser commented Jan 10, 2018

eric-wieser commented Jun 26, 2018

mhvk commented Jun 26, 2018

eric-wieser commented Jun 26, 2018 • edited Loading

mhvk commented Jun 26, 2018

eric-wieser commented Jun 26, 2018 • edited Loading

mattip commented Oct 16, 2018

eric-wieser commented Oct 16, 2018

charris commented Nov 13, 2018

charris commented Nov 14, 2018

charris commented Nov 14, 2018

eric-wieser commented Sep 12, 2019

seberg left a comment

Choose a reason for hiding this comment

eric-wieser commented Sep 12, 2019

mattip Sep 12, 2019

Choose a reason for hiding this comment

eric-wieser Sep 13, 2019

Choose a reason for hiding this comment

mattip commented Sep 12, 2019

eric-wieser commented Dec 2, 2017 •

edited

Loading

eric-wieser commented Jun 26, 2018 •

edited

Loading

eric-wieser commented Jun 26, 2018 •

edited

Loading