Skip to content

Data type precision problems? #5272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
durack1 opened this issue Nov 11, 2014 · 15 comments
Open

Data type precision problems? #5272

durack1 opened this issue Nov 11, 2014 · 15 comments
Labels

Comments

@durack1
Copy link

durack1 commented Nov 11, 2014

Hi folks I've just been investigating some type conversion and came across what appears unusual behaviour to me - I would have expected these statements to more faithfully print the values at their respective precisions:

>>> import numpy
>>> numpy.version.version
'1.9.0'
>>> a = numpy.array([3991.86795711963],dtype='float64')
>>> print a
[ 3991.86795712]
>>> print numpy.float32(a)
[ 3991.86791992]
>>> print numpy.float64(a)
3991.86795712 ; # Why does this print statement look different to the above (float32) - no brackets
>>> a = numpy.array([3991.86795711963],dtype='float128')
>>> print numpy.float128(a)
[ 3991.868] ; # Why is this truncated when it should have double the precision of float64?
>>> a
array([ 3991.868], dtype=float128)
>>> numpy.float128(a)
array([ 3991.868], dtype=float128)
>>> print a
[ 3991.868]
>>> print a*1
[ 3991.868]

I haven't investigated playing around with numpy.set_printoptions(precision=2)

@durack1
Copy link
Author

durack1 commented Nov 15, 2014

@juliantaylor @charris do you folks have any insight to provide to me here?

@matthew-brett
Copy link
Contributor

I don't think you should be addressing this to the maintainers directly, this is a routine question, and I don't think you have any reason to ask Julian or Chuck specifically.

I suggest you try this on the mailing list - I think your question is a request for clarification. Also, you might get some more insight by experimenting with ``set_printoptions` as you imply.

@juliantaylor
Copy link
Contributor

float64 converts to a python float which is probably the reason why it prints different. float32 and float128 are numpy types.
also very important, float128 is not quad precision, its actually long double, being 80 bit precision if your platform supports it (on 32 bit its called float96). Its probably the worst named thing in whole of numpy.

@durack1
Copy link
Author

durack1 commented Nov 15, 2014

@juliantaylor thanks for the clarification, my assumption was that all numpy.type declarations were numpy types - the point to note about float64 being a python type is a surprise to me.. And ouch about float128 too, I thought the name (128) and it's type were the same..

Thanks for the clarification, I'll trawl through the numpy docs and educate myself a little better on this..

@matthew-brett
Copy link
Contributor

I have suggested deprecating the name float128 etc in the past, but the discussion seemed to stall.

http://mail.scipy.org/pipermail/numpy-discussion/2011-October/058806.html

Is it time for another iteration of that discussion? The current naming is quite clearly not helpful.

@durack1
Copy link
Author

durack1 commented Nov 15, 2014

@matthew-brett I would vote with you for a name change.. It's clearly tripped me over (at least in my understanding) and also seems to be an omission in the numpy data type docs http://docs.scipy.org/doc/numpy/user/basics.types.html (I note that as a footnote to the data type 'table' the additional platform dependent types short,long and longlong are actually listed)

If it's not documented, I think either remove it or rename (and document) it..

It also wasn't clear to me that numpy.float64 is a python float, rather than an internal numpy float/data type - this quirk is also not clear in the docs (linked above) - what this means (python float vs numpy float) would also be a question in which I'd be interested to know the answer..

@matthew-brett
Copy link
Contributor

Also I think the shape change you noticed is very difficult to explain:

In [17]: a = np.arange(1, dtype=np.float)
In [18]: a.shape
Out[18]: (1,)
In [19]: np.float64(a).shape
Out[19]: ()
In [20]: np.float32(a).shape
Out[20]: (1,)

@durack1
Copy link
Author

durack1 commented Nov 15, 2014

@matthew-brett this shape change behavior seems to be consistent at least back to 1.7.1.. I'd certainly be interested to know why float64 is a python data type rather than a numpy internal data type..

@matthew-brett
Copy link
Contributor

It may be that float64 printing works via Python floats, but the output of np.float64(a) is still a numpy array:

In [3]: a = np.arange(1, dtype=np.float)
In [4]: np.float64(a).dtype
Out[4]: dtype('float64')

@durack1
Copy link
Author

durack1 commented Nov 15, 2014

@matthew-brett @juliantaylor I think the rename suggestion of float80_96 (32-bit) and float80_128 (64-bit) seems like a path forward, as described at http://mail.scipy.org/pipermail/numpy-discussion/2011-October/058820.html

@durack1
Copy link
Author

durack1 commented Dec 5, 2014

@juliantaylor @charris I do realise that type renaming is a very low priority, however how would I get this assigned to a milestone 1.9.2 or 1.10? Do you folks require any more information to make this fairly trivial name change..?

@njsmith
Copy link
Member

njsmith commented Dec 5, 2014

Basically what we'd need is a pull request implementing the change :-).
Note that backwards compatibility means that the old name(s) would need to
stick around for now.
On 5 Dec 2014 05:43, "Paul J. Durack" notifications@github.com wrote:

@juliantaylor https://github.com/juliantaylor @charris
https://github.com/charris I do realise that type renaming is a very
low priority, however how would I get this assigned to a milestone 1.9.2 or
1.10? Do you folks require any more information to make this fairly trivial
name change..?


Reply to this email directly or view it on GitHub
#5272 (comment).

orbeckst pushed a commit to MDAnalysis/GridDataFormats that referenced this issue Jan 20, 2018
* use longdouble instead of float128 in test_write_dx_ValueError

float128 doesn't exist on 32-bit arches and longdouble maps to float96
there. See also numpy/numpy#5272 .
* fixes issue #44.
@melissawm
Copy link
Member

melissawm commented Jul 13, 2022

As of NumPy 1.24.0, we get:

In [1]: a = np.array([3991.86795711963],dtype='float64')

In [2]: a
Out[2]: array([3991.86795712])

In [3]: np.float32(a)
Out[3]: array([3991.868], dtype=float32)

In [4]: np.float64(a)
Out[4]: 3991.86795711963

In [5]: np.float128(a)
Out[5]: array([3991.86795712], dtype=float128)

In [6]: a*1
Out[6]: array([3991.86795712])

I believe this is no longer relevant so I am closing - feel free to reopen if there are still pending issues.

See also #10288 for the float128 name discussion.

@WarrenWeckesser
Copy link
Member

Is the difference in shape between np.float32(a) and np.float64(a) considered correct behavior? Apparently there are $REASONS and $HISTORY, but that inconsistency looks like a bug to me.

@melissawm
Copy link
Member

Oops, good catch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants