Skip to content

np.concatenate loses endianness / byte order #7829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Cerno-b opened this issue Jul 12, 2016 · 3 comments
Open

np.concatenate loses endianness / byte order #7829

Cerno-b opened this issue Jul 12, 2016 · 3 comments
Labels
component: numpy._core

Comments

@Cerno-b
Copy link

Cerno-b commented Jul 12, 2016

It seems to me that the result of vstack does not inherit Endianness from the function parameters correctly.

Example:

a = np.zeros((3,3), dtype='>i2')
print a.dtype
b = np.vstack((a,a))
print b.dtype

The first print gives the correct '>i2' but the second one gives 'uint16' which should mean Little Endinness, since I'm running a 64bit Windows 7 machine.

The problem arose when trying to load and concatenate binary 16 bit pgm images. Pgm stores images in Big Endian, and my routine can read and store the images correctly. It is only when I try to concatenate two images with vstack that the resulting image is corrupted since the byte order gets switched.
@eric-wieser eric-wieser changed the title vstack loses endinness np.concatenate loses endianness / byte order Dec 12, 2017
@eric-wieser
Copy link
Member

Interestingly not true for single elements.

This applies to concatenate, which vstack happens to be implemented with.

@hmaarrfk hmaarrfk mentioned this issue Oct 6, 2018
Merged
5 tasks
@tylerjereddy tylerjereddy mentioned this issue Apr 10, 2022
Closed
@seberg
Copy link
Member

seberg commented Apr 11, 2022

I am not convinced we should attempt to preserve this. Although concatenate now has a dtype argument, and the stack could/should inherit that. That would give users a way to work around it if they really care about byte-order.

@seberg
Copy link
Member

seberg commented Apr 12, 2022

As mentioned in gh-21319: There is a bit more discussion around this topic in #15088. I do think for np.result_type "canonicalizing" the dtype makes sense. For concatenate, I agree there could be an argument against it. But I still think that unless you happen to write anything to do with serialization you are better off with the canonicalizing behaviour anyway.

So right now, it seems to me that preserving byte-order would really only benefit very few users who probably need to double check a lot of things anyway.

If htere is anyone much in favor of changing it, they should champion this change and argue for it. Until then, this is unlikely to be changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component: numpy._core
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

WIP, BUG: preserve endianness of concat tylerjereddy/numpy
3 participants
@seberg @eric-wieser @Cerno-b