Fix structured compare #5

mwiebe · 2010-10-19T19:42:36Z

Here's the patch for fixing structured array comparisons with multi-dimensional field types.

http://projects.scipy.org/numpy/ticket/1640

Would it make sense to also target it for 1.5.1 or 1.5.2?

Thanks,
Mark

charris · 2010-10-20T03:11:48Z

I'm inclined to commit this, but I'll wait until you and Pauli agree. I think it might be worth rewriting the history to just have one commit but maybe that's getting a bit too fancy. As to the backport, that is really up to Ralph at this point. If it was simple Python I'd say yes, but with the c source there is always the possibility of introducing subtle errors.

mwiebe · 2010-10-20T04:14:56Z

I rewrote the history to one commit. It should have all the changes for the comments you made, but it looks like rewriting the history threw the comments themselves away.

rgommers · 2010-10-20T12:14:08Z

The first release candidate for 1.5.1 is already out, so I don't think it's a good idea to put it in the 1.5.x branch now. As far as I can tell this is also not an especially urgent thing to fix in 1.5.x (please correct me if I'm wrong).

mwiebe · 2010-10-20T18:06:07Z

Yeah, it makes good sense to only fix regressions after the RC. I would argue that the chance for adverse interactions is very minimal, because all the C code is inside an if statement which will only be true when it's on its way to computing the wrong answer or crashing anyway, but it's not so urgent so as to override any precaution.

pv · 2010-10-20T20:05:43Z

This still allows broadcasting within arrays:

>>> import numpy as np
>>> x = np.zeros((1,), dtype=[('a', 'f4', (1,2)), ('b', 'i1')])
>>> y = np.zeros((1,), dtype=[('a', 'f4', (2,)), ('b', 'i1')])
>>> x == y
array([[[ True,  True]]], dtype=bool)

While the expected result is scalar False. The arrays are not comparable, and so array_richcompare should return Py_NotImplemented. Also, the following behavior originating from the same stuff seems wrong:

>>> x = np.zeros((1,), dtype=[('a', ('f4', (2,1))), ('b', 'i1')])
>>> y = np.zeros((1,), dtype=[('a', ('f4', (2,))), ('b', 'i1')])
>>> x == y
array([[[ True,  True],
        [ True,  True]]], dtype=bool)

The expected result here is False.

I think the correct fix is to check that a and b have the same shapes, before calling array_richcompare on them.

mwiebe · 2010-10-20T21:19:12Z

This seems to me like it's a bug in the dtype comparison. Instead of Py_NotImplemented, maybe array_richcompare should raise a PyExc_AssertionError if a and b are different shapes? I think it's an error rather than something to implement in the future.

Here's what I get for some dtype comparisons which seems quite wrong:

>>> np.dtype(('>f4',4)) == np.dtype(('<f4',(4,)))
True
>>> np.dtype(('>f4',4)) == np.dtype(('<f4',(2,2)))
True

It appears to ignore the shape entirely, and just check the itemsize.

charris · 2010-10-20T21:54:11Z

Py_NotImplemented is a message to the interpreter to try the object on the other side of the operator to see if it can handle the request, not to be confused with PyExc_NotImplementedError.

I think that comparisons of structured type just does a byte comparison if there are no named fields, or so ISTR from some comment in the code. Ignoring the shape in dtypes would seem consistent with that, although tt doesn't seem quite right.

I don't understand how comparisons between structured types is supposed to behave. Is it correct to say that broadcasting is not allowed?

…parison

mwiebe · 2010-10-28T00:58:48Z

Since there were no more comments on Fortran-style array subarray indexing, I went ahead and implemented removal of the special-casing. Unless this behaviour change is objected to, I think the patch should be good to go.

pv · 2010-10-31T18:47:06Z

Cleaned-up patch set here: http://github.com/pv/numpy/compare/master...bug/fix_structured_compare

I'll try to see tomorrow to commit it.

charris · 2010-11-09T02:19:31Z

Hi Pauli, did you mean to reopen this or was there some sort of github glitch?

pv · 2010-11-09T09:04:34Z

Yes, reopened as it's not committed yet -- I was busy last week with other stuff.

pv · 2010-11-20T00:28:55Z

Thanks, pushed finally.

DOC: Start documentation

BUG: Correct handling of nans

Cffi 1.x

# This is the 1st commit message: Add Windows config to GHA # This is the commit message numpy#2: update script [wheel build] # This is the commit message numpy#3: typo [wheel build] # This is the commit message numpy#4: fix typo? [wheel build] # This is the commit message numpy#5: fix linux builds? [wheel build] # This is the commit message numpy#6: typo [wheel build] # This is the commit message numpy#7: add license and pin to windows 2016 # This is the commit message numpy#8: skip tests [wheel build] # This is the commit message numpy#9: pin to windows 2019 instead [wheel build] # This is the commit message numpy#10: try to find out the error on windows [wheel build] # This is the commit message numpy#11: maybe fix? [wheel build] # This is the commit message numpy#12: maybe fix? [wheel build] # This is the commit message numpy#13: fix? [wheel build] # This is the commit message numpy#14: cleanup [wheel build]

commit 30228578ac65ec6b90961700a5783c23d8e1b879 Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Wed Nov 17 16:21:27 2021 -0800 Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt commit 4bd12df Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Mon Nov 15 17:28:47 2021 -0800 # This is a combination of 14 commits. # This is the 1st commit message: Add Windows config to GHA # This is the commit message numpy#2: update script [wheel build] # This is the commit message numpy#3: typo [wheel build] # This is the commit message numpy#4: fix typo? [wheel build] # This is the commit message numpy#5: fix linux builds? [wheel build] # This is the commit message numpy#6: typo [wheel build] # This is the commit message numpy#7: add license and pin to windows 2016 # This is the commit message numpy#8: skip tests [wheel build] # This is the commit message numpy#9: pin to windows 2019 instead [wheel build] # This is the commit message numpy#10: try to find out the error on windows [wheel build] # This is the commit message numpy#11: maybe fix? [wheel build] # This is the commit message numpy#12: maybe fix? [wheel build] # This is the commit message numpy#13: fix? [wheel build] # This is the commit message numpy#14: cleanup [wheel build] commit 444a721 Merge: 376ad69 22448b4 Author: Charles Harris <charlesr.harris@gmail.com> Date: Mon Nov 15 17:47:23 2021 -0700 Merge pull request numpy#20274 from h-vetinari/fix_15179 TST: Some fixes & refactoring around glibc-dependent skips in test_umath.py commit 376ad69 Merge: b75fe57 546c47a Author: Charles Harris <charlesr.harris@gmail.com> Date: Mon Nov 15 17:31:41 2021 -0700 Merge pull request numpy#20327 from seberg/rename-interpolation BUG,DEP: Fixup quantile/percentile and rename interpolation->method commit 546c47a Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 15 16:13:50 2021 -0600 DOC: Fixups for interpolation rename comments from review Co-authored-by: Charles Harris <charlesr.harris@gmail.com> commit b75fe57 Merge: 7310c09 cbc25d2 Author: Warren Weckesser <warren.weckesser@gmail.com> Date: Mon Nov 15 17:27:08 2021 -0500 Merge pull request numpy#20369 from bilderbuchi/missing_diagnostics_newlines MAINT: Fix newlines in diagnostics output of numpy.f2py. commit cbc25d2 Author: Christoph Buchner <bilderbuchi@phononoia.at> Date: Sun Nov 14 08:36:03 2021 +0100 MAINT: Fix newlines in diagnostics output of numpy.f2py. Linebreaks were not consistently added to errmess/outmess arguments, which led to very long lines and wrong concatenation with compiler messages in f2py console output. commit be15716 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Fri Nov 12 12:10:20 2021 -0600 DOC: Add release not for quantile `interpolation` rename Also updates the old release note to include the info (and generally tweaks it a bit) commit 7d8a8e7 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Fri Nov 12 11:57:22 2021 -0600 DOC: Update percentile/quantile docs Mainly fixes the method list slightly, tones down the warning a bit and fixes the link to the paper (I did not realize that the link failed to work due only because the reference was missing from nanquantile/nanpercentile). commit 5bd71fb Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Tue Nov 9 09:48:59 2021 -0600 DOC: Add ticks to quantile interpolation/method error Co-authored-by: abel <aoun@cerfacs.fr> commit 0d5fb81 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 20:39:50 2021 -0600 DOC: Remove reference to paper from quantile `method` kwarg Apparently, sphinx does not resolve references to footnotes from parameter descriptions. commit 8437663 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 18:25:37 2021 -0600 MAINT: Rename interpolation to method in percentile stubs commit 8cfb6b5 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:47:27 2021 -0600 TST: Add deprecation testcase for quantile interpolation rename commit a5ac5a5 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:41:24 2021 -0600 DOC: Fixup the percentile methods plot commit 85f3dda Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:37:41 2021 -0600 BUG: quantile discrete methods ended up using -1 as index sometimes Also, the closest-observation did not correctly support multiple quantiles calculated at the same time (broadcasting error). commit 3993408 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 15:38:30 2021 -0600 API,DEP: Rename percentile/quantile `interpolation=` to `method=` commit 22448b4 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 16:51:59 2021 +1100 TST: parametrize glibc-version check in test_umath.py commit 12923c2 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:21:01 2021 +1100 TST: skip coverage of large elements in sincos_float32 for ancient glibc Fixes numpy#15179 commit 56268d5 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:19:24 2021 +1100 TST: use existence of glibc-version to clean up a test commit 01443e8 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:18:52 2021 +1100 TST: turn glibc_older_than_2.17 into boolean instead of xfail this anticipates reuse of this boolean within the test module

commit 9c833bed5879d77e625556260690c349de18b433 Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Wed Nov 17 16:21:27 2021 -0800 Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Update cibw_test_command.sh commit 4bd12df Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Mon Nov 15 17:28:47 2021 -0800 # This is a combination of 14 commits. # This is the 1st commit message: Add Windows config to GHA # This is the commit message numpy#2: update script [wheel build] # This is the commit message numpy#3: typo [wheel build] # This is the commit message numpy#4: fix typo? [wheel build] # This is the commit message numpy#5: fix linux builds? [wheel build] # This is the commit message numpy#6: typo [wheel build] # This is the commit message numpy#7: add license and pin to windows 2016 # This is the commit message numpy#8: skip tests [wheel build] # This is the commit message numpy#9: pin to windows 2019 instead [wheel build] # This is the commit message numpy#10: try to find out the error on windows [wheel build] # This is the commit message numpy#11: maybe fix? [wheel build] # This is the commit message numpy#12: maybe fix? [wheel build] # This is the commit message numpy#13: fix? [wheel build] # This is the commit message numpy#14: cleanup [wheel build]

feat: Add vreinterpret intrinsics

mwiebe added 3 commits October 27, 2010 17:25

Implemented tests and a fix for record array comparison

9787a71

Added tests and code for handling of broadcasting during subarray com…

5a81e8c

…parison

Changed subarray field indexing to always append the shape, added test.

4ff03c1

numpy-gitbot mentioned this pull request Oct 19, 2012

segfault on searchsorted of wrong dtype in structured array (Trac #2189) #642

Closed

This was referenced Oct 19, 2012

importing umath_tests and then CTRL-D crashes with core dump (Trac #1425) #2023

Closed

segfault in any() on large object array (Trac #1522) #2119

Closed

This was referenced Oct 19, 2012

numpy.ndarray.clip segfaults on big-endian dtype (Trac #1588) #2184

Closed

segmentation fault with dot product of sparse matrices (Trac #1983) #2576

Closed

njsmith mentioned this pull request Dec 16, 2012

Segfault ufunc_object.c:1750 #2843

Closed

njsmith mentioned this pull request Jul 3, 2014

Segmentation fault when using memory allocation tracker #4834

Closed

njsmith mentioned this pull request May 2, 2016

Segmentation fault in PyArray_Item_INCREF #7595

Closed

asanakoy mentioned this pull request Jun 6, 2016

numpy 1.11. Segfault: numpy.random.permutation on list of long strings #7710

Closed

njsmith mentioned this pull request Nov 8, 2017

ENH: skip NPY_ALLOW_C_API for UFUNC_ERR_IGNORE #9985

Merged

bdrosen96 mentioned this pull request Sep 15, 2018

Occasional segfault with cacheing allocator and multiple threads #11942

Closed

roachlord mentioned this pull request Feb 18, 2019

Passing MagicMock to np.dtype causes a segmentation fault #12982

Closed

UKeyboard mentioned this pull request Feb 26, 2019

BUG: Segmentation fault in temporary eliding (backtrace() fails, probably related to pthread locking) #13042

Closed

mattip referenced this pull request in mattip/numpy Mar 20, 2019

Merge pull request #5 from bashtage/add-generator

3e3c9a9

DOC: Start documentation

bashtage referenced this pull request in bashtage/numpy Apr 8, 2019

Merge pull request #5 from bashtage/nan-protection-correction

d15ec91

BUG: Correct handling of nans

siddharth-singh-mindtickle mentioned this pull request Jul 24, 2019

numpy throws munmap_chunk(): invalid pointer when freeing its cache #14102

Closed

wadeliuyi mentioned this pull request Feb 19, 2020

numpy crash #15603

Closed

leakec mentioned this pull request Sep 29, 2020

Potential Issue with 2-D ARGOUTVIEWM Wrapping #17398

Closed

leidemon mentioned this pull request Dec 4, 2020

ImportError: unknown dlopen() error #17858

Closed

prototype99 referenced this pull request in prototype99/numpy Jan 4, 2021

Merged in cffi-1.x (pull request ganesh-k13#5)

73033fe

Cffi 1.x

tpham3783 mentioned this pull request Jun 3, 2022

Crash calling import_array() in C++-Api when python runs in separate thread #21588

Closed

magicse mentioned this pull request Mar 1, 2023

pip install --no-cache numpy on RAMv7 i get libs for ARMv8 arch #23304

Open

asmeurer mentioned this pull request Jun 2, 2023

ENH: add copy parameter for api.reshape function #23789

Merged

luyahan referenced this pull request in plctlab/numpy Apr 25, 2024

Merge pull request #5 from howjmay/vreinterpret

0245460

feat: Add vreinterpret intrinsics

limburgher mentioned this pull request Jan 7, 2025

Power9 instruction gets executed on Power8 #28124

Closed

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix structured compare #5

Fix structured compare #5

mwiebe commented Oct 19, 2010

charris commented Oct 20, 2010

mwiebe commented Oct 20, 2010

rgommers commented Oct 20, 2010

mwiebe commented Oct 20, 2010

pv commented Oct 20, 2010

mwiebe commented Oct 20, 2010

charris commented Oct 20, 2010

mwiebe commented Oct 28, 2010

pv commented Oct 31, 2010

charris commented Nov 9, 2010

pv commented Nov 9, 2010

pv commented Nov 20, 2010

Fix structured compare #5

Fix structured compare #5

Conversation

mwiebe commented Oct 19, 2010

charris commented Oct 20, 2010

mwiebe commented Oct 20, 2010

rgommers commented Oct 20, 2010

mwiebe commented Oct 20, 2010

pv commented Oct 20, 2010

mwiebe commented Oct 20, 2010

charris commented Oct 20, 2010

mwiebe commented Oct 28, 2010

pv commented Oct 31, 2010

charris commented Nov 9, 2010

pv commented Nov 9, 2010

pv commented Nov 20, 2010