TestAVXFloat32Transcendental test failure #15179

rgommers · 2019-12-26T06:42:34Z

This test is failing on 64-bit Linux on the conda-forge Azure CI when building NumPy 1.18.0, see conda-forge/numpy-feedstock#179. Failed on Python 3.6, 3.7 and 3.8, so it's consistent. Not sure what's different about that setup, could be glibc version, compiler, or some Azure - TravisCI delta.

=================================== FAILURES ===================================
_______________ TestAVXFloat32Transcendental.test_sincos_float32 _______________

self = <numpy.core.tests.test_umath.TestAVXFloat32Transcendental object at 0x7f6a5342e080>

    def test_sincos_float32(self):
        np.random.seed(42)
        N = 1000000
        M = np.int_(N/20)
        index = np.random.randint(low=0, high=N, size=M)
        x_f32 = np.float32(np.random.uniform(low=-100.,high=100.,size=N))
        # test coverage for elements > 117435.992f for which glibc is used
        x_f32[index] = np.float32(10E+10*np.random.rand(M))
        x_f64 = np.float64(x_f32)
>       assert_array_max_ulp(np.sin(x_f32), np.float32(np.sin(x_f64)), maxulp=2)
E       AssertionError: Arrays are not almost equal up to 2 ULP

This functionality came in in gh-13368, new in 1.18.0. Likely the accuracy is just slightly over 2 ULP, conda-forge tests need to be re-run with larger tolerance. (related: PR for improving the test function to show actual difference in gh-15178)

Cc @r-devulap

The text was updated successfully, but these errors were encountered:

rgommers · 2019-12-26T07:28:46Z

Update:

Arrays are not almost equal up to 3 ULP (max difference is 67 ULP)

This is too much I think, needs looking into before putting this on conda-forge.

mattip · 2019-12-26T08:08:11Z

When I look at conda-forge/numpy-feedstock#179, it seems all the azure runs fail trying to patch numpy. Which one shows this failure?

rgommers · 2019-12-26T08:24:34Z

py36 and py38 (py37 has an unrelated error). e.g. https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=106332&view=logs&j=d0d954b5-f111-5dc4-4d76-03b6c9d0cf7e&t=6d4b912b-175d-51da-0fd9-4d30fe1eb4e7

r-devulap · 2019-12-26T19:00:06Z

It is likely that this is coming from sin/cos computation for large numbers (> 117435.992f) in the C standard library used in that setup. These tests pass for Skylake client and Skylake server CPU's (which verify the FMA and AVX512F implementation of sin and cos). One way to confirm would be to check is the test passes if you remove the following line:

x_f32[index] = np.float32(10E+10*np.random.rand(M))

rgommers · 2019-12-30T06:02:12Z

Thanks for the suggestion @r-devulap!

h-vetinari · 2021-05-09T22:56:45Z

I tried exploratively removing the skip for that test (which hasn't changed in the conda-forge recipe since then) in conda-forge/numpy-feedstock#227, and the failure is still there.

=================================== FAILURES ===================================
_______________ TestAVXFloat32Transcendental.test_sincos_float32 _______________

self = <numpy.core.tests.test_umath.TestAVXFloat32Transcendental object at 0x7f2b74eee990>

    def test_sincos_float32(self):
        np.random.seed(42)
        N = 1000000
        M = np.int_(N/20)
        index = np.random.randint(low=0, high=N, size=M)
        x_f32 = np.float32(np.random.uniform(low=-100.,high=100.,size=N))
        # test coverage for elements > 117435.992f for which glibc is used
        x_f32[index] = np.float32(10E+10*np.random.rand(M))
        x_f64 = np.float64(x_f32)
>       assert_array_max_ulp(np.sin(x_f32), np.float32(np.sin(x_f64)), maxulp=2)
E       AssertionError: Arrays are not almost equal up to 2 ULP (max difference is 67 ULP)

M          = 50000
N          = 1000000
index      = array([121958, 671155, 131932, ..., 738271, 310195, 233966])
self       = <numpy.core.tests.test_umath.TestAVXFloat32Transcendental object at 0x7f2b74eee990>
x_f32      = array([-10.577719, -35.353283, -97.29114 , ..., -80.99214 , -42.875526,
       -87.8052  ], dtype=float32)
x_f64      = array([-10.57771873, -35.35328293, -97.2911377 , ..., -80.99214172,
       -42.87552643, -87.80519867])

from numpy/numpy#15179 (comment)

h-vetinari · 2021-05-11T19:46:26Z

It is likely that this is coming from sin/cos computation for large numbers (> 117435.992f) in the C standard library used in that setup. These tests pass for Skylake client and Skylake server CPU's (which verify the FMA and AVX512F implementation of sin and cos). One way to confirm would be to check is the test passes if you remove the following line:

x_f32[index] = np.float32(10E+10*np.random.rand(M))

Tried this in conda-forge/numpy-feedstock#227 with the following diff (see conda-forge/numpy-feedstock@e6e9ff9):

diff --git a/numpy/core/tests/test_umath.py b/numpy/core/tests/test_umath.py
index 858dac471..744111d87 100644
--- a/numpy/core/tests/test_umath.py
+++ b/numpy/core/tests/test_umath.py
@@ -1039,8 +1039,6 @@ class TestAVXFloat32Transcendental:
         M = np.int_(N/20)
         index = np.random.randint(low=0, high=N, size=M)
         x_f32 = np.float32(np.random.uniform(low=-100.,high=100.,size=N))
-        # test coverage for elements > 117435.992f for which glibc is used
-        x_f32[index] = np.float32(10E+10*np.random.rand(M))
         x_f64 = np.float64(x_f32)
         assert_array_max_ulp(np.sin(x_f32), np.float32(np.sin(x_f64)), maxulp=2)
         assert_array_max_ulp(np.cos(x_f32), np.float32(np.cos(x_f64)), maxulp=2)

and indeed, the tests pass.

@r-devulap @rgommers, what's the takeaway from this? Should the glibc-dependent part of the test be spun off into a separate test (with a certain skip-condition)?

r-devulap · 2021-05-11T20:53:34Z

@h-vetinari I think so. But I am wondering what would the skip condition be though?

h-vetinari · 2021-05-11T20:57:31Z

Can we say something about the glibc-version used in CI here? The conda-forge anvil-image is pretty old (CentOs6 based) and uses glibc 2.12.

r-devulap · 2021-05-14T03:13:43Z

I should be able to use os.confstr('CS_GNU_LIBC_VERSION') to check the GLIBC version, but I am not sure what the minimum version should be. Any idea?

mattip · 2021-05-14T07:45:30Z

FWIW, we are still shipping wheels to be used with glibc2.12, at least until we drop manylinux2010. Glibc version for the different wheels we build (the latest release adds the aliases to the file names to make this clear):

wheel	glibc
manylinux2010	glibc 2.12
manylinux2014	glibc 2.17

here is a nice summary of GLIBC version for various linux distributions.

Fixes numpy#15179

commit 30228578ac65ec6b90961700a5783c23d8e1b879 Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Wed Nov 17 16:21:27 2021 -0800 Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt Add Windows config to GHA update script [wheel build] typo [wheel build] fix typo? [wheel build] fix linux builds? [wheel build] typo [wheel build] add license and pin to windows 2016 skip tests [wheel build] pin to windows 2019 instead [wheel build] try to find out the error on windows [wheel build] maybe fix? [wheel build] maybe fix? [wheel build] fix? [wheel build] cleanup [wheel build] Update LICENSE_win32.txt Update LICENSE_win32.txt commit 4bd12df Author: Thomas Li <47963215+lithomas1@users.noreply.github.com> Date: Mon Nov 15 17:28:47 2021 -0800 # This is a combination of 14 commits. # This is the 1st commit message: Add Windows config to GHA # This is the commit message numpy#2: update script [wheel build] # This is the commit message numpy#3: typo [wheel build] # This is the commit message numpy#4: fix typo? [wheel build] # This is the commit message numpy#5: fix linux builds? [wheel build] # This is the commit message numpy#6: typo [wheel build] # This is the commit message numpy#7: add license and pin to windows 2016 # This is the commit message numpy#8: skip tests [wheel build] # This is the commit message numpy#9: pin to windows 2019 instead [wheel build] # This is the commit message numpy#10: try to find out the error on windows [wheel build] # This is the commit message numpy#11: maybe fix? [wheel build] # This is the commit message numpy#12: maybe fix? [wheel build] # This is the commit message numpy#13: fix? [wheel build] # This is the commit message numpy#14: cleanup [wheel build] commit 444a721 Merge: 376ad69 22448b4 Author: Charles Harris <charlesr.harris@gmail.com> Date: Mon Nov 15 17:47:23 2021 -0700 Merge pull request numpy#20274 from h-vetinari/fix_15179 TST: Some fixes & refactoring around glibc-dependent skips in test_umath.py commit 376ad69 Merge: b75fe57 546c47a Author: Charles Harris <charlesr.harris@gmail.com> Date: Mon Nov 15 17:31:41 2021 -0700 Merge pull request numpy#20327 from seberg/rename-interpolation BUG,DEP: Fixup quantile/percentile and rename interpolation->method commit 546c47a Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 15 16:13:50 2021 -0600 DOC: Fixups for interpolation rename comments from review Co-authored-by: Charles Harris <charlesr.harris@gmail.com> commit b75fe57 Merge: 7310c09 cbc25d2 Author: Warren Weckesser <warren.weckesser@gmail.com> Date: Mon Nov 15 17:27:08 2021 -0500 Merge pull request numpy#20369 from bilderbuchi/missing_diagnostics_newlines MAINT: Fix newlines in diagnostics output of numpy.f2py. commit cbc25d2 Author: Christoph Buchner <bilderbuchi@phononoia.at> Date: Sun Nov 14 08:36:03 2021 +0100 MAINT: Fix newlines in diagnostics output of numpy.f2py. Linebreaks were not consistently added to errmess/outmess arguments, which led to very long lines and wrong concatenation with compiler messages in f2py console output. commit be15716 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Fri Nov 12 12:10:20 2021 -0600 DOC: Add release not for quantile `interpolation` rename Also updates the old release note to include the info (and generally tweaks it a bit) commit 7d8a8e7 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Fri Nov 12 11:57:22 2021 -0600 DOC: Update percentile/quantile docs Mainly fixes the method list slightly, tones down the warning a bit and fixes the link to the paper (I did not realize that the link failed to work due only because the reference was missing from nanquantile/nanpercentile). commit 5bd71fb Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Tue Nov 9 09:48:59 2021 -0600 DOC: Add ticks to quantile interpolation/method error Co-authored-by: abel <aoun@cerfacs.fr> commit 0d5fb81 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 20:39:50 2021 -0600 DOC: Remove reference to paper from quantile `method` kwarg Apparently, sphinx does not resolve references to footnotes from parameter descriptions. commit 8437663 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 18:25:37 2021 -0600 MAINT: Rename interpolation to method in percentile stubs commit 8cfb6b5 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:47:27 2021 -0600 TST: Add deprecation testcase for quantile interpolation rename commit a5ac5a5 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:41:24 2021 -0600 DOC: Fixup the percentile methods plot commit 85f3dda Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 16:37:41 2021 -0600 BUG: quantile discrete methods ended up using -1 as index sometimes Also, the closest-observation did not correctly support multiple quantiles calculated at the same time (broadcasting error). commit 3993408 Author: Sebastian Berg <sebastian@sipsolutions.net> Date: Mon Nov 8 15:38:30 2021 -0600 API,DEP: Rename percentile/quantile `interpolation=` to `method=` commit 22448b4 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 16:51:59 2021 +1100 TST: parametrize glibc-version check in test_umath.py commit 12923c2 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:21:01 2021 +1100 TST: skip coverage of large elements in sincos_float32 for ancient glibc Fixes numpy#15179 commit 56268d5 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:19:24 2021 +1100 TST: use existence of glibc-version to clean up a test commit 01443e8 Author: H. Vetinari <h.vetinari@gmx.com> Date: Tue Nov 2 14:18:52 2021 +1100 TST: turn glibc_older_than_2.17 into boolean instead of xfail this anticipates reuse of this boolean within the test module

rgommers added 00 - Bug component: numpy._core labels Dec 26, 2019

h-vetinari added a commit to h-vetinari/numpy-feedstock that referenced this issue May 11, 2021

try suggestion from numpy issue for fixing test failure

e6e9ff9

from numpy/numpy#15179 (comment)

h-vetinari mentioned this issue May 27, 2021

remove obsolete test skip conda-forge/numpy-feedstock#235

Merged

h-vetinari mentioned this issue Jun 7, 2021

Test suite failure on non-AVX512 due to glibc (version-dependent) bug on f64 underflow #19192

Closed

h-vetinari mentioned this issue Nov 2, 2021

TST: Some fixes & refactoring around glibc-dependent skips in test_umath.py #20274

Merged

h-vetinari added a commit to h-vetinari/numpy that referenced this issue Nov 2, 2021

TST: skip coverage of large elements in sincos_float32 for ancient glibc

12923c2

Fixes numpy#15179

charris closed this as completed in #20274 Nov 16, 2021

h-vetinari mentioned this issue Nov 24, 2021

BUG: Test failures with 1.22.0rc1 on non-AVX512 hardware (with old glibc) #20448

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TestAVXFloat32Transcendental test failure #15179

TestAVXFloat32Transcendental test failure #15179

rgommers commented Dec 26, 2019

rgommers commented Dec 26, 2019 •

edited

Loading

mattip commented Dec 26, 2019

rgommers commented Dec 26, 2019

r-devulap commented Dec 26, 2019

rgommers commented Dec 30, 2019

h-vetinari commented May 9, 2021

h-vetinari commented May 11, 2021

r-devulap commented May 11, 2021

h-vetinari commented May 11, 2021

r-devulap commented May 14, 2021

mattip commented May 14, 2021

TestAVXFloat32Transcendental test failure #15179

TestAVXFloat32Transcendental test failure #15179

Comments

rgommers commented Dec 26, 2019

rgommers commented Dec 26, 2019 • edited Loading

mattip commented Dec 26, 2019

rgommers commented Dec 26, 2019

r-devulap commented Dec 26, 2019

rgommers commented Dec 30, 2019

h-vetinari commented May 9, 2021

h-vetinari commented May 11, 2021

r-devulap commented May 11, 2021

h-vetinari commented May 11, 2021

r-devulap commented May 14, 2021

mattip commented May 14, 2021

rgommers commented Dec 26, 2019 •

edited

Loading