Skip to content

BUG: Test failures with 1.22.0rc1 on non-AVX512 hardware (with old glibc) #20448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
h-vetinari opened this issue Nov 24, 2021 · 4 comments · Fixed by #20558
Closed

BUG: Test failures with 1.22.0rc1 on non-AVX512 hardware (with old glibc) #20448

h-vetinari opened this issue Nov 24, 2021 · 4 comments · Fixed by #20558
Assignees
@r-devulap
Labels
00 - Bug

Comments

@h-vetinari
Copy link
Contributor

h-vetinari commented Nov 24, 2021 •
edited
Loading

While building for conda-forge, I noticed that the first run had some failures on linux that appeared only in one out of three runs (py38/py39/py310).

From previous experience that such failures occasionally can depend on absence/presence of CPU features (and the fact that the azure agents are randomly assigned but have different capabilities), I checked whether that was the case, and indeed, the failures occurred only for non-AVX512 machines.

I went to some length to validate this by forcing runs to explicitly have both either AVX512 or not (by failing and restarting if otherwise), and indeed, the following two tests then fail consistently in the non-AVX512 case:

=========================== short test summary info ============================
FAILED core/tests/test_umath.py::TestSpecialFloats::test_exp2 - AssertionErro...
FAILED core/tests/test_umath_accuracy.py::TestAccuracy::test_validate_transcendentals
= 2 failed, 17280 passed, 848 skipped, 1293 deselected, 20 xfailed, 5 xpassed in 212.59s (0:03:32) =

Here's a link to the CI run for that.

Finally, given previous issues with #15179 / #19192, I then checked if things fail with a "newer" glibc (2.17 in CentOS 7), and then, the failures are gone, see this run.

From the discussions in those previous glibc-related issues, I take that the appetite to spend time on these things is low, but I'd hope that skipping those tests (based on the glibc version check introduced in #19209) would be acceptable?

=================================== FAILURES ===================================
_________________________ TestSpecialFloats.test_exp2 __________________________

self = <numpy.core.tests.test_umath.TestSpecialFloats object at 0x7feb4e16ab00>

    def test_exp2(self):
46Z         with np.errstate(all='ignore'):
47Z             in_ = [np.nan, -np.nan, np.inf, -np.inf]
13Z             out = [np.nan, np.nan, np.inf, 0.0]
12Z             for dt in ['f', 'd']:
97Z                 in_arr = np.array(in_, dtype=dt)
03Z                 out_arr = np.array(out, dtype=dt)
73Z                 assert_equal(np.exp2(in_arr), out_arr)
41Z
75Z         for value in [2000.0, -2000.0]:
58Z             with np.errstate(over='raise', under='raise'):
30Z                 for dt in ['f', 'd']:
03Z >                   assert_raises(FloatingPointError, np.exp2,
17Z                                   np.array(value, dtype=dt))
39Z
42Z dt         = 'd'
87Z in_        = [nan, nan, inf, -inf]
76Z in_arr     = array([ nan,  nan,  inf, -inf])
96Z out        = [nan, nan, inf, 0.0]
32Z out_arr    = array([nan, nan, inf,  0.])
85Z self       = <numpy.core.tests.test_umath.TestSpecialFloats object at 0x7feb4e16ab00>
29Z value      = -2000.0
59Z
47Z ../[...]/lib/python3.10/site-packages/numpy/core/tests/test_umath.py:1277:
03Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
76Z ../[...]/lib/python3.10/unittest/case.py:730: in assertRaises
27Z     return context.handle('assertRaises', args, kwargs)
08Z         args       = (<ufunc 'exp2'>, array(-2000.))
32Z         context    = None
55Z         expected_exception = <class 'FloatingPointError'>
60Z         kwargs     = {}
55Z         self       = <numpy.testing._private.utils._Dummy testMethod=nop>
62Z ../[...]/lib/python3.10/unittest/case.py:200: in handle
80Z     with self:
85Z         args       = [array(-2000.)]
92Z         callable_obj = <ufunc 'exp2'>
41Z         kwargs     = {}
89Z         name       = 'assertRaises'
07Z         self       = None
32Z ../[...]/lib/python3.10/unittest/case.py:223: in __exit__
90Z     self._raiseFailure("{} not raised by {}".format(exc_name,
85Z         exc_name   = 'FloatingPointError'
17Z         exc_type   = None
59Z         exc_value  = None
00Z         self       = <unittest.case._AssertRaisesContext object at 0x7feb4e16a920>
22Z         tb         = None
64Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
61Z
27Z self = <unittest.case._AssertRaisesContext object at 0x7feb4e16a920>
45Z standardMsg = 'FloatingPointError not raised by exp2'
05Z
17Z     def _raiseFailure(self, standardMsg):
49Z         msg = self.test_case._formatMessage(self.msg, standardMsg)
25Z >       raise self.test_case.failureException(msg)
17Z E       AssertionError: FloatingPointError not raised by exp2
53Z
32Z msg        = 'FloatingPointError not raised by exp2'
21Z self       = <unittest.case._AssertRaisesContext object at 0x7feb4e16a920>
24Z standardMsg = 'FloatingPointError not raised by exp2'
89Z
73Z ../[...]/lib/python3.10/unittest/case.py:163: AssertionError
80Z __________________ TestAccuracy.test_validate_transcendentals __________________
79Z
23Z self = <numpy.core.tests.test_umath_accuracy.TestAccuracy object at 0x7feb4e05ea40>
26Z
14Z     @platform_skip
57Z     def test_validate_transcendentals(self):
42Z         with np.errstate(all='ignore'):
92Z             data_dir = path.join(path.dirname(__file__), 'data')
64Z             files = os.listdir(data_dir)
93Z             files = list(filter(lambda f: f.endswith('.csv'), files))
24Z             for filename in files:
66Z                 filepath = path.join(data_dir, filename)
68Z                 with open(filepath) as fid:
24Z                     file_without_comments = (r for r in fid if not r[0] in ('$', '#'))
70Z                     data = np.genfromtxt(file_without_comments,
19Z                                          dtype=('|S39','|S39','|S39',int),
34Z                                          names=('type','input','output','ulperr'),
80Z                                          delimiter=',',
52Z                                          skip_header=1)
76Z                     npname = path.splitext(filename)[0].split('-')[3]
51Z                     npfunc = getattr(np, npname)
15Z                     for datatype in np.unique(data['type']):
04Z                         data_subset = data[data['type'] == datatype]
51Z                         inval  = np.array(str_to_float(data_subset['input'].astype(str), data_subset['type'].astype(str)), dtype=eval(datatype))
93Z                         outval = np.array(str_to_float(data_subset['output'].astype(str), data_subset['type'].astype(str)), dtype=eval(datatype))
60Z                         perm = np.random.permutation(len(inval))
87Z                         inval = inval[perm]
27Z                         outval = outval[perm]
27Z                         maxulperr = data_subset['ulperr'].max()
08Z >                       assert_array_max_ulp(npfunc(inval), outval, maxulperr)
45Z E                       AssertionError: Arrays are not almost equal up to 4 ULP (max difference is 65 ULP)
56Z
88Z data       = array([(b'np.float32', b'0xfd97ece0', b'0xc11186e9', 4),
17Z        (b'np.float32', b'0x8013bb34', b'0x8013bb34', 4),
46Z     ...84', b'0x3fd0aa740f394c2a', 4)],
64Z       dtype=[('type', 'S39'), ('input', 'S39'), ('output', 'S39'), ('ulperr', '<i8')])
61Z data_dir   = '/home/conda/feedstock_root/build_artifacts/numpy_1637731623243/_test_env_placehold_placehold_placehold_placehold_plac...cehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.10/site-packages/numpy/core/tests/data'
41Z data_subset = array([(b'np.float32', b'0xfd97ece0', b'0xc11186e9', 4),
21Z        (b'np.float32', b'0x8013bb34', b'0x8013bb34', 4),
12Z     ...0x7e1301c7', b'0xbf0ee63e', 4)],
95Z       dtype=[('type', 'S39'), ('input', 'S39'), ('output', 'S39'), ('ulperr', '<i8')])
08Z datatype   = b'np.float32'
80Z fid        = <_io.TextIOWrapper name='/home/conda/feedstock_root/build_artifacts/numpy_1637731623243/_test_env_placehold_placehold_..._placehold_/lib/python3.10/site-packages/numpy/core/tests/data/umath-validation-set-tan.csv' mode='r' encoding='UTF-8'>
96Z file_without_comments = <generator object TestAccuracy.test_validate_transcendentals.<locals>.<genexpr> at 0x7feb4e06b4c0>
83Z filename   = 'umath-validation-set-tan.csv'
94Z filepath   = '/home/conda/feedstock_root/build_artifacts/numpy_1637731623243/_test_env_placehold_placehold_placehold_placehold_plac...acehold_placehold_placehold_placehold_/lib/python3.10/site-packages/numpy/core/tests/data/umath-validation-set-tan.csv'
71Z files      = ['umath-validation-set-arcsinh.csv', 'umath-validation-set-arcsin.csv', 'umath-validation-set-arccos.csv', 'umath-validation-set-log.csv', 'umath-validation-set-arccosh.csv', 'umath-validation-set-exp2.csv', ...]
55Z inval      = array([ 5.77792631e-39, -6.69404060e-39, -5.71724832e-01,  8.80537391e-01,
59Z        -1.23391426e+38,  1.13332816e-38, -7...007841e-38, -4.91161418e-39,  6.94417715e-01, -2.77933895e+38,
76Z         1.48694371e+38, -5.16387463e-01], dtype=float32)
24Z maxulperr  = 4
06Z npfunc     = <ufunc 'tan'>
05Z npname     = 'tan'
84Z outval     = array([ 5.77792631e-39, -6.69404060e-39, -6.43404722e-01,  1.21098876e+00,
49Z         7.21419334e+00,  1.13332816e-38, -9...007841e-38, -4.91161418e-39,  8.32790315e-01, -1.58004427e+00,
76Z         1.82193553e+00, -5.67774892e-01], dtype=float32)
62Z perm       = array([685, 182, 376, 688, 691, 695, 494, 324, 647, 169, 390, 517, 453,
41Z        397, 666, 171, 260,  79, 634, 283, 186,...,  15, 609, 496, 163, 568, 699, 296, 605, 313, 711,
27Z        669, 342, 595, 587, 133,  13,  49, 619, 110, 285, 407, 652])
28Z self       = <numpy.core.tests.test_umath_accuracy.TestAccuracy object at 0x7feb4e05ea40>
17Z
10Z ../[...]/lib/python3.10/site-packages/numpy/core/tests/test_umath_accuracy.py:58: AssertionError
@h-vetinari h-vetinari mentioned this issue Nov 24, 2021
Closed
@r-devulap r-devulap self-assigned this Nov 26, 2021
@r-devulap
Copy link
Member

From the discussions in those previous glibc-related issues, I take that the appetite to spend time on these things is low, but I'd hope that skipping those tests (based on the glibc version check introduced in #19209) would be acceptable?

Sounds good to me. I will have a patch to fix this.

@rgommers
Copy link
Member

Makes sense to me too to simply skip for too old glibc.

@h-vetinari h-vetinari mentioned this issue Nov 26, 2021
Closed
@h-vetinari
Copy link
Contributor Author

@r-devulap, are you still planning (or going to have time) to come up with a patch? Otherwise I can give it a shot as well.

@r-devulap
Copy link
Member

Hey, apologies for the delay (in the middle of a chaotic vacation). Please go ahead if you can.

@h-vetinari h-vetinari mentioned this issue Dec 10, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@r-devulap r-devulap

Labels
00 - Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

TST: move get_glibc_version to np.testing; skip 2 more tests for old glibc h-vetinari/numpy
3 participants
@rgommers @h-vetinari @r-devulap