Skip to content

Issues with Numpy 1.11 release candidate #5950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jenshnielsen opened this issue Jan 31, 2016 · 21 comments
Closed

Issues with Numpy 1.11 release candidate #5950

jenshnielsen opened this issue Jan 31, 2016 · 21 comments
Milestone

Comments

@jenshnielsen
Copy link
Member

I tried running the test suite for 1.5.1 on numpy 1.11 beta 2 and I am seeing a single failure that we should investigate.

Python 3.5 OSX

======================================================================
FAIL: matplotlib.tests.test_ticker.test_AutoMinorLocator
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 118, in wrapped_function
    func(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/tests/test_ticker.py", line 50, in test_AutoMinorLocator
    assert_almost_equal(ax.xaxis.get_ticklocs(minor=True), test_value)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 517, in assert_almost_equal
    return assert_array_almost_equal(actual, desired, decimal, err_msg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 912, in assert_array_almost_equal
    precision=decimal)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/testing/utils.py", line 688, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals

(shapes (21,), (22,) mismatch)
 x: array([ 0.05,  0.1 ,  0.15,  0.25,  0.3 ,  0.35,  0.45,  0.5 ,  0.55,
        0.65,  0.7 ,  0.75,  0.85,  0.9 ,  0.95,  1.05,  1.1 ,  1.15,
        1.25,  1.3 ,  1.35])
 y: array([ 0.05,  0.1 ,  0.15,  0.25,  0.3 ,  0.35,  0.45,  0.5 ,  0.55,
        0.65,  0.7 ,  0.75,  0.85,  0.9 ,  0.95,  1.  ,  1.05,  1.1 ,
        1.15,  1.25,  1.3 ,  1.35])

----------------------------------------------------------------------
@jenshnielsen
Copy link
Member Author

Furthermore it raises 2 new future warnings very frequently:

/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/ma/core.py:3113: 
FutureWarning: Currently, slicing will try to return a view of the data, but will return a copy of the mask. 
In the future, it will try to return both as views.

and

/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/numpy/ma/core.py:3192: 
FutureWarning: Currently, slicing will try to return a view of the data, but will return a copy of the mask. 
In the future, it will try to return both as views. This means that using `__setitem__` will propagate 
values back through all masks that are present.

We should probably suppress them somehow.

@jenshnielsen
Copy link
Member Author

I have not tested 2.x or master yet

@efiring
Copy link
Member

efiring commented Jan 31, 2016

Both the "expected" test values and those that are obtained are wrong. The algorithm is intended to remove minor ticks that coincide with major ticks, but the expected values in the test include 1, which is a major tick. The test is now failing because floating point arithmetic is causing the erroneous algorithm to break in a different place.
@anntzer, would you like to look at this? I think you are the current master of dealing with such floating point problems.
Longer term, we might consider how minor ticks should be handled in a larger context. If they should be prevented from landing on major ticks, then this should be handled systematically and uniformly, not as a special case in AutoMinorLocator.

@anntzer
Copy link
Contributor

anntzer commented Jan 31, 2016

I would say that the old test was actually written in a way to accomodate a bug in matplotlib (probably itself stemming from a bug in numpy... numpy/numpy#7022 is the obvious suspect here): if you only plot minor ticks

xlim(0, 1.39)
gca().minorticks_on()
gca().tick_params("x", length=0)

you see that there is a spurious minor tick at x=1 (but not at the other major tick positions). I don't know exactly how to handle this in the tests while maintaining backcompatibility but from a correctness POV it is pretty clear that the correct test_value should not include 1 (and the actually returned values are now correct).

@seberg
Copy link
Contributor

seberg commented Feb 5, 2016

If you have time, maybe you could check how much numpy/numpy#7187 helps with the future warning spam? If it is almost gone, you can maybe work around it (it is a bit ugly in any case).
Frankly, if one downstream package like matplotlib has to silence the warning globally, I see basically no point in even having it in the first place.

@jenshnielsen
Copy link
Member Author

@seberg Thanks I will try to have a look over the weekend

@anntzer
Copy link
Contributor

anntzer commented Feb 10, 2016

wrt. the test failure I'd suggest switching between testing for the new value (which is after all the correct one...) and the old value depending on the version of numpy.

@jenshnielsen
Copy link
Member Author

Numpy 10.11 beta 3 is now released on pypi so our allow failures job with python 3.6 and --pre is now picking it up i.e. https://travis-ci.org/matplotlib/matplotlib/jobs/107907120

@tacaswell tacaswell added this to the 2.0.1 (next bug fix release) milestone Feb 15, 2016
@jenshnielsen
Copy link
Member Author

The issues with the numpy release candidate have changed a bit and I now see:

======================================================================
FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test
    '(RMS %(rms).3f)'%err)
matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected.png (RMS 19.050)

======================================================================
FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test
    '(RMS %(rms).3f)'%err)
matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small_pdf.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected_pdf.png (RMS 27.854)

======================================================================
FAIL: matplotlib.tests.test_axes.test_formatter_large_small.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test
    '(RMS %(rms).3f)'%err)
matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small_svg.png vs. /Users/jhn/src/python/matplotlib/result_images/test_axes/formatter_large_small-expected_svg.png (RMS 18.371)

======================================================================
FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test
    '(RMS %(rms).3f)'%err)
matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster.png vs. /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster-expected.png (RMS 14.603)

======================================================================
FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 220, in do_test
    '(RMS %(rms).3f)'%err)
matplotlib.testing.exceptions.ImageComparisonFailure: images not close: /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster_pdf.png vs. /Users/jhn/src/python/matplotlib/result_images/test_bbox_tight/bbox_inches_tight_raster-expected_pdf.png (RMS 14.634)

======================================================================
FAIL: matplotlib.tests.test_bbox_tight.test_bbox_inches_tight_raster.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/decorators.py", line 210, in do_test
    self._tol, in_decorator=True)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/compare.py", line 331, in compare_images
    rms = calculate_rms(expectedImage, actualImage)
  File "/Users/jhn/Envs/numpy111betamatplotlib151/lib/python3.5/site-packages/matplotlib/testing/compare.py", line 248, in calculate_rms
    "actual size {1}".format(expectedImage.shape, actualImage.shape))
matplotlib.testing.exceptions.ImageComparisonFailure: image sizes do not match expected size: (449, 575, 3) actual size (450, 575, 3)

----------------------------------------------------------------------

@jenshnielsen
Copy link
Member Author

The changes are in auto limits in these 2 tests (duplicated in png, svg and pdf)

This seems to mainly be due to a change in the implementation of divmod in numpy.

For example:

Numpy 1.10

In [1]: divmod(1.0, 0.2)
Out[1]: (4.0, 0.19999999999999996)

In [2]: divmod(np.float64(1.0), 0.2)
Out[2]: (5.0, 0.19999999999999996)

In [2]: divmod(1.0, np.float64(0.2))
Out[2]: (5.0, 0.19999999999999996)

In [2]: divmod(np.float64(1.0), np.float64(0.2))
Out[2]: (5.0, 0.19999999999999996)

Numpy 1.11rc2

In [1]: divmod(np.float64(1.0), 0.2)
Out[1]: (4.0, 0.19999999999999996)

divmod is used among other places within ticker.MaxNLocator.bin_boundaries which is responsible for calculating the min and max boundaries in autoscaling.

@jenshnielsen
Copy link
Member Author

The numpy 1.10 result is obviously wrong in this case since 5.0 * 0.2 + 0.19999999999999996 = 1.2 and not 1 as expected.

@jenshnielsen jenshnielsen changed the title Issues with Numpy 1.11 beta2 Issues with Numpy 1.11 release candidate Mar 20, 2016
@anntzer
Copy link
Contributor

anntzer commented Mar 20, 2016

Pretty sure this is at least partially handled by #5768 (which includes a local fix to divmod, and led me to comment with that specific example on numpy's bug tracker).

@jenshnielsen
Copy link
Member Author

@anntzer Looks good. I somehow missed that PR, I think we should make a push for getting this in.

@tacaswell
Copy link
Member

With #5768 merged the test_bbox_inches_tight_raster failures are gone.

@jenshnielsen
Copy link
Member Author

The remaining issues with test_formatter_large_small is apprently due to change in floor division which is used within the new _raw_ticks method.

Numpy 1.10

In [1]: 0.40000000099999999//2e-10
Out[1]: 2000000004.0

In [2]: np.float64(0.40000000099999999)//2e-10
Out[2]: 2000000005.0

Numpy 1.11rc2

In [2]: 0.40000000099999999//2e-10
Out[2]: 2000000004.0

In [3]: np.float64(0.40000000099999999)//2e-10
Out[3]: 2000000004.0

@QuLogic
Copy link
Member

QuLogic commented Apr 2, 2016

There seemed to have been a few bugs in the betas; have you tried again with final 1.11?

@tacaswell
Copy link
Member

I get the second behavior with 1.11 from conda.

tacaswell added a commit to tacaswell/matplotlib that referenced this issue Apr 2, 2016
This test only over passed due to an error arising from a bug
in numpy's divmod being fixed (numpy/numpy#6127).

See matplotlib#5950
tacaswell added a commit to tacaswell/matplotlib that referenced this issue Apr 2, 2016
This test only over passed due to an error arising from a bug
in numpy's divmod being fixed (numpy/numpy#6127).

See matplotlib#5950
@tacaswell
Copy link
Member

This is another fallout from numpy/numpy#6127 which #6192 was working around. In those cases the incorrect rounding was hurting us, in this case it was helping us.

This is the situation for np 1.10

In [11]: divmod(0.500000001, 2e-10)
Out[11]: (2500000004.0, 1.9999995350196985e-10)

In [12]: divmod(np.asarray(0.500000001), 2e-10)
Out[12]: (2500000005.0, 1.9999995350196985e-10)

In [13]: divmod(np.asarray(0.500000001), 2e-10)[1] - 2e-10
Out[13]: -4.6498030157606349e-17

so we are clearly hitting the edges of 64bit float precision and the cpython / np1.11 values are clearly correct.

In [21]: 0.500000001 == ((divmod(0.500000001, 2e-10)[0] * 2e-10) + divmod(0.500000001, 2e-10)[1])
Out[21]: True

In [22]: 0.500000001 == ((1 + divmod(0.500000001, 2e-10)[0]) * 2e-10)
Out[22]: True

In [23]: divmod(0.500000001, 2e-10)[1] ==  2e-10
Out[23]: False

tacaswell added a commit to tacaswell/matplotlib that referenced this issue Apr 3, 2016
This test only over passed due to an error arising from a bug
in numpy's divmod being fixed (numpy/numpy#6127).

See matplotlib#5950
@tacaswell
Copy link
Member

have we fixed these issues?

@jenshnielsen
Copy link
Member Author

AFAIK we are skipping the relevant tests on 2.x and master i.e. 26cba4b

On 1.5.x I think there are a few additional tests we need to skip. (they are probably failing in the nightly build on travis which installs directly from pypi and skips the wheelhouse.

@QuLogic QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0.2 (next bug fix release) May 3, 2017
@anntzer
Copy link
Contributor

anntzer commented Jul 25, 2017

Closing as (AFAICT) the only remaining thing to do is to remove the version check on numpy once we only support numpy 1.11+ (but there will be plenty of these).

@anntzer anntzer closed this as completed Jul 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants