The problem here seems to do with plotting very small numbers. with max_x = 1000, it gives only two non-zero values out of 300, 1.72083369e-260 and 2.05807755e-291; and with max_x = 1100 only one, 7.90845000e-317. Naturally if you change, 300 to 301 then the error does not occur, but that goes beside the point. Matplotlib should not give such errors when plotting small numbers.

Anyway for completeness I get the following warnings for both plt.show and plt.savefig

transforms.py:2558: RuntimeWarning: overflow encountered in double_scalars
  y_scale = 1.0 / inh
transforms.py:2560: RuntimeWarning: invalid value encountered in double_scalars
  [0.0    , y_scale, (-inb*y_scale)],

with savefig additionally throwing the assertion error

transforms.py, line 2107, in __init__
    assert x_transform.is_separable

import matplotlib.pyplot as plt
from scipy.stats import norm as scipy_stats_norm
from math import sqrt
import numpy as np

fig, ax = plt.subplots()

mean = -0.0938
stdev = sqrt(0.008812)
max_x = 1100  # this fails
#max_x = 1000  # this works

x = np.linspace(-max_x, max_x, 300) 
y = scipy_stats_norm.pdf(x, loc=mean, scale=stdev)

ax.plot(x, y)
ax.set_xlim(-max_x, max_x)

#filename = '/tmp/fail.png'
#plt.savefig(filename, bbox_inches='tight')
plt.show()

Oddly, the error seems to be coming out of drawing the tick labels.

The issue here seems to be that this is pushing the limits of floats. On my system I only get 1 non-zero value which is

In [47]: np.max(y)
Out[47]: 7.9084499991689783e-317

which is notably smaller than what http://docs.scipy.org/doc/numpy/reference/generated/numpy.MachAr.html#numpy.MachAr claims the smallest float should be which makes me think something awful is going on with non-normal floats (and I am now out of my depth on the numerics side of things.).

I think the solution is to make sure we do a better job making sure that auto-limits are non-singular.

Adding ax.set_ylim(0, 1e-300) 'fixes' this problem.

Hmm, slightly different numbers, what system do you use? I work on debian jessie 64bit python 2.7.9

Ubuntu beta (varient?, something with a v) but using py3k from anaconda.

and 32bit or 64bit?

oh, I see what you mean...

In [1]: import numpy as np

In [2]: np.MachAr(float).xmin
Out[2]: 2.2250738585072014e-308

So, a possible bug in numpy? Combined with a possible bug in matplotlib. I mean if numpy gives us numbers that shouldn't exist, then that would explain why our calculations fail. I think we should first get some clarification from numpy about this and then decide how to patch it (atm I think doing a test based around (x < np.MachAr().xmin).any(), but I would want to hear from numpy first).

I remember seeing an issue go by on numpy a while ago which was talking
about 'denormal' floats (http://en.wikipedia.org/wiki/Denormal_number) so I
don't think it is a bug in numpy. If the top and bottom of the range are
the same we have some code that make the range non-singular, but it seems
that this is falling in the gap of what triggers that code path and what
works.

On Wed, Apr 8, 2015 at 8:43 AM OceanWolf notifications@github.com wrote:

and 32bit or 64bit?

oh, I see what you mean...

In [1]: import numpy as np

In [2]: np.MachAr(float).xmin
Out[2]: 2.2250738585072014e-308

So, a possible bug in numpy? Combined with a possible bug in matplotlib. I
mean if numpy gives us numbers that shouldn't exist, then that would
explain why our calculations fail. I think we should first get some
clarification from numpy about this and then decide how to patch it (atm I
think doing a test based around (x < np.MachAr().xmin).any(), but I want
to hear from numpy first).

—
Reply to this email directly or view it on GitHub
#4318 (comment)
.

Thanks for the link, so definitly a matplotlib bug, but I still think a bug in numpy exists over MachAr... I have just submitted a bug on this to numpy numpy/numpy#5755.

@tacaswell forgive my ignorance, but what made you say that it came from the ticks? The overflow error appears from doing a transform for drawing a line:

lines.py, line 712, in draw
  tpath, affine = transf_path.get_transformed_path_and_affine()

Nothing changes with respect to this bug when I add:

ax.set_xticks([])
ax.set_yticks([])

Pardon my poor knowledge of the draw code, I only know the backends well atm.

The trace back is raised from in side a text draw inside an axis draw
inside an axes draw.

It would also be worth looking if the same thing happens with one of the
vector backend s

On Wed, Apr 8, 2015, 12:40 OceanWolf notifications@github.com wrote:

@tacaswell https://github.com/tacaswell forgive my ignorance, but what
made you say that it came from the ticks? The overflow error appears from
doing a transform for drawing a line:

lines.py, line 712, in draw
tpath, affine = transf_path.get_transformed_path_and_affine()

This still occurs with the error even if I add:

ax.set_xticks([])
ax.set_yticks([])

Pardon my poor knowledge of the draw code, I only know the backends well
atm.

—
Reply to this email directly or view it on GitHub
#4318 (comment)
.

Ahh, the traceback from there, I looked at the first problem, the overflow error, so I stuck a traceback in that line of code... that error happens in BboxTransformFrom.get_matrix and does 1/<y_canvas_max> but as subnormal only extends the minimum exponent, we get an overflow when taking the reciprocal.

This BboxTransformFrom.get_matrix transforms all points from grid space to canvas space, and thus gets called when plotting the line.

So yes, I think we need to change the code to detect singularity from something like if dy == 0: raise Error to if dy < 1/np.MachAr(float).xmax: raise Error. I currently hunt for that line that tests singularity, I have found some very close to the source but only do so on DEBUG, and have now just found a line in axes._base.set_ylim that looks like it, but putting in print statements and passing in a true singular set of data to plot, (i.e. not just from a computational perspective), it has already done so before, so now I will trace up until I find the source of the de-singularisation of the bounding box :).

desingularization happens in the autoscaling, IIRC. So, it would be prior
to the draw()/show().

On Wed, Apr 8, 2015 at 3:14 PM, OceanWolf notifications@github.com wrote:

Ahh, the traceback from there, I looked at the first problem, the overflow
error, so I stuck a traceback in that line of code... that error happens in
BboxTransformFrom.get_matrix and does 1/ but as subnormal only extends
the minimum exponent, we get an overflow when taking the reciprocal.

This BboxTransformFrom.get_matrix transforms all points from grid space
to canvas space, and thus gets called when plotting the line.

So yes, I think we need to change the code to detect singularity from
something like if dy == 0: raise Error to if dy <
1/np.MachAr(float).xmax: raise Error. I currently hunt for that line that
tests singularity, I have found some very close to the source but only do
so on DEBUG, and have now just found a line in axes._base.set_ylim that
looks like it, but putting in print statements and passing in a true
singular set of data to plot, (i.e. not just from a computational
perspective), it has already done so before, so now I will trace up until I
find the source of the de-singularisation of the bounding box :).

—
Reply to this email directly or view it on GitHub
#4318 (comment)
.

Ahh yes, found it, the transforms.nonsingular function.

Yes, this line, I don't get it:

if vmax - vmin <= max(abs(vmin), abs(vmax)) * tiny:

shouldn't it just read:

if vmax - vmin <= 1/np.MachAr(float).xmax:

what do I miss?

For plotting purposes, we don't necessarily want to go to the minimum possible fp number; to be meaningful, a range of numbers has to be bigger than that. Also, if and when the ultimate small value check is appropriate, wouldn't one use the xmin attribute instead of 1/xmax?

When the autoscaling check occurs, what are the values of vmax and vmin? If they are both denormalized, is their difference a NaN, in which case the test would evaluate to False and the singularity would not be detected? If this is the case, the solution is to form the vmax-vmin difference first, and flag the singularity based on not np.isfinite(vrange).

@efiring as far as I can tell denormalised numbers work fine for all operations, the problem here comes from overflow caused by the asymmetry introduced by allowing denormalised numbers, i.e., that 1/smallest_positive_number >> largest_positive_number, because denomalisation only applies to the small numbers, the transform uses 1/(vmax - vmin) as a scaling factor.

Thus to ensure we don't cause an overflow in the scaling factor we need to ensure:

1/(vmax - vmin) <= np.MachAr(float).xmax

or rearranging:

vmax - vmin <= 1/np.MachAr(float).xmax

Of course we may want to add additional wiggle room there, something like:

vmax - vmin <= factor/np.MachAr(float).xmax

Yes, it works! But I needed to use a factor. Basically the method above prevents the overflow warning, but later on in the savefig routine, another transform gets used that multiplies 1/vmax-vmin by the number of pixels, otherwise we get a np.inf when trying to multiply a large number close to the max, by the dimension in pixels, which in turn gives np.nan when multiplied by 0 (in the identity matrix).

Thus when I test with a factor = 1e4 it works. i.e. we must use a factor equal to or greater than the width/height in pixels.

Not sure why it doesn't throw an error during show(), only savefig, but I shall save that for someone else. I feel confident that other solutions also exist (perhaps through some internal rescaling to move numbers out of the danger zone).

@OceanWolf Is there a PR that goes with this?

@tacaswell not yet, I have only made the diagnosis, and I don't have time at the moment to make a decent fix (i.e. one that doesn't limit subnormals), see my last comment for ideas.

Punted as this seems like a corner case and looks involved to fix.

@OceanWolf, by "doesn't limit subnormals", do you mean allowing plot ranges that are in the subnormal domain? We don't need to do this. All we need is a robust system for covering reasonably small ranges, and never generating an exception. This is a matter of plotting library default autoscaling strategy, not ultimate numerics.
The basic problem here is that the max abs of vmin and vmax is too small; I think that what we need is an additional check for that condition, not a modification of the basic expansion algorithm. In other words, the problem is that the first check being done in nonsingular is not stringent enough; it is only checking for inf and nan, but it should be requiring something along the lines of max abs vmin and vmax being greater than 1e5 times MachAr(float).xmin. I will try a quick PR.

@OceanWolf, Thanks for your diagnoses and explanations, which were very helpful. Also, I didn't know about np.MachAr() until you pointed it out.

@efiring rereading, I meant that we need do need to properly deal with small ranges, and if we do that I think we can safely plot subnormals for plotting.

I think we just need to some linear rescaling, x' = mx + c, on par with what we do at the moment for ranges, i.e similar to what we already have with this code (at least visually, I haven't looked at the internals):

from matplotlib import pyplot as plt
plt.plot([1000, 1001, 1002])  # scales with m = 1, c = -1000
plt.show()

I was able to reproduce the issue:

#!/usr/bin/env python3

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

d = {'job': list(range(1, 3)),
     'duration': [83066.639344, 820.700000],
     'type': ['A', 'B']}

df = pd.DataFrame(d)

plot = sns.catplot(x="duration", y="job", data=df, hue='type',
                   color="b", kind="bar", height=3, aspect=4)

ax = plot.axes.flat[0]
for p in plt.gca().patches:
    ax.text(p.get_width(),
            p.get_y() + p.get_height() / 2,
            p.get_width())

plot.savefig("barplot.png")

Here's the full stacktrace:

posx and posy should be finite values
posx and posy should be finite values
/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/numpy/core/fromnumeric.py:83: RuntimeWarning: invalid value encountered in reduce
  return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
posx and posy should be finite values
posx and posy should be finite values
posx and posy should be finite values
Traceback (most recent call last):
  File "/Users/dzieciou/projects/example/gocd/reproduce.py", line 31, in <module>
    plot.savefig("barplot.png")
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/seaborn/axisgrid.py", line 37, in savefig
    self.fig.savefig(*args, **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/figure.py", line 2094, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2075, in print_figure
    **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 510, in print_png
    FigureCanvasAgg.draw(self)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 402, in draw
    self.figure.draw(self.renderer)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/figure.py", line 1649, in draw
    renderer, self, artists, self.suppressComposite)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/image.py", line 138, in _draw_list_compositing_images
    a.draw(renderer)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 2610, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/image.py", line 138, in _draw_list_compositing_images
    a.draw(renderer)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/axis.py", line 1185, in draw
    ticks_to_draw = self._update_ticks(renderer)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/axis.py", line 1023, in _update_ticks
    tick_tups = list(self.iter_ticks())  # iter_ticks calls the locator
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/axis.py", line 967, in iter_ticks
    majorLocs = self.major.locator()
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/ticker.py", line 1985, in __call__
    return self.tick_values(vmin, vmax)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/ticker.py", line 1993, in tick_values
    locs = self._raw_ticks(vmin, vmax)
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/ticker.py", line 1932, in _raw_ticks
    nbins = np.clip(self.axis.get_tick_space(),
  File "/Users/dzieciou/virtualenvs/seaborn/lib/python3.7/site-packages/matplotlib/axis.py", line 2543, in get_tick_space
    return int(np.floor(length / size))
ValueError: cannot convert float NaN to integer

Any workaround for it?

@dzieciou Your issue is that you create a text at a nan position. A minimal example would be

import numpy as np
import matplotlib.pyplot as plt

plt.text(0, np.nan, "My Text")

plt.savefig("barplot.png", bbox_inches="tight")
plt.show()

Workarounds:

• Don't try to position text at undefined positions.
• Don't use bbox_inches="tight" (the default for seaborn), but rather plot.savefig("barplot.png", bbox_inches=None).

@ImportanceOfBeingErnest, your comment saved me some time tracking this down! Since the issue was data dependent in my case and I had several plt.text(), this wrapper was useful:

# plots text raising ArgumentError for NaN positions, avoiding "cryptic" error:
#  https://github.com/matplotlib/matplotlib/issues/4318#issuecomment-436266884
def pltText(x, y, s, **kwargs):
  if np.isnan(x + y):
    raise ArgumentError(x, y)
  return plt.text(x, y, s, **kwargs)

Maybe plt.text() should do something similar by default; it currently appears to allow what plt.savefig() cannot handle. (I tested only with matplotlib 2.2.4, though, the latest version pip updated to.)

Just to be clear, this bug doesn't exist in master anymore because we now check the bboxes for finite-values before we add them to the tightbbox calculations...

Great. I should have checked master...

No problem! I just wanted it on the record here so folks don’t try to track this bug down.

This error also shows when using tight_layout with nan value.
Here's an example:

import matplotlib
import matplotlib.pyplot as plt
import numpy as np

plt.rcParams['figure.figsize'] = 10,10

class MidpointNormalize(matplotlib.colors.Normalize):
    def __init__(self, vmin=None, vmax=None, midpoint=None, clip=False):
        self.midpoint = midpoint
        matplotlib.colors.Normalize.__init__(self, vmin, vmax, clip)

    def __call__(self, value, clip=None):
        # Note that I'm ignoring clipping and other edge cases here.
        result, is_scalar = self.process_value(value)
        x, y = [self.vmin, self.midpoint, self.vmax], [0, 0.5, 1]
        return np.ma.array(np.interp(value, x, y), mask=result.mask, copy=False)

# make these smaller to increase the resolution
dx, dy = 0.15, 0.05

# generate 2 2d grids for the x & y bounds
y, x = np.mgrid[slice(-3, 3 + dy, dy),
                slice(-3, 3 + dx, dx)]
z = (1 - x / 2. + x ** 5 + y ** 3) * np.exp(-x ** 2 - y ** 2)

z = z[:-1, :-1]
z_min, z_max = -np.abs(z).max(), np.abs(z).max()
z [0:2,0:2] = np.nan

plt.pcolormesh(x, y, z, cmap='seismic',norm=MidpointNormalize(midpoint=0.,vmin=-3,vmax=3))
plt.title('pcolormesh')
plt.axis([x.min(), x.max(), y.min(), y.max()])
plt.colorbar(extend='min')

# plt.tight_layout(w_pad=0.1, h_pad=1)
plt.savefig('test.png', dpi = 720)

When I comment out plt.tight_layout(w_pad=0.1, h_pad=1), I got this:

E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\colorbar.py:1042: RuntimeWarning: invalid value encountered in true_divide
  z = np.take(y, i0) + (xn - np.take(b, i0)) * dy / db
E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\transforms.py:402: RuntimeWarning: invalid value encountered in double_scalars
  return points[1, 0] - points[0, 0]
Traceback (most recent call last):
  File "C:\Users\Xin\Desktop\test_extend.py", line 37, in <module>
    plt.savefig('test.png', dpi = 720)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\pyplot.py", line 689, in savefig
    res = fig.savefig(*args, **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\figure.py", line 2094, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\backend_bases.py", line 2075, in print_figure
    **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\backends\backend_agg.py", line 510, in print_png
    FigureCanvasAgg.draw(self)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\backends\backend_agg.py", line 402, in draw
    self.figure.draw(self.renderer)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\figure.py", line 1649, in draw
    renderer, self, artists, self.suppressComposite)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\image.py", line 138, in _draw_list_compositing_images
    a.draw(renderer)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\axes\_base.py", line 2628, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\image.py", line 138, in _draw_list_compositing_images
    a.draw(renderer)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\artist.py", line 50, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\axis.py", line 1185, in draw
    ticks_to_draw = self._update_ticks(renderer)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\axis.py", line 1023, in _update_ticks
    tick_tups = list(self.iter_ticks())  # iter_ticks calls the locator
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\axis.py", line 967, in iter_ticks
    majorLocs = self.major.locator()
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\ticker.py", line 1985, in __call__
    return self.tick_values(vmin, vmax)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\ticker.py", line 1993, in tick_values
    locs = self._raw_ticks(vmin, vmax)
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\ticker.py", line 1932, in _raw_ticks
    nbins = np.clip(self.axis.get_tick_space(),
  File "E:\miniconda3\envs\satpy\lib\site-packages\matplotlib\axis.py", line 2159, in get_tick_space
    return int(np.floor(length / size))
ValueError: cannot convert float NaN to integer

The solution:

Add vmin and vmax:

plt.pcolormesh(x, y, z, vmin=z_min, vmax=z_max, cmap='seismic',norm=MidpointNormalize(midpoint=0.,vmin=-3,vmax=3))

@zxdawn Can you please open a new issue with that example and include the version information?

@tacaswell OK, submit to #15093 .