bins='auto' just gets passed to numpy's histogram https://numpy.org/doc/stable/reference/generated/numpy.histogram_bin_edges.html#numpy.histogram_bin_edges
If that returns a million bins you are going to have trouble plotting them.

I have no idea what a levy distribution is, but I assume its best not histogrammed with linear bins, so you probably need to pass it a set fixed of nonlinearly spaced bins, or transform the data to something more linear and then histogram it.

For a non-sparse example:

fig, ax = plt.subplots()
ax.hist(10**(np.random.randn(1000)*2), bins='auto')
plt.show()

brings my machine to halt and clearly shouldn't be binned with linear bins.

Please see scipy/scipy#14563 for context.

Sure, but what can matplotlib do to help here?

I'm not sure, but @tacaswell may have something in mind. I posted at his request.

There have been requests to make 'auto' our default (in fact the work on auto in numpy started as a Matplotlib PR that we asked the contributor to move upstream!).

Either this is something we should have a check to warn / fail on,

This pathological example is failing because it has ~427k bins

In [11]: list(map(len, np.histogram([1, 2, 3, 4, 1e6], bins='auto')))
Out[11]: [427494, 427495]

which means in the default mode of ax.hist we are trying to draw 427k rectangles, most of which are 0 height! One of the bottlenecks is just making that many Rectangle instances (it takes ~minutes and 4.7G of ram) and it will render (slowly), but because the bins are a tiny fraction of a pixel wide....they are invisible:

The options I see:

1. do nothing, if the user asks for auto, they get auto and can wait
2. add a warning if the number of bins produced by auto (or probably passed in by the user) is >10k we issue a user visible warning saying "you have asked to draw more bins than we suspect you have pixels for. We are going to try but you have been warned!"
3. add an exception and raise if we can detect it is probably going to go badly

I'm currently between 1 and 2 (and maybe a bit closer to 2).

I also think that this means making auto the default of hist is a non-starter. We could add more layers of heuristics on top to try the different auto algorithms until one has "few enough" bins. I think we would want to push that back to numpy to add 'auto-min-bins' or some way of constraining the algorithms.

I guess a 4th option is use "stair" by default if there are over X bins, where X is something like the screen resolution?

I'm currently between 1 and 2 (and maybe a bit closer to 2).

I'd be in favor of a warning. Using that many bins is almost never a good solution, and if somebody really needs it, they should really not try to dray individual rectangles. - Maybe don't hard-code to 10k but to N times number of horizontal pixels. e.g. N = 3, which is clearly oversampling.

I guess a 4th option is use "stair" by default if there are over X bins, where X is something like the screen resolution?

I have some concerns with dynamically switching the output type depending on inputs and screen size.

I guess a 5th option would be to change the default to use filled stairs. - Basically we made stairs to be able to easily draw np.histogram outputs. Should we not use it internally as well?
While this is an API change and we need a deprecation cycle, I doubt many people mess with the individual bar rectangles (I cannot imagine a reasonalbe use-case).

Maybe don't hard-code to 10k but to N times number of horizontal pixels. e.g. N = 3, which is clearly oversampling.

Unfortunately we need to know this at artist creation time when we may not yet know the actual size of the output yet.

For reasons I do not understand yet

ax.hist([1, 2, 3, 4, 1e6], bins='auto', histtype='step')

is still slow (but not as slow) to create the artist. I suspect we are doing some loops in Python that we could be vectorizing.
On the other hand

ax.stairs(*np.histogram([1, 2, 3, 4, 1e6], bins='auto'))

is quick (and because it is drawing the edge is actually visible as the edge is fixed width).

I thought we did not go with option 5 when we added the stairs is because the return types are too different?

Yeah, I think making stairs the default will be hard as it will break a lot of people.

There are two (at least) problems

1. sometimes bars are too thin to be plotted
2. plotting many bars is really slow.

The first seems a bit weird to me, verging on a bug. I feel like this has been discussed before, but why does a thin bar not have any visual output?

For the case where 1 and 2 are a problem, say for nbins=10k or 100k, warning and using stair is in my opinion a reasonable thing to do that is back-compatible because it just broke gracelessly before, and better than just warning and then hanging or throwing an out-of-memory error.

I suspect we are doing some loops in Python that we could be vectorizing.

probably

and

I thought we did not go with option 5 when we added the stairs is because the return types are too different?

ARAICS we did not discuss this. hist() is only mentioned a few times in the PR #18275 and the related issuse. I suppose we focuessed primarily on getting the API and underlying Artist itself right.

No matter what, we should add two new variants to plt.hist(..., histtype=...) named 'stairs' and 'stairs-filled' (or 'filled-stairs'). It's reasonalbe to provide users with that functionality in hist(), and it's the fast one.

We can separately decide whether we want to auto-switch for large bins, or maybe change the default later on.

See also #12493 re: default artist type. Back then there was only step, not stairs, but the idea is basically the same.

More context in which the problem of "auto" producing catastrophically small bin widths is discussed:

• Request: change hist bins default to 'auto' #16403
• Feature request: change hist bins default to 'auto' numpy/numpy#15569

Numpy seemed open to adding a bins="auto-capped" (or whatever) option, but momentum stalled.