In the case where everything is one color and 1 marker size I think we take a short-cut and fallback to the code path used for plot which renders the marker once and then 'stamps' it on to the canvas enough times. In the more general case every marker gets rendered, colored, and then clipped (as in general every marker could be a different color and size).

If you have order 50 colors and are not using the continuous size variability of scatter I suggest using a loop + plot with no line style.

@mdboom suggests this would be not so easy to do with Agg (#2156 (comment)). I don't know Agg's internal, but this issue is solved by https://github.com/anntzer/mpl_cairo (... if you can manage to install it :-))

@tacaswell In the typical case where all markers are the same size, there is no need to render markers that cannot be inside the viewport. Also, when zooming in, there is no need to render markers which were previously not visible (zooming in can only eliminate markers from the viewport, not introduce them).

I tried your suggestion of calling scatter() in a loop with a single color at a time:

N = 3000000
C = 10
x = np.random.random(N)
y = np.random.random(N)
coloridx = np.random.randint(0, C, N)
colors = np.random.random((C, 3))

# old way: plt.scatter(x, y, 1, colors.take(coloridx), '.')

for ii in range(C):
    mask = ii == coloridx
    plt.scatter(x[mask], y[mask], 1, colors[ii], '.')

plt.show()

It is indeed much faster, though it produces a different Z order (before zooming in, all markers appear to be the same color, because they were drawn in color order).

I can live with the Z order difference since it provides so much better performance. Thank you for the suggestion. Now that we know how to do it, what do you think about making Matplotlib enable this efficient behavior in a more obvious way? For example, the API could take a new type of color map:

plt.scatter(x, y, 1, coloridx, '.', colortable=colors) # hypothetical API

The hypothetical colortable parameter would require the c parameter to be an array of integer indexes, and the colors would then be looked up. This is similar to ColorMap but discrete.

One could also imagine transparently accelerating the existing API by first checking if the number of distinct values in c is much smaller than the number of data points, and reusing stamps if so.

I would quite strongly oppose adding an API that provides an approximate behavior to work around a less-than-optimal implementation.

@anntzer I too would prefer that the existing API simply be made more efficient. Right now, its performance is quite terrible for one of its most common use cases.

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

Generating the stamp in monochrome (or rather, alpha-only) and coloring it on-demand should definitely help; in mplcairo this is implemented in PatternCache::mask and ultimately relies on cairo_mask (https://www.cairographics.org/manual/cairo-cairo-t.html#cairo-mask) to do the coloring of the alpha mask.

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!