Optimize 3D display #6085

AlexandreAbraham · 2016-03-01T09:42:06Z

While printing 3D objects, I noticed that the display was pretty slow. The reason is that matplotlib relies a lot on list of objects instead of using big numpy arrays to fasten 3D projections.

This PR aims at solving part of this problem. I replaced some airthmetic on list of vectors by artihmetic on big matrices. This impacts mainly 3D display.

I work on neuroimaging data. I noticed an approximate 25% speedup optimization on 3D display (depending on the complexity of the figure). Refreshment in interactive plotting is also faster. Example script:

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from nibabel import gifti


surf_mesh = '/path/to/spm12b/canonical/cortex_20484.surf.gii'

coords, faces = gifti.read(surf_mesh).getArraysFromIntent(1008)[0].data, \
                gifti.read(surf_mesh).getArraysFromIntent(1009)[0].data

limits = [coords.min(), coords.max()]

cmap = 'coolwarm'

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d', xlim=limits, ylim=limits)
ax.view_init(elev=0, azim=0)
ax.set_axis_off()

p3dcollec = ax.plot_trisurf(coords[:, 0], coords[:, 1], coords[:, 2],
                            triangles=faces, linewidth=0.,
                            antialiased=False,
                            color='white')

fig.savefig('brain.png')
plt.close(fig)

Possible other optimizations:

For display, matplotlib relies on a list of Path objects. This could be made faster with numpy arrays.
Poly3DCollection and others: a lot of unnecessary asarray and paddings of ones to obtain quaternions are done. This could be simplified by keeping a quaternion numpy array in the object and using appropriate numpy views but requires a deeper refactoring.
not plotting unseen triangle: For the moment, all polygons are drawn. if there are a lot, isn't it worth to compute which one are hidden by others and remove them?

@agramfort and @juhuntenburg, do you notice significant running time reduction with this PR?

tacaswell · 2016-03-01T15:59:53Z

attn @WeatherGod

WeatherGod · 2016-03-01T16:04:47Z

You have a lot of test failures: https://travis-ci.org/matplotlib/matplotlib/jobs/112810572

WeatherGod · 2016-03-01T16:08:27Z

lib/mpl_toolkits/mplot3d/art3d.py

-        minz = 1e9
-        for (xs, ys, zs) in xyslist:
-            minz = min(minz, min(zs))
+        minz = np.min(xys[:, :, 2])


We are not guaranteed a non-empty array, hence the initialization of minz with 1e9.

kingjr · 2016-05-13T15:41:25Z

@AlexandreAbraham any progress on this front?

choldgraf · 2016-05-13T16:46:33Z

+1 to this. I still haven't found a good method for plotting 3-D brains w/ points on top of it in python. Making a trisurf is super slow when I have lots of triangles.

AlexandreAbraham · 2016-05-23T08:53:12Z

Hi,
Yes, I have understood why the tests were failing. I have unfortunately made some assumptions on the format of input data that turned out to be wrong. The optimization is still possible but requires more work. I had no time so far to fix it but I'll give it a try today.

tacaswell · 2016-07-16T20:15:40Z

'power cycled' to restart the CI against current master.

jenshnielsen · 2016-07-18T16:16:24Z

Looks like there is a problem in one of the examples

Warning, treated as error:
/home/travis/build/matplotlib/matplotlib/doc/examples/mplot3d/pathpatch3d_demo.rst:8: WARNING: Exception occurred in plotting pathpatch3d_demo
 from /home/travis/build/matplotlib/matplotlib/doc/mpl_examples/mplot3d/pathpatch3d_demo.py:
Traceback (most recent call last):
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/sphinxext/plot_directive.py", line 654, in render_figures
    figman.canvas.figure.savefig(img.filename(format), dpi=dpi)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/figure.py", line 1666, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backend_bases.py", line 2227, in print_figure
    **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backends/backend_agg.py", line 517, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/backends/backend_agg.py", line 464, in draw
    self.figure.draw(self.renderer)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/artist.py", line 68, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/figure.py", line 1262, in draw
    renderer, self, dsu, self.suppressComposite)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py", line 279, in draw
    for patch in self.patches]
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py", line 279, in <listcomp>
    for patch in self.patches]
  File "/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/art3d.py", line 320, in do_3d_projection
    s = np.vstack(self._segments3d, np.ones(self._segments3d.shape[1]))
AttributeError: 'PathPatch3D' object has no attribute '_segments3d'

anntzer · 2016-09-21T17:01:17Z

I guess my point is more "let's not add more public API".

NelleV · 2016-12-18T15:57:40Z

Hi @AlexandreAbraham
I think this is almost ready to be merged. Can you make the new public function private and rebase? The rebase might be quite painful…
If you are too busy with the new job, I can take over your branch :)

mwaskom · 2017-07-12T14:05:53Z

Hi, I ended up here from this paper: https://riojournal.com/articles.php?id=12342

I'd be very interested in using the method in the paper, but I'm finding the speed that it can be accomplished with the current implementation of 3d meshses is a dealbreaker. Is there hope for this PR moving forward?

choldgraf · 2017-07-12T14:11:12Z

+1 to this PR moving forward

WeatherGod · 2017-07-13T14:13:47Z

ok, I'll make it a priority this week during SciPy.

…

On Wed, Jul 12, 2017 at 10:11 AM, Chris Holdgraf ***@***.***> wrote: +1 to this PR moving forward — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-Lv2kePQIHFLTlgb98P4Wy33BQm1ks5sNNQCgaJpZM4HmSzk> .

tacaswell · 2017-07-13T16:09:14Z

I think that this is the PR that @GaelVaroquaux mentioned to me earlier this week.

choldgraf · 2017-07-14T15:28:51Z

lmk if you want someone to usertest this at the sprint. I can bring some brain surfaces :-)

WeatherGod · 2017-07-15T16:30:05Z

There are several unaddressed comments from @Kojoley . I am currently working on rebasing this as well. I'll force-push a rebase soon (if github allows me).

WeatherGod · 2017-07-15T23:03:31Z

doesn't look like github will let me, so, I'll do it the old way and submit a PR against your branch.

WeatherGod · 2017-07-15T23:09:19Z

hmmph, that won't work, either. @AlexandreAbraham , do you see any option to allow maintainers to push changes to your PR on this page?

eric-wieser · 2017-07-19T23:19:29Z

lib/mpl_toolkits/mplot3d/art3d.py

+            self._segments3d_data = np.vstack(segments)
+            # Add a fourth dimension for quaternions
+            self._segments3d_data = np.hstack([self._segments3d_data,
+                                               np.ones((n_segments, 1))])


This would be marginally better as

self._segments3d_data = np.empty((n_segments, 4)) self._segments3d_data[:,:3] = segments self._segments3d_data[:,3] = 1

Also, are you sure you mean "quaternion" and not "homogenous coordinate"?

eric-wieser · 2017-07-19T23:20:18Z

lib/mpl_toolkits/mplot3d/art3d.py

+        cum_s = 0
+        for s in self._seg_sizes:
+            segments_2d.append(xys[cum_s:cum_s + s, :2])
+            cum_s += s


I think this loop is np.split in some form

eric-wieser · 2017-07-19T23:25:44Z

lib/mpl_toolkits/mplot3d/proj3d.py

+        tis = vecw[1] < 1
+    vecw /= vecw[3]
+    # Integrating tis in the numpy array for optimization purposes
+    vecw[3, :] = tis


Get rid of this colon - it makes the function fail on 1d inputs

eric-wieser

Storing coordinates in the first axis of the array is kind of unusual (see how np.linalg and scipy.misc.imread behave).

Perhaps these should all be xyz[..., 0] instead of xyz[0]?

jklymak · 2018-05-02T17:10:46Z

This seems to have attracted lots of interest, but eventually never made it over the hump. @WeatherGod is there still some hope for this, or has its momment passed?

WeatherGod · 2018-05-02T17:17:47Z

Perhaps parts of this needs to be separated out into smaller PRs. There is some good stuff here, but there were also some questionable stuff, too, that never got addressed.

…

On Wed, May 2, 2018 at 1:10 PM, Jody Klymak ***@***.***> wrote: This seems to have attracted lots of interest, but eventually never made it over the hump. @WeatherGod <https://github.com/WeatherGod> is there still some hope for this, or has its momment passed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6085 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-ENj7x-jsaEbU7onzowiCV_Giuu8ks5tueiYgaJpZM4HmSzk> .

jklymak · 2018-05-02T17:28:27Z

Closing as inactive. The goodies are here for anyone who wants them!

rth · 2018-07-05T12:08:14Z

This seems to have attracted lots of interest, but eventually never made it over the hump
Closing as inactive.

If it's closed, what's the mechanism then for contributors to know that this PR is stalled, needs help and can be continued ?

do you see any option to allow maintainers to push changes to your PR on this page?

Should be allowed by Github by default now.

I'll make it a priority this week during SciPy. [in 2017]

Maybe someone might be interested in looking into it during the scipy sprint next week?

rth · 2018-07-05T12:25:26Z

If I understand correctly this PR rebased (as of July 2017) can be found in @WeatherGod 's https://github.com/WeatherGod/matplotlib/tree/pr/6085 branch.

rth · 2018-07-15T21:56:46Z

This PR was continued in #11577

mdboom added the status: needs review label Mar 1, 2016

WeatherGod reviewed Mar 1, 2016
View reviewed changes

WeatherGod added status: needs revision topic: mplot3d and removed status: needs review labels Mar 1, 2016

AlexandreAbraham added 9 commits June 8, 2016 22:55

Replace lists by numpy array in projection

a4a8a86

Remove profiling code

3d295af

Vectorize output of proj_transform_vec

5239e1a

Reach vectorized projection functions directly

7285fed

Address comments

328c2f4

More fixes

a973da8

Fix rotation of axes

14853d0

Fix some failures

c4de047

Use a contiguous representation in memory and views for the segments

6315961

AlexandreAbraham force-pushed the optimize_3d branch from a3bdaf8 to 6315961 Compare June 8, 2016 20:55

tacaswell closed this Jul 16, 2016

tacaswell reopened this Jul 16, 2016

tacaswell added status: needs review and removed status: needs revision labels Jul 16, 2016

tacaswell added this to the 2.1 (next point release) milestone Jul 16, 2016

AlexandreAbraham added 2 commits September 20, 2016 10:37

Fix bug in PathPatch

bb948bb

Fix vstack bug

e6211bd

NelleV self-assigned this Dec 18, 2016

tacaswell modified the milestones: 2.2 (next next feature release), 2.1 (next point release) Jul 16, 2017

eric-wieser reviewed Jul 19, 2017

View reviewed changes

eric-wieser suggested changes Jul 19, 2017

View reviewed changes

jklymak modified the milestones: needs sorting, v3.0 May 2, 2018

jklymak added the status: needs rebase label May 2, 2018

jklymak closed this May 2, 2018

QuLogic removed the status: needs review label May 2, 2018

QuLogic removed this from the v3.0 milestone May 2, 2018

rth pushed a commit to rth/matplotlib that referenced this pull request Jul 5, 2018

Optimize 3D display (matplotlib#6085)

2a75026

rth mentioned this pull request Jul 5, 2018

[MRG] Optimize 3D display (continued) #11577

Closed

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimize 3D display #6085

Optimize 3D display #6085

AlexandreAbraham commented Mar 1, 2016

tacaswell commented Mar 1, 2016

WeatherGod commented Mar 1, 2016

WeatherGod Mar 1, 2016

kingjr commented May 13, 2016

choldgraf commented May 13, 2016

AlexandreAbraham commented May 23, 2016

tacaswell commented Jul 16, 2016

jenshnielsen commented Jul 18, 2016

anntzer commented Sep 21, 2016

NelleV commented Dec 18, 2016

mwaskom commented Jul 12, 2017

choldgraf commented Jul 12, 2017

WeatherGod commented Jul 13, 2017 via email

tacaswell commented Jul 13, 2017

choldgraf commented Jul 14, 2017

WeatherGod commented Jul 15, 2017

WeatherGod commented Jul 15, 2017

WeatherGod commented Jul 15, 2017

eric-wieser Jul 19, 2017

eric-wieser Jul 19, 2017

eric-wieser Jul 19, 2017

eric-wieser left a comment

jklymak commented May 2, 2018

WeatherGod commented May 2, 2018 via email

jklymak commented May 2, 2018 •

edited

Loading

rth commented Jul 5, 2018

rth commented Jul 5, 2018

rth commented Jul 15, 2018

Optimize 3D display #6085

Optimize 3D display #6085

Conversation

AlexandreAbraham commented Mar 1, 2016

tacaswell commented Mar 1, 2016

WeatherGod commented Mar 1, 2016

WeatherGod Mar 1, 2016

Choose a reason for hiding this comment

kingjr commented May 13, 2016

choldgraf commented May 13, 2016

AlexandreAbraham commented May 23, 2016

tacaswell commented Jul 16, 2016

jenshnielsen commented Jul 18, 2016

anntzer commented Sep 21, 2016

NelleV commented Dec 18, 2016

mwaskom commented Jul 12, 2017

choldgraf commented Jul 12, 2017

WeatherGod commented Jul 13, 2017 via email

tacaswell commented Jul 13, 2017

choldgraf commented Jul 14, 2017

WeatherGod commented Jul 15, 2017

WeatherGod commented Jul 15, 2017

WeatherGod commented Jul 15, 2017

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

eric-wieser Jul 19, 2017

Choose a reason for hiding this comment

eric-wieser left a comment

Choose a reason for hiding this comment

jklymak commented May 2, 2018

WeatherGod commented May 2, 2018 via email

jklymak commented May 2, 2018 • edited Loading

rth commented Jul 5, 2018

rth commented Jul 5, 2018

rth commented Jul 15, 2018

jklymak commented May 2, 2018 •

edited

Loading