Skip to content

Add a ax.voxels(bool3d) function #6404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Aug 10, 2017
Merged

Conversation

eric-wieser
Copy link
Contributor

@eric-wieser eric-wieser commented May 11, 2016

I found myself needing a voxel plot to visualize a subset of RGB space, and thought that the plot was sufficiently useful that it would do well to belong upstream.

I'm not convinced the argument choices I've made line up too well with the rest of matplotlib, so I'd appreciate feedback on what I should be parameterizing this with.

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

Plotting a simple boolean array

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x, y, z = np.indices((10, 10, 10))
voxels = (x == y) | (y == z)
ax.voxels(voxels)

plt.show()

image

Attaching color information

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x, y, z = np.indices((10, 10, 10))
voxels = (x == y) | (y == z)
colors = np.zeros((10, 10, 10), dtype=np.object_)
colors.fill('b')
colors[(x<5) & (y < 5)] =  'r'
colors[(x + z) < 10] =  'g'
ax.voxels(voxels, colors)

plt.show()

image

Possible improvements:

  • Allow control of origin, scale, and rotation - possibly a transform argument of a (3,4) matrix? Done via x, y z arguments!
  • cmap support
  • np.ma.array variant that omits the filled argument
  • Return something sensible Done by returning a dict keyed by coordinate

@tacaswell tacaswell added this to the 2.1 (next point release) milestone May 12, 2016
@tacaswell
Copy link
Member

Cool!

I suggest returning the whole set of collections added to the axes.

This should probably get a test.

How does this do with alpha in the colors?

attn @WeatherGod

@eric-wieser
Copy link
Contributor Author

eric-wieser commented May 12, 2016

I suggest returning the whole set of collections added to the axes.

  • As a list/tuple?

  • As a 3-tuple of 3d numpy arrays indexed by location?

    data = np.zeros(10, 20, 30)
    xfaces, yfaces, zfaces = voxels(data)
    
    assert xfaces.shape == (11,20,30)
    assert yfaces.shape == (10,21,30)
    assert zfaces.shape == (10,20,31)
    
  • As a 3d-numpy array of sets indexed by bordering pixel?

This should probably get a test.

Not entirely sure how to do this, especially since I'm unable to pip install matplotlib from source on windows, so can't actually run this code

How does this do with alpha in the colors?

Untested. The machine I tested this on had an old enough version of matplotlib that poly.set_alpha(a) had no effect at all, which was weird.

It doesn't draw internal faces, so a solid cube surrounded by translucent cubes will be invisible. It also becomes hard to decide what color to draw the faces if alpha is involved.

@WeatherGod
Copy link
Member

Older versions of mplot3d does not handle alphas correctly, this was fixed
in v1.4.1, IIRC.

I'll look through this in the next few days.

On Thu, May 12, 2016 at 12:41 PM, Eric Wieser notifications@github.com
wrote:

I suggest returning the whole set of collections added to the axes.

  • As a list/tuple?

    As a 3-tuple of 3d numpy arrays indexed by location?

    data = np.zeros(10, 20, 30)
    xfaces, yfaces, zfaces = voxels(data)

    assert xfaces.shape == (11,20,30)
    assert yfaces.shape == (10,21,30)
    assert zfaces.shape == (10,20,31)

    As a 3d-numpy array of sets indexed by bordering pixel?

This should probably get a test.

Not entirely sure how to do this, especially since I'm unable to pip
install matplotlib from source on windows, so can't actually run this code

How does this do with alpha in the colors?
Untested. The machine I tested this on had an old enough version of
matplotlib that poly.set_alpha(a) had no effect at all, which was weird.

It doesn't draw internal faces, so a solid cube surrounded by translucent
cubes will be invisible. It also becomes hard to decide what color to draw
the faces if alpha is involved.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#6404 (comment)

@eric-wieser
Copy link
Contributor Author

eric-wieser commented May 12, 2016

Behaviour with alphas in a newer matplotlib:

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x, y, z = np.indices((10, 10, 10))
v1 = x == y
v2 = np.abs(x - y) < 2
colors = np.zeros((10, 10, 10, 4))
colors[v2] = [1, 0, 0, 0.5]
colors[v1] = [0, 1, 0, 0.5]
ax.voxels(v1 | v2, colors)

plt.show()

image

Note that the inner green voxels are not rendered, because we only render the surface

eric-wieser added a commit to eric-wieser/matplotlib that referenced this pull request May 31, 2016
@eric-wieser
Copy link
Contributor Author

Updated to return something

@eric-wieser
Copy link
Contributor Author

This build failure seems entirely unrelated

@jenshnielsen
Copy link
Member

2 of them are unreleated and known transient failures see #

However the one that builds the docs is probably related.

/home/travis/build/matplotlib/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py:docstring of mpl_toolkits.mplot3d.axes3d.Axes3D.set_zscale:43: WARNING: Inline interpreted text or phrase reference start-string without end-string.


Returns
-------
list of `Poly3DCollection
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, missed a ``` here

@jenshnielsen
Copy link
Member

It's most likely because you are missing a ``` after Poly3DCollection

@eric-wieser
Copy link
Contributor Author

Yep, just caught that too. Fixup'd and repushed

@eric-wieser
Copy link
Contributor Author

I don't think this error is real?

@eric-wieser
Copy link
Contributor Author

@WeatherGod, is this still in your queue?

@WeatherGod
Copy link
Member

Yeah, it still is in my queue. Just been punting on it as this would have
to go into 2.1, and we are still working on 2.0.

On Tue, Jul 19, 2016 at 3:51 AM, Eric Wieser notifications@github.com
wrote:

@WeatherGod https://github.com/WeatherGod, is this still in your queue?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#6404 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AARy-Pu-YTG0UsiX3fYiFsKPZ6DHbMG1ks5qXIH8gaJpZM4Icczh
.

@eric-wieser
Copy link
Contributor Author

Nice job on the 2.0 release!

Could you give me some direction on how to construct the image tests for this? Do I need to commit the PNGs as well? Do I need a specific rendering backend available for the tests to pass?

@eric-wieser
Copy link
Contributor Author

I'd still appreciate some tips on finishing this off - it would be great to get it in

@tacaswell tacaswell requested a review from WeatherGod June 30, 2017 18:16
@WeatherGod
Copy link
Member

WeatherGod commented Jul 2, 2017 via email

Copy link
Member

@WeatherGod WeatherGod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to see more clarification in the documentation before deciding how I feel about the return type. What is the rational for not keeping everything in one single Poly3DCollection? If everything was in one collection, we stand a chance to actually avoid "Escher Effects".

In addition, we need a "What's New" entry and unit tests.

if np.ndim(color) <= 1:
color, _ = np.broadcast_arrays(
color,
filled[np.index_exp[...] + np.index_exp[np.newaxis] * np.ndim(color)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What version of numpy was this introduced? I am not familiar with np.index_exp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since numpy/numpy@f897e1f back in 2002, so I assume this is fine. It's essentially the same as np.s_[...], but returns a tuple

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow... I have never seen this function before. Gonna have to remember this one.

Returns
-------
list of `Poly3DCollection`
The square faces which were plotted, in an arbitrary order
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From this description, it isn't clear what is the structure of the return object. Is it that each voxel is a Poly3DCollection of six faces, and that each element of the returned list is a voxel?

Also, "arbitrary order" is too ambiguous.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also add a .. verson added :: 2.1 (or whatever the right syntax is)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each element of the returned list is a single face.

Also, "arbitrary order" is too ambiguous.

The order is [XY-oriented] + [YZ-orientated] + [ZX-orientated], where each of those is in a lexicographic ordering over some permutation of their position.

The goal here was just to return a set of objects that borders etc could be put on, so the ordering is not documented.

I suppose in principle the return value could be a dict indexed by coordinate, containing a single Poly3DCollection for all the faces rendered for that voxel, but that would be more work

color : array_like
Either a single value or an array the same shape as filled,
indicating what color to draw the faces of the voxels. If None,
plot all voxels in the same color, the next in the color sequence
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, is this an array of strings? tuples?

color,
filled[np.index_exp[...] + np.index_exp[np.newaxis] * np.ndim(color)]
)
elif np.ndim(color) < 3:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not allow 2D arrays that could be broadcasted to 3D?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because that would make 3d arrays ambiguous.

The issue here is that I want to support colors[x,y,z] == 'the-color-name' (.shape=(X,Y,Z)) and colors[x,y,z] == [r,g,b] (.shape=(X,Y,Z,3)).

If we allowed broadcasting from 2D, then it's not clear whether to interpret a 3d array as .shape=(Y,Z,3) or .shape=(X, Y, Z).

I suppose I could switch on dtype, but I'm not clear what all the supported color formats are.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but doesn't the same ambiguity problem apply for 1-D?

boundary_found(p0 + square_rot, color[i0])

# draw middle faces
for r1, r2 in zip(rinds, rinds[1:]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this explicitly zip(rinds[:-1], rinds[1:]).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

if filled[i0]:
boundary_found(p0 + square_rot, color[i0])

# draw middle faces
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a bit more comments here explaining what the extra logic here is for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which extra logic are you referring to? The goal here is to not double-draw a face, which would cause clipping issues

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly that. "don't double-draw a face" is a nice, clear explanation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment added at the start of this loop

@@ -2649,6 +2649,114 @@ def calc_arrow(uvw, angle=15):

quiver3D = quiver

def voxels(self, filled, color=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no support here for specifying other properties such as edgecolors and edgewidth. Perhaps allow a **kwarg that can be any valid Poly3DCollection property?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with adding **kwarg is that it becomes harder to introduce new kwargs to voxels, in case they clash with Poly3DCollection properties. But I guess matplotlib has already gone down that route, so that doesn't seem unreasonable.

If I do that, then I should presumably rename color to facecolor, to make it clear that voxels is in control of that behaviour?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use bar3d() or hist3d() as a guide here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like good advice, thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, forwarding of kwargs is done, and there's a test

@eric-wieser
Copy link
Contributor Author

Thanks for the review! I'll go through your comments shortly

we need [...] unit tests.

Can you point me to some documentation on how best to do these? In particular, how do I generate the image that the tests compare against? Isn't that platform-dependent?

@WeatherGod
Copy link
Member

@eric-wieser
Copy link
Contributor Author

What are the odds of that working on Windows?

@tacaswell
Copy link
Member

@eric-wieser High, we run tests on appevyor 😄 and the same image tests pass on mac, windows, and linux.

The appevyor configuration files may be a good guide to getting a dev enviroment set up with conda.

@QuLogic
Copy link
Member

QuLogic commented Jul 16, 2017

If filled is required, then put it in the call signature. There's no need for the clever trick.

@eric-wieser
Copy link
Contributor Author

eric-wieser commented Jul 16, 2017

@QuLogic: Can you show me what you mean? I'm trying to make all of these cases work, as those are the cases allowed by the PEP 457 spec of voxels([x, y, z, ] /, filled=filled, **kwargs)

voxels(filled, **kwargs)
voxels(filled=filled, **kwargs)
voxels(x, y, z, filled, **kwargs)
voxels(x, y, z, filled=filled, **kwargs)

Additionally, this should fail:

voxels(x, y, z, filled, filled=filled2, **kwargs)
# TypeError: voxels() got multiple values for argument 'filled'

The clever trick does a very good job of providing this behaviour, and provides the right error messages on python 3 too. It also makes it super easy to add more both-positional-and-keyword arguments

@WeatherGod
Copy link
Member

The behavior you are looking for looks a lot like pcolor(). Perhaps look to that function for how we do that?

@anntzer
Copy link
Contributor

anntzer commented Jul 18, 2017

Just my 2c; I would prefer all these crazy argument parsing tricks to just use inspect.Signature.{bind,apply_defaults} (yes, that'd need the proper backport to be done).

@eric-wieser
Copy link
Contributor Author

eric-wieser commented Jul 18, 2017

I would prefer all these crazy argument parsing tricks to just use inspect.Signature

Unfortunately Signature doesn't implement PEP457 yet, so that doesn't help too much. Also, there's no inspect.Signature.from_string method, so we'd need to do something like sig = inspect.signature(lambda x, y, z, *, kwonly: pass). Of course, we can't backport that, because it uses py3 syntax. So now we need to write our own function get_sig("x, y, z, *, kwonly") to implement the py3 grammar!

I do think you're right that a generic argument-parsing helper might be handy, but I don't think that inspect.Signature is up to our needs.

Perhaps look to that function for how we do that?

pcolor does a worse job at processing arguments than voxels (comparing the C and filled arguments) in all of these cases:

pcolor()  # TypeError: Illegal arguments to pcolor; see help(pcolor)
voxels()  # TypeError: voxels() missing 1 required positional argument: 'filled'
pcolor(arr2d)  # ok
voxels(arr3d)  # ok
pcolor(C=arr2d)  # TypeError: Illegal arguments to pcolor; see help(pcolor)
voxels(filled=arr3d)  # ok
pcolor(arr2d, C=arr2d) # AttributeError: Unknown property c
voxels(arr3d, filled=arr3d) #TypeError:  voxels() got multiple values for argument 'filled'

@eric-wieser
Copy link
Contributor Author

eric-wieser commented Jul 18, 2017

Here's a possibly clearer implementation:

        # Use a lambda function to parse out the two forms of arguments.
        # Leading underscores are used to indicate positional-only arguments.
        if len(args) >= 3:
            parser = lambda __x, __y, __z, filled, **kwargs: (
                (__x,__y,__z), filled, kwargs)
        else:
            parser = lambda filled, **kwargs: (
                None, filled, kwargs)
        parser.__name__ = str('voxels')   # to avoid unicode_literals
        xyz, filled, kwargs = parser(*args, **kwargs)

Would that be acceptable?

@anntzer
Copy link
Contributor

anntzer commented Jul 18, 2017

Unfortunately Signature doesn't implement PEP457 yet, so that doesn't help too much. Also, there's no inspect.Signature.from_string method, so we'd need to do something like sig = inspect.signature(lambda x, y, z, *, kwonly: pass). Of course, we can't backport that, because it uses py3 syntax. So now we need to write our own function get_sig("x, y, z, *, kwonly") to implement the py3 grammar!

Actually it nearly does, it's just tucked in a private function (used to parse Argument Clinic signature strings):

In [21]: inspect._signature_fromstr(inspect.Signature, None, "(x, y, z, /, filled=None, **kwargs)", False)
Out[21]: <Signature (x, y, z, /, filled=None, **kwargs)>

(if we don't want to rely on a private function we can just copy-paste the implementation).

Square brackets (optional positional-only arguments) are not implemented (I guess because it's not really possible to represent them as Signature objects anyways), but a possibility would be to provide a list of signatures which would be tried one after the other until one matches.

@eric-wieser
Copy link
Contributor Author

eric-wieser commented Jul 18, 2017

Oops, you're right. I did know about that, but I assumed it worked by evaling a function with that signature, which in hindsight wouldn't work.

a list of signatures which would be tried one after the other until one matches.

I think I'd be in favor of using Signature here, but I don't think backporting that should hold up this PR. There is a backport in the funcsigs package (https://github.com/aliles/funcsigs), but it looks unmaintained.

@anntzer
Copy link
Contributor

anntzer commented Jul 18, 2017

That's definitely not a blocker. (I agree it should be a separate PR.)

@eric-wieser
Copy link
Contributor Author

I've updated the implementation to use a form similar to the above comment. It fails, but only because of docs.scipy.org being inaccessible.

@anntzer
Copy link
Contributor

anntzer commented Jul 19, 2017

Actually I think I would still favor something like

parsers = []
@parsers.append
def voxels(x, y, z, filled, **kwargs): return ...
@parsers.append
def voxels(filled, **kwargs): return ...
for parser in parsers:
    try: args, kwargs = parser(*args, **kwargs)
    except TypeError: pass
    else: break
else: raise TypeError("no signatures match") # some appropriate error message.

(but not a big deal either)

@eric-wieser
Copy link
Contributor Author

eric-wieser commented Jul 19, 2017

@anntzer: That approach makes it difficult to produce useful error messages.

@anntzer
Copy link
Contributor

anntzer commented Jul 19, 2017

Actually it's the opposite: Python is not able to autogenerate error messages for such signature-overloaded functions, so it should be better to just provide the error message ourselves.

@QuLogic
Copy link
Member

QuLogic commented Jul 20, 2017

Sorry, I did not intend there to be such a long detour on what was ultimately a misunderstanding on my part wrt whether filled is needed.

On to something I did understand though, PEP457 may be a standard, but I'm not so sure we can rely on users having read it (I certainly haven't until now.) So I think you will have to list out all the individual ways of calling the function (as we do in, e.g., pcolor), regardless of what the implementation ends up being.

@eric-wieser
Copy link
Contributor Author

So I think you will have to list out all the individual ways of calling the function

Done, along with some other fixes to the parameter documentation

@tacaswell
Copy link
Member

Thanks @eric-wieser !

This is a pretty cool feature to get in 😄

@eric-wieser
Copy link
Contributor Author

Pleased to see it finally made it! Thanks everyone for the feedback :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants