Skip to content

Ipython/Webagg integration #2054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 28, 2013
Merged

Conversation

mdboom
Copy link
Member

@mdboom mdboom commented May 23, 2013

This integrates the WebAgg backend with the IPython notebook.

It works by running its own Tornado server in its own thread in the kernel. (There doesn't seem to be any logical way to integrate mpl's tornado server with IPython's, since IPython's runs in the frontend, not the kernel). This actually seems to work quite well, despite all of the gotchas with Python threading -- remember, this is how all of matplotlib's GUI backends have worked since time immemorial ;)

Since IPython doesn't have any native understanding of WebAgg, the backend must be set in your matplotlibrc file, and ipython started without the --pylab flag. Anything else will likely give you the wrong backend. It should be straightforward for IPython to add webagg as an option for pylab (or use it instead of inline) to avoid needing to be so careful in the future.

To display figures inline, either call plt.show() as the last line of an input block to display its output (this will show all figures), or simply display a figure object.

As an aside change, this removes the need for putting special element ids on things so they can be found later -- this now just builds all of the DOM elements by hand and assigns them as it goes to the figure object. Much cleaner, and less risk of element id namespace clashing.

This may be too big a chunk of work to properly vet before 1.3.0, but it should be sufficient to demo and discuss with IPython folks at Scipy.

@ellisonbg
Copy link

I haven't looked in detail about this yet, but there is a critical problem: while it is sometimes true, you can't make the assumption that the kernel is running on the same host as the main IPython notebook server. In the bigger picture, kernels will likely be running on other hosts that are not exposed to the public internet (security risk). What this means is that the only officially supported way of talking between browser and kernel is the established WebSocket/ZMQ channels that IPython has. I know this probably seems extremely limited, but we have thought about this extensively and see no way of officially supporting these "3rd party channels" even though a number of people are doing this.

But, we realize that right now, our own IPython channels are probably not fully sufficient for doing what you want. We plan on addressing that starting this summer and plan on having a working solution by the end of the year. It would be great to interact with you and other mpl devs at SciPy and our upcoming dev meeting (details to follow) to figure out exactly what we need to do to make this possible.

In the meantime there is no harm in your current design moving forward - I just want you to be aware of its limitations and the upcoming changes on the IPython side of things.

@mdboom
Copy link
Member Author

mdboom commented May 23, 2013

@ellisonbg: That certainly makes sense. This PR can probably serve as a roadmap for the kinds of things we would need to pipe through IPython channels.

As for whether this gets merged as-is or not I think remains an open question. I know personally a number of people who are chomping at the bit for something that works now and don't care that it's restricted to having the kernel/frontend on the same trusted machine etc. But I agree this is probably not the long-term solution we'd like it to be.

@epifanio
Copy link

some fee note after a brief testing :
seems that after a first plot in a notebook cell, in order to add a new plot in an other cell I need to close the previous plot.
this means we can't have more than one plot active at time, if i don't close the plot, each show() will appended the plot to the previos one.

thanks for the great work!
looking fwd to follow its developments and continue testing.

@andreabedini
Copy link

I implemented the same thing in ipython/ipython#4077

@mdboom
Copy link
Member Author

mdboom commented Aug 26, 2013

Sorry this has languished so long. I think this is ready for a final review and merge.

In discussions with the IPython folks at their dev meeting, it was determined that there's not much harm in committing this approach now, even if IPython provides a better way forward in the future.

As for my own needs at my employer, we'd like to see this merged not for the IPython integration but because it makes embedding a plot in any web app much easier. If there's opposition to the IPython stuff going in now, I can back that out. Next steps for that also include generalizing the Tornado callbacks so the content can be retrieved from another server other than Tornado -- but I think that's best left as a follow-on to this PR.

@ellisonbg
Copy link

I am fine with the the IPython stuff going into mpl now.

On Mon, Aug 26, 2013 at 1:19 PM, Michael Droettboom <
notifications@github.com> wrote:

Sorry this has languished so long. I think this is ready for a final
review and merge.

In discussions with the IPython folks at their dev meeting, it was
determined that there's not much harm in committing this approach now, even
if IPython provides a better way forward in the future.

As for my own needs at my employer, we'd like to see this merged not for
the IPython integration but because it makes embedding a plot in _any_web app much easier. If there's opposition to the IPython stuff going in
now, I can back that out. Next steps for that also include generalizing the
Tornado callbacks so the content can be retrieved from another server other
than Tornado -- but I think that's best left as a follow-on to this PR.


Reply to this email directly or view it on GitHubhttps://github.com//pull/2054#issuecomment-23291455
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@andreabedini
Copy link

I have been thinking on how to decouple this from ipython specific parts, so we can offer a sort of widget, easy to incorporate into any webapp. I reckon the only part the specific webapp has to implement would be the message exchange, that is taking care of passing messages between mpl and its html widget. This could be simply realised by having the figure manager take a socket-like object that it will use to communicate with the widget as it does now. Applications using this interface could use for this purpose a Tornado WebSocket or any other message exchange system (IPython could use its own).

my 2c, happy to help out if needed.

@mdboom
Copy link
Member Author

mdboom commented Aug 26, 2013

@andreabedini : I'd be glad to have your help on that, and is definitely what I had in mind. My employer has a direct need for that, and it would also help us be more future proof to any changes in IPython. Do you want to take the first crack at it (based on this PR, and not master, in all likelihood).

@andreabedini
Copy link

Sure, but be aware I have a day job :) I code mostly on nights&weekends. Also @mdboom I'd like to ask: have you experimented with painting on a html canvas? I did some tests and it works well too. Perhaps it would be better use of bandwidth?

@mdboom
Copy link
Member Author

mdboom commented Aug 26, 2013

See https://code.google.com/p/mplh5canvas/

It's a good approach for a lot of things, but it's much harder to support the same kind of interactivity as the webagg approach. I think they are both useful for different use cases.

mdboom added a commit that referenced this pull request Aug 28, 2013
@mdboom mdboom merged commit bdfdbdf into matplotlib:master Aug 28, 2013
@mdboom mdboom mentioned this pull request Sep 3, 2013
@beojan
Copy link

beojan commented Oct 2, 2013

I realize this has already been merged, but using webagg causes the IPython kernel to die with IPython 2.0 (from git)

@mdboom
Copy link
Member Author

mdboom commented Oct 2, 2013

@beojan: I can't reproduce with current masters of matplotlib and ipython (926bddc and 597ff8c62fee respectively). I did notice that the new IPython forcibly resets the matplotlib backend, and I have to explicitly use:

import matplotlib
matplotlib.use("WebAgg")

but I couldn't get it to crash.

Can you open a new issue with more detail? What script are you running exactly that causes it to crash and which version of matplotlib you are using and on which platform?

@beojan
Copy link

beojan commented Oct 2, 2013

I am on Debian.
The issue is here: #2487

@mdboom mdboom deleted the ipython-webagg-integration branch August 7, 2014 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants