Skip to content

adding a notebook to python converter #423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

choldgraf
Copy link
Contributor

@choldgraf choldgraf commented Oct 14, 2018

Over the months we've gotten several requests to support people storing examples as Jupyter Notebooks. Recently I also found there's even a fork of sphinx-gallery whose primary feature is just to support docs w/ notebooks (https://github.com/Chilipp/sphinx-nbexamples).

In #245 it was mentioned that a relatively easy solution would be to support a notebook -> .py converter. @chsasank helpfully provided a script that they use to convert notebooks (using the json module so it's relatively lightweight). I've modified/updated that script a bit, and included in this PR as a helper function, along with tests and an example that shows the .ipynb -> .py -> .ipynb roundtrip.

closes #245

note that this was attempted once last year by @chsasank (#220) and it ultimately didn't make it through. My hope is that making this a "helper function" instead of part of the core build process, we have a bit more wiggle-room for inconsistencies.

Here's a link to the built page from a notebook: https://613-25860190-gh.circle-artifacts.com/0/alabaster_html/auto_examples/sin_func/plot_notebook.html#sphx-glr-auto-examples-sin-func-plot-notebook-py

@@ -24,7 +24,7 @@ jobs:
- restore_cache:
keys:
- cache-pip
- run: pip install --user numpy scipy matplotlib seaborn sphinx_rtd_theme pillow sphinx pytest vtk traits traitsui pyface pyqt5 mayavi memory_profiler
- run: pip install --user -r doc/doc-requirements.txt
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note - this PR also splits out the docs requirements into their own file to make things clearer for would-be developers or users.

@choldgraf choldgraf force-pushed the ipynb_convert branch 7 times, most recently from b7b8ca1 to 9d2ed84 Compare October 14, 2018 18:44
@choldgraf choldgraf added this to the v0.3.0 milestone Oct 14, 2018
@choldgraf choldgraf force-pushed the ipynb_convert branch 2 times, most recently from 7d4c301 to e6df2b0 Compare October 14, 2018 20:47
@codecov-io
Copy link

codecov-io commented Oct 14, 2018

Codecov Report

Merging #423 into master will decrease coverage by 0.13%.
The diff coverage is 90.32%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #423      +/-   ##
==========================================
- Coverage   95.48%   95.34%   -0.14%     
==========================================
  Files          29       29              
  Lines        2235     2297      +62     
==========================================
+ Hits         2134     2190      +56     
- Misses        101      107       +6
Impacted Files Coverage Δ
sphinx_gallery/tests/test_notebook.py 100% <100%> (ø) ⬆️
sphinx_gallery/notebook.py 94.95% <88.23%> (-5.05%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d0a0a6...6a21bde. Read the comment docs.

@choldgraf
Copy link
Contributor Author

The one travis error is a dependency clash with Python 3.4

UnsatisfiableError: The following specifications were found to be in conflict:
  - pypandoc -> python[version='>=2.7,<2.8.0a0'] -> readline=7
  - pypandoc -> python[version='>=2.7,<2.8.0a0'] -> tk=8.6
  - python=3.4

I can try fixing that (and adding the dependency to the appveyor build) if folks are generally +1 on this approach, but I don't wanna spend a bunch of time getting pandoc/pypandoc working if it's gonna be rejected :-) our only other option would be to write a ton of custom md->rst code, similar to https://github.com/sphinx-gallery/sphinx-gallery/blob/master/sphinx_gallery/notebook.py#L62

@Titan-C
Copy link
Member

Titan-C commented Oct 26, 2018

Thank you for taking time to do this @choldgraf
As I also said in #245 (comment)
can we make this converter part of a separate repository please, one for more open contributions we don't guaranty support. I know it is a lot of extra work, and I haven't had time for this, but I don't want to mix.
Unfortunately for now Sphinx only supports rst, and Jupyter notebooks are amazing but they work on markdown. I don't want to throw us into problems we barely have time to support. We'll have to deal with a lot of conversions, which pandoc does not even support. Reason why #220 was turned down in the end.

@larsoner
Copy link
Contributor

@choldgraf what should we do with this? Move to SG-extra?

I have a feeling it will need some reworking to fit with the new scraping changes.

Does this supersede #422 and #420?

@choldgraf
Copy link
Contributor Author

yeah probably, and anyway at this point I'd probably recommend that people use Jupytext anyway...if we want to support jupyter notebooks more effectively I'd rather it happen by improving the story of two-way sync between ipynb and .py in that package. I'll close this one

@choldgraf choldgraf closed this Jun 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support jupyter notebooks
4 participants