ENH, DOC: Add support for interactive examples for NumPy with jupyterlite-sphinx
#26745
Conversation
Co-Authored-By: Albert Steppi <1953382+steppi@users.noreply.github.com>
[skip cirrus] [skip azp]
|
This could use a release note. |
| .. try_examples:: | ||
|
|
||
| >>> import numpy as np | ||
|
|
||
| >>> a = np.memmap('newfile.dat', dtype=float, mode='w+', shape=1000) | ||
| >>> a[10] = 10.0 | ||
| >>> a[30] = 30.0 | ||
| >>> del a | ||
|
|
||
| >>> b = np.fromfile('newfile.dat', dtype=float) | ||
| >>> print(b[10], b[30]) | ||
| 10.0 30.0 | ||
|
|
||
| >>> a = np.memmap('newfile.dat', dtype=float) | ||
| >>> print(a[10], a[30]) | ||
| 10.0 30.0 |
There was a problem hiding this comment.
I haven't tested whether every example works because there are way too many, but I did test this and this is one of the examples that doesn't, and fails with the error
ValueError: mmap length is greater than file size
and reducing the shape from 1000 to 100 doesn't help: OSError: [Errno 43] No such device (edit: this error occurs on subsequent runs on the same cell because newfile.dat cannot be loaded)
There was a problem hiding this comment.
I'm not sure if there is a way to enable the Pyodide file system or interact with Pyodide directly through a Pythonic interface like this.
[skip cirrus] [skip azp]
[skip cirrus] [skip azp]
08a1ecf to
6b416ae
Compare
|
I forgot to skip Azure Pipelines and Cirrus in my previous commit – please don't mind the force-push, my apologies |
|
I notice that for some reason the CircleCI build isn't able to load the examples, most likely because Edit: I think I figured it out – CircleCI is storing the notebooks in a different path, but GitHub Pages does not do so. Please use my GitHub Pages site to test things out: https://agriyakhetarpal.github.io/numpy/reference/maskedarray.generic.html, I have another branch at agriyakhetarpal#2 and I shall keep both of them in sync based on further commits that I push here. |
@melissawm, I'm not sure – I think you'll need to refresh your |
|
Nevermind - I do see it now! |
[skip actions] [skip azp] [skip cirrus]
|
d818f8c disables source maps for JupyterLite, based on scikit-learn/scikit-learn#26246. This brings the size of the JupyterLite build in the |
This reverts commit ef3f14b. [skip actions] [skip azp] [skip cirrus]
[skip actions] [skip azp] [skip cirrus]
[skip actions] [skip azp] [skip cirrus]
|
Hi @rgommers, we published https://github.com/jupyterlite/jupyterlite-sphinx/releases/tag/v0.18.0 some moments ago (thanks @steppi!), and I've updated the versions of |
[skip actions] [skip azp] [skip cirrus]
[skip actions] [skip azp] [skip cirrus]
100b557 to
e7972a5
Compare
|
Commit e7972a5 bumps to https://github.com/jupyterlite/pyodide-kernel/releases/tag/v0.5.1 that was released today, which brings the recent https://github.com/pyodide/pyodide/releases/tag/0.27.1. (Apologies for the force push!) |
There was a problem hiding this comment.
Hi @agriyakhetarpal ! The changes look good to me, I've successfully built this PR locally and it seems to work (although the jupyterlite blocks don't work locally).
The CircleCI version of the "Try it" works fine, but when I click "Open in new tab" I see this:
Is this expected? (Maybe it's because it's in CI)
|
Hi @melissawm, thanks for the review! I don't think I can reproduce it with CircleCI. Could you please retry with incognito mode, or with a different internet connection or mobile data if you can? This is what I get: 
For reference, I tried this link: https://output.circle-artifacts.com/output/job/4b11fff4-e43b-4010-af4e-de2e84deed66/artifacts/0/doc/build/html/lite/notebooks/index.html?path=88efbc39_8e7b_4e21_86c0_519bb66f4f6a.ipynb Based on your screenshot, I guess you're using Firefox. I tried with it as well, both with incognito mode and without, and the JupyterLite UI loads perfectly.
They should work locally as well. Could you please share any errors or output that you might be seeing? |
[skip actions] [skip azp] [skip cirrus]
|
I think they had a partial outage eariler today so that may have been just bad luck. I'll wait for the check to finish here and test again. |
There was a problem hiding this comment.
This looks pretty clean, nice work @agriyakhetarpal! Time to give this a go. Thanks to @melissawm and everyone else who contributed as well.
@agriyakhetarpal could you update us on the next steps here? I think it includes upgrading to NumPy 2.2 in Pyodide and enabling use of nightlies, and probably also enabling functionality on the User Guide? Anything else?
Thanks, @rgommers! The following steps here are indeed those, and also, we need to add some docs to resolve gh-27310, similar to @melissawm's SciPy PR (scipy/scipy#22385, which I'll be reviewing and we can adopt similar messaging for NumPy when done).
|
|
Thanks Agriya. Docs would be a nice next step here indeed, leveraging the work Melissa did for SciPy (the setups and considerations are almost identical). It'd also be fine with me to have a shorter explanation and link to the SciPy docs on the topic for more details.
This would be good to prioritize. It seems like there was a way forward (or two possibly) there, by reverting making symbols private for example. Re User Guide: agreed with waiting for better tracebacks and doing 1-2 pages first. |
Description
This PR adds support for WASM-powered interactive examples with JupyterLite and
jupyterlite-sphinxin the NumPy documentation.A similar enhancement was made for the documentation for SciPy (scipy/scipy#20019) and that for PyWavelets (PyWavelets/pywt#728).
How to test this
Tip
I have a GitHub Pages website deployed at https://agriyakhetarpal.github.io/numpy/ through which these changes can be experimented with. The CircleCI deployment does not render the examples properly, however, a fix is available as reported in #26745 (comment) and will be fixed upstream in jupyterlite/jupyterlite-sphinx#182.
Update, 24/07/2024: this fix is now available in
jupyterlite-sphinxversion 0.16.2, and the CircleCI deployment should render the examples properly as well.Key changes (so far)
jupyterlite-sphinxindoc/source/conf.pyanddoc/Makefile, coupled with changes to styling indoc/source/_static/numpy.cssnumpy/under the "Examples" section in the docstrings for various modulesNote
I have currently left out the following:
_datasource.py,_utils_impl.py,_version.py,_array_api_info.py,ctypeslib.py, files undernumpy/_typing/, andnumpy/_core/numerictypes.py.Tip
The "NumPy user guide" section has not been modified at this time due to the size of the PR (in terms of the number of files modified). However, it can be modified or adapted to a Jupytext notebook similar to scipy/scipy#20303 to make it work well in an interactive set-up.
Important
The NumPy documentation will have to be rebuilt and reuploaded in https://github.com/numpy/doc/ for these changes to show up in the official documentation.
What issue does this PR address?
Closes #25969
Additional context
try_examples.jsonconfiguration file. A custom Pyodide distribution or a custom wheel of NumPy is not provided with this PR, and a solution is being explored with nightly WASM wheels for NumPy (established in CI, BLD: Push NumPy's Emscripten/Pyodide wheels nightly to Anaconda.org PyPI index #26134) via Web proxy to work around the lack of CORS headers for nightly wheels uploaded to the Anaconda.org index pyodide/pyodide#4898.