The sklearn.set_config(display='diagram') feature highlighted here needs more prominence. At least, it should be easy to find it when googling scikit-learn diagram display. It isn't. scikit-learn display=diagram is much the same, and scikit-learn "display=diagram" is on the mark but the results are not easy to parse as being relevant.

We probably need an example, or a user guide page, showing how different estimators are diagrammed, and perhaps also instructing users to use IPython.display's display function when needed.

So you need examples along with that guide page so that it shows more prominence such as diagrams and all?

Hey @jnothman can you please help me with open source contributions to projects pertaining to machine learning. I have worked on research projects though I have never contributed here. Also all the issues seem so overwhelming I can't decide where to begin with. It would be wonderful if you could guide me through. Thanks 😁✨

@thomasjpfan What is the best place to put an example of the pipeline diagram call?
cc: @Mariam-ke

a) Here?
https://github.com/scikit-learn/scikit-learn/blob/647fcb1ac13abd8c2eb2554d526f4ad41fee6778/doc/modules/compose.rst

b) Release highlights
https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_0_23_0.html#sphx-glr-download-auto-examples-release-highlights-plot-release-highlights-0-23-0-py

c) some other file?

I'm not sure the right way to go is to introduce diagram rendering in the UG at random places. In most cases this will distract from the original purpose of the code snippets.

I would prefer to have a dedicated example for illustrating the diagrams of different estimator, as Joel proposed above. Note that we already have https://scikit-learn.org/dev/auto_examples/compose/plot_column_transformer_mixed_types.html#html-representation-of-pipeline . Maybe we can add the keyword "Diagram" to this subsection for SEO.

We can also add a small and short note at the end of this section of the getting started guide: https://scikit-learn.org/dev/getting_started.html

Or a section (in Getting Started or otherwise) on how to configure Scikit-learn for a Jupyter Notebook.

How about,

1. in this file: HTML representation of Pipeline,
   we change the title from "HTML representation of Pipeline" to "HTML Representation of Pipeline (Display Diagram)"

2. in this file: Getting Started, can add:

Pipelines: displaying diagrams in Jupyter notebook
---------------------------------------------------

The default configuration for displaying a pipeline is 'text':  `set_config(display='text')`.  To visualize the diagram in Jupyter Notebook, use `set_config(display='diagram')` and then call the pipeline object.

  >>>from sklearn.pipeline import Pipeline
  >>>from sklearn.svm import SVC
  >>>from sklearn.decomposition import PCA
  >>>estimators = [('reduce_dim', PCA()), ('clf', SVC())]
  >>>pipe = Pipeline(estimators)
  >>>pipe
  Pipeline(steps=[('reduce_dim', PCA()), ('clf', SVC())])

  >>>from sklearn import set_config
  >>>set_config(display='diagram')
  >>>pipe

OK for 1

For 2: I would rather not make a new section in the getting started guide. I think a note would be enough and to keep it short, we can just re-use the already defined pipeline in the corresponding section, and use the config_context manager (We can also mention set_config)

In getting_started.rst, can make the following edit, at the end of the section "Pipelines: chaining pre-processors and estimators", can add:

The default configuration for displaying a pipeline is 'text' where `set_config(display='text')`.  To visualize the diagram in Jupyter Notebook, use `set_config(display='diagram')` and then call the pipeline object.

I think users would benefit from seeing the diagram in action:

To render estimators as diagrams in notebooks, use the `display='diagram'` option:

>>> with config_context(display='diagram'):
>>>    pipe
<diagram renders here>

You may also call `set_config` (link https://scikit-learn.org/stable/modules/generated/sklearn.set_config.html) at the top of the notebook.

This would ideally be withing a .. note:: Diagram rendering of estimators

I didn't realize that the code in the UG is not executed by sphinx, and thus we can't see diagrams in the UG. It seems that we should create a dedicated example then, with various pipelines / estimators of different complexity. Would you be interested in doing that @reshamas ? @hongshaoyang also showed interest

Some examples of estimators worth rendering (feel free to deviate):

• single estimator like SVC
• basic pipeline
• complex pipeline with ColumnTransformer (as in the other example linked above)
• GridSearch within a pipeline, or GridSerach over a pipeline

@NicolasHug
Would the examples:

1. be created here: https://github.com/scikit-learn/scikit-learn/tree/master/examples
2. Would we add a new sub-folder for "examples/pipelines" OR

• put SVC example under "/svm"
• put ColumnTransformer under "/preprocessing"
• put basic pipeline --> ?
• put GridSearch --> ?

I think we can just put it in examples/miscellaneous (we only need 1 example)

You mean we need a few examples, but in one file called /examples/miscellaneous/pipeline.py?

yes, though the file name needs to be named plot_xyz.py so I'd suggest plot_display.py

By example I mean one entry in https://scikit-learn.org/stable/auto_examples/index.html. Each of these are 1 file. In the example, we can illustrate more than one estimator (all of them, actually)

where? should this just be the example? I'm not sure we have enough material about scikit-learn in jupyter to have a whole new UG chapter

I wouldn't recommend a tutorial because our tutorials page needs some work: #18257. We use the examples nowadays for tutorial-like entries, e.g. this or this

I'm also curious about what else our recommendations are regarding scikit-learn + jupyter?

@NicolasHug @jnothman
Could we have a folder with example notebooks, that people could easily download and use?

@NicolasHug @cmarmo @jnothman
The Modin project has a "jupyter" folder in their /examples/. Here it is:
https://github.com/modin-project/modin/tree/master/examples/jupyter

Is this something we could consider doing for scikit-learn? It seems a bit more flexible.

@reshamas why not create a regular example as suggested in #18305 (comment)? The examples from the gallery can be downloaded as .ipynb files if users want to use them as notebooks.

@NicolasHug I recall the problem with creating a regular example was that sphinx did not produce the diagram in the documentation. But, in a Jupyter notebook, the diagram is produced. That way, users can see the code with the diagram.

The examples are able to generate the html disaply properly, if that's what you mean: https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html#sphx-glr-auto-examples-compose-plot-column-transformer-mixed-types-py

The issue with the html renderring and sphinx, IIRC, was only in the UG because the code isn't executed there

@cmarmo
I haven't had time to work on this, so if someone wants to take it over, it's available.