DOC Merge plot_svm_margin.py and plot_separating_hyperplane.py into plot_svm_hyperplane_margin.py #31045

SwathiR1999 · 2025-03-21T09:31:14Z

Reference Issues/PRs

References #30621

What does this implement/fix? Explain your changes.

Consolidated two SVM example scripts (plot_svm_margin.py and plot_separating_hyperplane.py) into a single file plot_svm_hyperplane_margin.py.
Updated explanation for clarity.

Any other comments?

This PR addresses the feedback from #30975, which was closed.

@StefanieSenger Looking forward to your feedback. Thanks!

…ngle example

github-actions · 2025-03-21T09:32:28Z

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 8612d2d. Link to the linter CI: here}

StefanieSenger

Hi @SwathiR1999,

thank you for your patience and for your efforts to merge the examples. It looks very good and I like the design of the plots. The plots all have heading and legend and are easy on the eye.

I think we could add some information and I have left you some suggestions on that, too.

When I build the docs locally, I cannot click to open it and also there is some text overflow in the background. I believe this is because of a rending issue in the heading, and I have added a suggestions for a fix.

What is lacking is to remove the old examples and to add redirections from both old links to the new example link in the doc/conf.py file. You can have a look at #30028 to see how to add the redirections.

You need to delete all links to the old examples as well (and replace it with the new one if suitable). There are some, probably too many, links to plot_separating_hyperplane_unbalanced currently.

StefanieSenger · 2025-04-28T12:13:23Z

examples/svm/plot_svm_hyperplane_margin.py

+- **Small C (e.g., 0.05)**:
+    - Allows some misclassifications, resulting in a wider margin.
+- **Moderate C (e.g., 1)**:
+    - Balances classification accuracy and margin width.
+- **Large C (e.g., 1000)**:
+    - Prioritizes classifying all points correctly, leading to a narrower margin.


I think while reworking the example, we could add some more information.

Do you think you could add some suggestion of when to use a small, a moderate or a high C value in a practical use case?

We could also talk about overfitting and underfitting in this context.

Also, we could talk about miss-classification / error for different margins.

I think this could be added after the code for the plot using the # % syntax to make this example more notebook-like. (Also have a look at #30028 for a good example on the notebook style.)

StefanieSenger · 2025-04-28T12:27:51Z

examples/svm/plot_svm_hyperplane_margin.py

+"""
+=========================================================================
+SVM: Effect of Regularization (C) on Maximum Margin Separating Hyperplane
+========================================================================


Suggested change

========================================================================

=========================================================================

The lacking "=" leads to the title not being rendered correctly. Adding "=" should fix it.

I also hope that this fixes the problem with the example not being clickable from the main examples page and the text overflow there (see other comment).

StefanieSenger · 2025-04-28T12:36:56Z

examples/svm/plot_svm_hyperplane_margin.py

+    clf.fit(X, y)
+
+    plt.subplot(1, 3, i)
+    plt.scatter(X[:, 0], X[:, 1], c=y, s=30, cmap=plt.cm.Paired, edgecolors="k")


Please add axis labels. Something simple like "Feature 1" and "Feature 2" would be enough.

This is resolved. (Just the "resolved" button doesn't show here.)

StefanieSenger · 2025-04-28T12:39:08Z

examples/svm/plot_svm_hyperplane_margin.py

+        s=100,
+        linewidth=1.5,
+        facecolors="none",
+        edgecolors="r",


What do you think about having a different color for miss-classified samples?

This is resolved. (Just the "resolved" button doesn't show here.)

…nto doc_addlink

…earn into doc_addlink

SwathiR1999 · 2025-05-03T08:20:56Z

Hi @StefanieSenger , thanks for the feedback!

There are some, probably too many, links to plot_separating_hyperplane_unbalanced currently.

Just to clarify - should I also apply similar changes (merge/update/remove references) to plot_separating_hyperplane_unbalanced.py, or is the current focus only on replacing plot_svm_margin.py and plot_separating_hyperplane.py with the new plot_svm_hyperplane_margin.py example?

StefanieSenger · 2025-05-03T09:44:30Z

Just to clarify - should I also apply similar changes (merge/update/remove references) to plot_separating_hyperplane_unbalanced.py, or is the current focus only on replacing plot_svm_margin.py and plot_separating_hyperplane.py with the new plot_svm_hyperplane_margin.py example?

Oh, my bad. I didn't see there were plot_separating_hyperplane_unbalanced.py AND plot_separating_hyperplane.py.
No please keep the current focus and don't let you be confused by me. :)

…nto doc_addlink

- Modified \`doc/conf.py\` and \`doc/modules/svm.rst\` to update documentation and add example reference.\n- Removed outdated example: \`examples/svm/plot_svm_margin.py\`.

SwathiR1999 · 2025-05-07T06:53:52Z

Hi @StefanieSenger,

I’ve implemented your suggestions. I wanted to check with you on one small point:

While plot_svm_margin.py is fully covered by the merged version and can be removed, I feel that plot_separating_hyperplane.py still serves a valuable purpose. It provides a clean and minimal visual introduction to the concept of an SVM decision boundary and support vectors, which may be helpful for first-time readers.

It is currently referenced early in the SVM documentation, which aligns well with its simplicity. The new merged example, on the other hand, focuses more on the effect of C, making it a good fit for a later section - ideally where C is discussed in more detail.

Would it be alright to:

Retain plot_separating_hyperplane.py as an introductory visual example?
If so, would you suggest improving clarity by adding axis labels (like 'Feature 1', 'Feature 2') and annotations such as 'Support Vectors'?
Add a cross-reference to the new example (plot_svm_hyperplane_margin.py) later in the documentation, once the role of C has been introduced?

Looking forward to your thoughts!

StefanieSenger

Hi @SwathiR1999,

thank you for your further work. I went through the example and left the next round of comments. (I might have more afterwards. 😬)

In general, please feel very free to add any information you think would help people who start learning on SVCs and regularization to understand. We're very happy to add new passages if they are helpful and fitting the context of the example.

On your question:

Retain plot_separating_hyperplane.py as an introductory visual example?

I can definitely see what you mean and sort of agree. But on the other hand, we have too many examples to take care of and plot_separating_hyperplane.py is pretty similar to the re-newed example you are working on. This new example would (with your help) ideally grow into a demonstration that also helps beginners.

Maybe @adrinjalali has a different opinion on it?

StefanieSenger · 2025-05-28T14:26:03Z

doc/modules/svm.rst

@@ -404,13 +404,14 @@ Tips on Practical Use

 * **Setting C**: ``C`` is ``1`` by default and it's a reasonable default
  choice.  If you have a lot of noisy observations you should decrease it:
-  decreasing C corresponds to more regularization.
+  decreasing C corresponds to more regularization (see example below).


Suggested change

decreasing C corresponds to more regularization (see example below).

decreasing C corresponds to more regularization.

I think it's better to leave this unchanged. We add links in suitable spots, but these quasi-links just occupy the reader's attention without adding real value.

StefanieSenger · 2025-05-28T14:26:25Z

doc/modules/svm.rst


  :class:`LinearSVC` and :class:`LinearSVR` are less sensitive to ``C`` when
  it becomes large, and prediction results stop improving after a certain
  threshold. Meanwhile, larger ``C`` values will take more time to train,
  sometimes up to 10 times longer, as shown in [#3]_.

+


Suggested change

Let's also leave this unchanged.

StefanieSenger · 2025-05-28T14:27:22Z

doc/modules/svm.rst

+.. rubric:: Examples
+
+* :ref:`sphx_glr_auto_examples_svm_plot_svm_hyperplane_margin.py`


It would be great to add the reference to the example in the text instead, including a description of what the example is about.

StefanieSenger · 2025-05-28T14:28:27Z

doc/modules/svm.rst

@@ -632,7 +636,11 @@ indicates a perfect prediction. But problems are usually not always perfectly
 separable with a hyperplane, so we allow some samples to be at a distance :math:`\zeta_i` from
 their correct margin boundary. The penalty term `C` controls the strength of
 this penalty, and as a result, acts as an inverse regularization parameter


Suggested change

this penalty, and as a result, acts as an inverse regularization parameter

this penalty, and as a result, acts as an inverse regularization parameter:

StefanieSenger · 2025-05-28T14:29:08Z

doc/modules/svm.rst

@@ -632,7 +636,11 @@ indicates a perfect prediction. But problems are usually not always perfectly
 separable with a hyperplane, so we allow some samples to be at a distance :math:`\zeta_i` from
 their correct margin boundary. The penalty term `C` controls the strength of
 this penalty, and as a result, acts as an inverse regularization parameter
-(see note below).
+(see the figure below). Also please refer to the note below.


Suggested change

(see the figure below). Also please refer to the note below.

I would suggest to simply finish the last sentence with a colon and then show the plot directly.

StefanieSenger · 2025-05-28T14:40:41Z

examples/svm/plot_svm_hyperplane_margin.py

+
+# %% [markdown]
+# - **Small `C` (e.g., 0.01, 0.05)**:
+#   - Use when:


The Use when are not rendered correctly.

StefanieSenger · 2025-05-28T14:42:23Z

examples/svm/plot_svm_hyperplane_margin.py

+plt.show()
+
+# %% [markdown]
+# - **Small `C` (e.g., 0.01, 0.05)**:


Would be nice to have the representation of C (like in: "Small `C`") the same as above, either with or without the backticks everywhere.

StefanieSenger · 2025-05-28T14:52:07Z

examples/svm/plot_svm_hyperplane_margin.py

+This script demonstrates the concept of maximum margin separating hyperplane
+in a two-class separable dataset using a Support Vector Machine (SVM)
+with a linear kernel and how different values of `C` influence margin width.


I would suggest to add a short note on what a margin is to make this more beginner-friendly.

StefanieSenger · 2025-05-28T14:53:04Z

examples/svm/plot_svm_hyperplane_margin.py

+# Visualize
+plt.figure(figsize=(12, 4))
+for i, C_val in enumerate(C_values, 1):
+    clf = svm.SVC(kernel="linear", C=C_val)


I think we should add a random state here, so this example looks the same with every run.

StefanieSenger · 2025-05-28T14:55:54Z

examples/svm/plot_svm_hyperplane_margin.py

+        X[misclassified, 0],
+        X[misclassified, 1],
+        facecolors="none",
+        edgecolors="k",


Using the same color (edgecolors="k") for correctly classified and for misclassified points, doesn't make them stand out as expected. Could you fix this please?

SwathiR1999 and others added 2 commits March 21, 2025 14:37

Merged plot_svm_margin.py and plot_separating_hyperplane.py into a si…

30bf31a

…ngle example

Merge branch 'scikit-learn:main' into doc_addlink

b5018ef

SwathiR1999 and others added 2 commits March 21, 2025 15:24

Formatted code using Black and fixed linting issues with Ruff

4211cca

Merge branch 'main' into doc_addlink

cbcffba

SwathiR1999 changed the title ~~Doc Merge plot_svm_margin.py and plot_separating_hyperplane.py into plot_svm_hyperplane_margin.py~~ DOC Merge plot_svm_margin.py and plot_separating_hyperplane.py into plot_svm_hyperplane_margin.py Mar 21, 2025

github-actions bot added the Documentation label Mar 21, 2025

StefanieSenger self-requested a review April 28, 2025 11:26

StefanieSenger reviewed Apr 28, 2025

View reviewed changes

StefanieSenger mentioned this pull request Apr 28, 2025

Add links to examples from the docstrings and user guide #30621

Open

SwathiR1999 added 11 commits May 3, 2025 07:59

Merge branch 'main' of https://github.com/scikit-learn/scikit-learn i…

3116f1f

…nto doc_addlink

Merge branch 'doc_addlink' of https://github.com/SwathiR1999/scikit-l…

8c1fa62

…earn into doc_addlink

updated plot_svm_hyperplane_margin.py

4216bec

updated conf.py

499fc1b

updated plot_svm_hyperplane_margin.py

7aa4078

updated plot_svm_hyperplane_margin.py

b50fefa

updated plot_svm_hyperplane_margin.py

902a787

updated plot_svm_hyperplane_margin.py

d29225d

updated plot_svm_hyperplane_margin.py

9fb3c50

updated plot_svm_hyperplane_margin.py

2de5c47

updated plot_svm_hyperplane_margin.py

49ee55e

updated plot_svm_hyperplane_margin.py

2ef1d5f

SwathiR1999 added 6 commits May 7, 2025 08:28

Merge branch 'main' of https://github.com/scikit-learn/scikit-learn i…

53bd994

…nto doc_addlink

formatted plot_svm_hyperplane_margin.py

2257e4f

formatted plot_svm_hyperplane_margin.py

a730252

formatted plot_svm_hyperplane_margin.py

bd74817

Update SVM docs and remove deprecated example

8b1eb56

- Modified \`doc/conf.py\` and \`doc/modules/svm.rst\` to update documentation and add example reference.\n- Removed outdated example: \`examples/svm/plot_svm_margin.py\`.

modified svm.rst

762644d

modified svm.rst

8612d2d

StefanieSenger reviewed May 28, 2025

View reviewed changes

	========================================================================
	=========================================================================

	decreasing C corresponds to more regularization (see example below).
	decreasing C corresponds to more regularization.

		.. rubric:: Examples

		* :ref:`sphx_glr_auto_examples_svm_plot_svm_hyperplane_margin.py`

	this penalty, and as a result, acts as an inverse regularization parameter
	this penalty, and as a result, acts as an inverse regularization parameter:

Uh oh!

DOC Merge plot_svm_margin.py and plot_separating_hyperplane.py into plot_svm_hyperplane_margin.py #31045

Are you sure you want to change the base?

DOC Merge plot_svm_margin.py and plot_separating_hyperplane.py into plot_svm_hyperplane_margin.py #31045

Uh oh!

Conversation

SwathiR1999 commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

github-actions bot commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Linting Passed

Uh oh!

StefanieSenger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanieSenger Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SwathiR1999 commented May 3, 2025

Uh oh!

StefanieSenger commented May 3, 2025

Uh oh!

SwathiR1999 commented May 7, 2025

Uh oh!

StefanieSenger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SwathiR1999 commented Mar 21, 2025 •

edited

Loading

github-actions bot commented Mar 21, 2025 •

edited

Loading

StefanieSenger Apr 28, 2025 •

edited

Loading