[DOC] GSoC BayesianGaussianMixture formula #7284

tguillemot · 2016-08-29T13:47:30Z

This PR adds the doc concerning the derivation formula of the BayesianGaussianMixture class.

I've done a separate PR because this is just formula and I thought it wil be easier to review if it was extracted on another PR.

Of course, this can't be merge before #6651.

ogrisel · 2016-08-29T14:02:09Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+   \end{eqnarray*}
+
+References
+----------


Some references are missing here :)

Oups indeed.

TomDLT · 2016-08-29T14:27:34Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+
+Notation
+--------
+Add notation of NK xk and Sk


tguillemot · 2016-08-29T14:32:58Z

Here is the link to the compiled doc

ogrisel · 2016-08-30T07:53:12Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+    \bar{\mathbf{x}}_k & = & \frac{1}{N_k} \sum_{n=1}^N {r_{nk} \mathbf{x}_n}
+    \mathbf{S}_k & = & \frac{1}{N_k} \sum_{n=1}^N {r_{nk} (\mathbf{x}_n - \bar{\mathbf{x}}_k) (\mathbf{x}_n - \bar{\mathbf{x}}_k)^T}
+
+   \end{eqnarray*}


I got the following LaTeX rendering error:

Extra alignment tab has been changed to \cr. <template> \egroup \endtemplate l.15 \mathbf{S}_k & = & \frac{1}{N_k} \sum_{n=1}^N {r_{nk} (\mathbf{x}_n - ... ! Missing $ inserted. <inserted text> $ l.16 ! Missing } inserted. <inserted text> } l.16 ! Missing } inserted. <inserted text> } l.16 ! Missing \cr inserted. <inserted text> \cr l.16 ! Missing { inserted. <inserted text> { l.16 ! Missing $ inserted. <inserted text> $ l.16 ! Missing $$ inserted. <to be read again> \par l.16 ! LaTeX Error: Too many columns in eqnarray environment. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.17 \end{eqnarray*} ! Misplaced \cr. \@@eqncr ...l \@eqnswtrue \global \@eqcnt \z@ \cr l.17 \end{eqnarray*} ! Extra }, or forgotten \endgroup. \endeqnarray ->\@@eqncr \egroup \global \advance \c@equation \m@ne $$\@ignor... l.17 \end{eqnarray*} ! Missing $ inserted. <inserted text> $ l.17 \end{eqnarray*} ! Display math should end with $$. <to be read again> \endgroup l.17 \end{eqnarray*} [1]

tguillemot · 2016-08-30T09:36:12Z

New link

TomDLT · 2016-08-30T09:38:13Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+      <http://www.springer.com/kr/book/9780387310732>`_,
+      Bishop, Christopher M. Springer, Vol. 4 No. 4, 2006.
+
+While the book presents the parts of the variationnl inference algorithm, it


variationnl

ogrisel · 2016-09-01T09:30:19Z

Please rebase this on top of master to make sure that the class link works as expected and add a link to the derivation page from the narrative documentation on variational Bayes mixture models.

ogrisel · 2016-09-01T09:31:40Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+Estimate Z
+~~~~~~~~~~
+
+Using the general result of variational inference, and take the expectation of


"taking". Also I don't think the comma is necessary before "and".

tguillemot · 2016-09-02T08:32:17Z

Rebase done.
This is the new doc

ogrisel · 2016-09-08T14:38:54Z

C(\boldsymbol{\alpha}_0) is not defined. As alpha_0 is symmetric (same value for all the components, we should have:

C(\alpha_0) = \frac{\Gamma(\alpha_0){\Gamma(\alpha_0)^K}

I think.

ogrisel · 2016-10-10T09:22:48Z

Please update the link at the end of mixture.rst and remove dp-derivation.rst.

ogrisel · 2016-10-10T09:55:28Z

BTW, this PR need a rebase on top of the current master and also it should be stated that this derivation is valid only for the dirichlet distribution prior type (not the dirichlet process prior type).

ogrisel

I did another pass and found the following issues:

ogrisel · 2016-10-10T13:21:07Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+   \end{eqnarray*}
+
+Estimate lower bound terms
+--------------------------


I think this should be a subsection of the previous one which means using ~~~~~ instead of ----.

ogrisel · 2016-10-10T13:21:26Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+   \end{eqnarray*}
+
+Lower bound combination and simplifications
+...........................................


Again this should be ~~~~.

ogrisel · 2016-10-10T13:24:41Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

+
+Lower bound
+...........
+The lower bound :math:`\mathcal{L}` is defined by


Please add a comment as to why this is useful to estimate the lower bound:

This variational lower bound is guaranteed to be non-decreasing. It can therefore be computed to monitor the convergence of the model. It can also be used to catch implementation bugs if an update causes a decrease of this quantity.

ogrisel · 2016-10-10T13:26:56Z

doc/modules/mixture.rst

+.. toctree::
+    :hidden:
+
+    bayesian_gaussian_mixture_derivation.rst


The derivation has a weird level of nesting for the headers. Maybe this is caused by inconsistencies in the underline style. They should be updated to use the same ordering for underline styles as in the mixture.rst file.

I've removed toctree:: :hidden:

Conflicts: examples/mixture/plot_bayesian_gaussian_mixture.py

tguillemot · 2016-10-17T13:36:55Z

Thanks @ogrisel

cmarmo

@tguillemot are you still interested in working on this? I know it has been a while, but if so, do you mind triggering a build as we can access the rendering of the documentation? Thanks a lot for your patience!

cmarmo · 2020-08-20T15:39:29Z

doc/modules/bayesian_gaussian_mixture_derivation.rst

@@ -0,0 +1,636 @@
+.. _bayesain_gaussian_mixture_derivation:


Suggested change

.. _bayesain_gaussian_mixture_derivation:

.. _bayesian_gaussian_mixture_derivation:

lorentzenchr · 2025-03-31T14:57:35Z

Seems very stalled and much too many formulae, so closing.

tguillemot changed the title ~~[MRG] GSoC BayesianGaussianMixture formulas~~ [MRG] GSoC BayesianGaussianMixture formula Aug 29, 2016

tguillemot changed the title ~~[MRG] GSoC BayesianGaussianMixture formula~~ [DOC] GSoC BayesianGaussianMixture formula Aug 29, 2016

ogrisel reviewed Aug 29, 2016
View reviewed changes

ogrisel mentioned this pull request Aug 29, 2016

[MRG+1] Bayesian Gaussian Mixture (Integration of GSoC2015 -- second step) #6651

Merged

8 tasks

TomDLT reviewed Aug 29, 2016
View reviewed changes

doc/modules/bayesian_gaussian_mixture_derivation.rst Outdated

Notation

--------

Add notation of NK xk and Sk

Copy link

Member

TomDLT Aug 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

tguillemot force-pushed the GSoC-BGM-formulas branch from 7bc59ba to fb16cbb Compare August 29, 2016 15:01

ogrisel reviewed Aug 30, 2016
View reviewed changes

TomDLT reviewed Aug 30, 2016
View reviewed changes

tguillemot force-pushed the GSoC-BGM-formulas branch 3 times, most recently from e258117 to 426a6a2 Compare August 30, 2016 15:51

ogrisel reviewed Sep 1, 2016
View reviewed changes

tguillemot force-pushed the GSoC-BGM-formulas branch 3 times, most recently from 6bf7b9a to b25523e Compare September 1, 2016 16:04

RPGOne approved these changes Oct 10, 2016

View reviewed changes

ogrisel requested changes Oct 10, 2016

View reviewed changes

tguillemot added 2 commits October 17, 2016 12:06

Add the derivation formulas of the BGM.

3e97e51

Conflicts: examples/mixture/plot_bayesian_gaussian_mixture.py

Add references + intro + notations.

7e0f6f9

tguillemot added 2 commits October 17, 2016 12:07

Add r_nk definition.

494b5ed

Fix review.

b060c15

tguillemot force-pushed the GSoC-BGM-formulas branch 4 times, most recently from fb9e809 to 672f448 Compare October 17, 2016 13:36

[ci skip] Correct the doc.

e513ab2

tguillemot force-pushed the GSoC-BGM-formulas branch from 672f448 to e513ab2 Compare October 17, 2016 13:39

lesteve mentioned this pull request Mar 16, 2018

dp-derivation.rst isn't included in any toctree #10814

Closed

tguillemot mentioned this pull request May 23, 2018

Question about the computation of the lower bound in mixture.bayesian_mixture.py #8823

Open

amueller added the Waiting for Reviewer label Aug 5, 2019

cmarmo reviewed Aug 20, 2020

View reviewed changes

cmarmo added Stalled help wanted and removed Waiting for Reviewer labels Dec 15, 2020

Base automatically changed from master to main January 22, 2021 10:49

cmarmo added the module:mixture label Dec 16, 2021

lorentzenchr closed this Mar 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DOC] GSoC BayesianGaussianMixture formula #7284

[DOC] GSoC BayesianGaussianMixture formula #7284

tguillemot commented Aug 29, 2016

ogrisel Aug 29, 2016

tguillemot Aug 29, 2016

TomDLT Aug 29, 2016

tguillemot commented Aug 29, 2016 •

edited

Loading

ogrisel Aug 30, 2016 •

edited

Loading

tguillemot commented Aug 30, 2016

TomDLT Aug 30, 2016

ogrisel commented Sep 1, 2016

ogrisel Sep 1, 2016

tguillemot commented Sep 2, 2016

ogrisel commented Sep 8, 2016 •

edited

Loading

ogrisel commented Oct 10, 2016

ogrisel commented Oct 10, 2016

ogrisel left a comment

ogrisel Oct 10, 2016

ogrisel Oct 10, 2016

ogrisel Oct 10, 2016

ogrisel Oct 10, 2016

tguillemot Oct 17, 2016

tguillemot commented Oct 17, 2016

cmarmo left a comment

cmarmo Aug 20, 2020

lorentzenchr commented Mar 31, 2025

	.. _bayesain_gaussian_mixture_derivation:
	.. _bayesian_gaussian_mixture_derivation:

[DOC] GSoC BayesianGaussianMixture formula #7284

[DOC] GSoC BayesianGaussianMixture formula #7284

Conversation

tguillemot commented Aug 29, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tguillemot commented Aug 29, 2016 • edited Loading

ogrisel Aug 30, 2016 • edited Loading

Choose a reason for hiding this comment

tguillemot commented Aug 30, 2016

Choose a reason for hiding this comment

ogrisel commented Sep 1, 2016

Choose a reason for hiding this comment

tguillemot commented Sep 2, 2016

ogrisel commented Sep 8, 2016 • edited Loading

ogrisel commented Oct 10, 2016

ogrisel commented Oct 10, 2016

ogrisel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tguillemot commented Oct 17, 2016

cmarmo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lorentzenchr commented Mar 31, 2025

tguillemot commented Aug 29, 2016 •

edited

Loading

ogrisel Aug 30, 2016 •

edited

Loading

ogrisel commented Sep 8, 2016 •

edited

Loading