GMM -> Gaussian mixtures

I guess the rendered page is not as intended.

I was wondering what will be the result and it's ugly :).
I'll change that.

Since this is not a class ref, could we have this as Dirichlet Gaussian Mixture?

Cut the sentence:

representation). The number of components actually used almost always depends on the data.

This reference is broken.

Two problems:

• This new line breaks the sphinx rendering in classes.rst. I am not sure how this should be fixed.
• The default value is 'dirichlet_process', not 'full'.

I don't know how to fix the first problem because if I put it on a unique line pep8 will not be happy.

I don't agree with this analysis. Let me suggest the following instead:

This example demonstrates the behavior of Gaussian mixture models on data that was not generated by a mixture of Gaussian random variables. The dataset is formed by 100 points loosely spaced following a noisy sine curve. There is therefore no ground truth value for the number of Gaussian components.

The first model is a classical Gaussian Mixture Model with 10 components fit with the Expectation Maximization algorithm.

The second model is a Bayesian Gaussian Mixture Model with a Dirichlet process prior fit with variational inference. The low value of the concentration prior makes the model favor a lower number of active components. This models "decides" to focus its modeling power on the big picture of the structure of the datasets: groups of points with alternating directions modeled by non-spherical covariance matrices. Those alternating directions roughly capture the alternating nature of the original sine signal.

The third model is also Bayesian Gaussian Mixture Model with a Dirichlet process prior but this time the value of the concentration prior is higher giving the model more liberty to try to model the finer-grained structure of the data. The result is a mixture with a larger number of active components that is similar to the first model where we decided to fix the number of components to 10 arbitrarily.

Which model is the best is a matter of subjective judgement: do we want to favor models that only capture the big picture to summarize and explain most of the structure of the data while ignoring the details or do we prefer models that closely follow the high density regions of the signal?

The last two panels show how we can sample from the last two models. The resulting samples distributions do not look exactly like the original data distribution. The difference primarily stems from the approximation error we made by using a model that assumes that the data was generated by a finite number of Gaussian components instead of a continuous noisy sine curve.