Skip to content

ENH Enhanced correlation models and noise estimation for Gaussian Process #2930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from

Conversation

jmetzen
Copy link
Member

@jmetzen jmetzen commented Mar 3, 2014

Support in Gaussian-Process regression for enhanced correlation models and learning the noise magnitude (the nugget) from training data.

Correlation models have been extended as follows:

  • Matern correlation models for nu=1.5 and nu=2.5 have been added (see https://en.wikipedia.org/wiki/Mat%C3%A9rn_covariance_function). An example script showing the potential benefit of the Matern correlation model compared to squared-exponential and absolute-exponential was added under examples/gaussian_process/plot_matern_kernel.py (see attached image)
  • squared_exponential, absolute_exponential and Matern correlation models support factor analysis distance. This can be seen as an extension of learning dimension-specific length scales in which also correlations of feature dimensions can be taken into account. See Rasmussen and Williams 2006, p107 for details. An example script showing the potential benefit of this extension was added under examples/gaussian_process/plot_gp_learning_curve.py (see attached image). This feature required that correlation modes get passed the componentwise differences rather than the componentwise distances (their absolute value).

Learning the noise (the "nugget effect") by GaussianProcess is now supported by setting the parameter learn_nugget to True. This allows to learn a homoscedastic noise model, i.e., it assumes that the noise has globally the same magnitude. The script examples/gaussian_process/plot_gp_regression.py was modified accordingly, i.e., it learns the noise magnitude and does not rely on specifying it externaly.

Furthermore, a typo in gp_diabetes_dataset.py was fixed and a not yet merged bugfix (#2867 and #2798) is included.

To be discussed:

  • The factor analysis distance has hyperparameters that can can take on arbitrary real values (not just positive ones). Since sklearn enforces the hyperparameters theta to be positive, this is internally handled by taking the log of the corresponding components of theta. Are there better ideas?
  • Learning the noise (the nugget) is internally handled by appending it to the vector theta in _arg_max_reduced_likelihood_function(). This was the way which required the least changes in the current implementation but is not necessarily the best way. Are their any opinions on that?

plot_gp_learning_curve
plot_matern_kernel

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 87475eb on jmetzen:gp_correlation_models into e721508 on scikit-learn:master.

@jmetzen
Copy link
Member Author

jmetzen commented Jul 15, 2014

This PR is superseded by #3388

@jmetzen jmetzen closed this Jul 15, 2014
@jmetzen jmetzen deleted the gp_correlation_models branch September 15, 2014 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants