[MRG+2] Gaussian process revisited#4270

Closed

jmetzen wants to merge 165 commits intoscikit-learn:masterfrom

Commits on Aug 4, 2015

ADD New kernel module for GPs which supports kernel engineering
Jan Hendrik Metzen
committed
ADD New implementation of GaussianProcessRegression
Jan Hendrik Metzen
committed
ADD Gaussian Process classification based on Laplace approximation
Jan Hendrik Metzen
committed
TST Tests for GP regression and classification
Jan Hendrik Metzen
committed
ADD Examples for GP classification
Jan Hendrik Metzen
committed
ADD Examples for GP regression
Jan Hendrik Metzen
committed
FIX cross_correlation of anisotropic RBF kernel computed correctly
Jan Hendrik Metzen
committed
FIX Fix some bugs in kernel module discovered during writing unit test
Jan Hendrik Metzen
committed
TST Test for GP kernels
Jan Hendrik Metzen
committed
REFACTOR auto_correlation() -> auto(), cross_correlation() -> cross()
Jan Hendrik Metzen
committed
DOC GP's kernel module documented
Jan Hendrik Metzen
committed
DOC Revised GP examples (added doc etc.)
Jan Hendrik Metzen
committed
ENH GPR.predict allows returning standard-deviations of predictions
Jan Hendrik Metzen
committed
DOC Revised examples based on @eickenberg's comments
Jan Hendrik Metzen
committed
MISC Revised gaussian_process package based on @eickenberg's suggestions
Jan Hendrik Metzen
committed
TST Dealing with failing tests from test_common
Jan Hendrik Metzen
committed
REFACTOR Merging kernel methods auto() and cross() in __call__()
Jan Hendrik Metzen
committed
ENH Adding PairwiseKernel for GPs (wraps kernels from metrics.pairwise)
Jan Hendrik Metzen
committed
REFACTOR GaussianProcessClassification/Regression -> ...Classifier/Regressor
Jan Hendrik Metzen
committed
ENH More efficient approximation of gradient in PairwiseKernel
Jan Hendrik Metzen
committed
ENH Avoid to create square matrix W_sr in all but one place (TODO)
Jan Hendrik Metzen
committed
ENH Adding RationalQuadratic and ExpSineSquared kernels
Jan Hendrik Metzen
committed
FIX Dealing with singular kernel matrix (-inf log-marginal-likelihood)
Jan Hendrik Metzen
committed
ENH Adding DotProduct kernel and fixing ExpSine2 kernel
Jan Hendrik Metzen
committed
DOC Initial version of Mauna Loa CO2 example
Jan Hendrik Metzen
committed
REFACTOR Hyperparameter optimization in GPs enabled explicitly by optimizer parameter
Jan Hendrik Metzen
committed
DOC Revising CO2 example
Jan Hendrik Metzen
committed
TST Cover more kernels in tests
Jan Hendrik Metzen
committed
ENH Analytic gradients for ExpSine2, RationalQuadratic, DotProduct kernels
Jan Hendrik Metzen
committed
DOC Extended documentation of GP kernels module
Jan Hendrik Metzen
committed
MISC Reorganizing kernel module
Jan Hendrik Metzen
committed
ENH Adding method is_stationary to GP kernels
Jan Hendrik Metzen
committed
TST Testing kernel stationarity
Jan Hendrik Metzen
committed
DOC Illustrating different kernels in plot_gpr_prior_posterior.py
Jan Hendrik Metzen
committed
FIX test_converged_to_local_maximum deals with convergence to boundary
Jan Hendrik Metzen
committed
ENH More efficient computation of lml-gradient in GPR using einsum
Jan Hendrik Metzen
committed
FIX GPs do not modify kernel attribute but store optimized kernels in kernel_
Jan Hendrik Metzen
committed
REFACTOR GP kernels use separate specification of bounds and params (no joint param_space)
Jan Hendrik Metzen
committed
REFACTOR REnamed kernel property params to theta
Jan Hendrik Metzen
committed
ENH Kernel implements equality check and get_params() method
Jan Hendrik Metzen
committed
TST Tests for cloning kernels with sklearn.base's clone function
Jan Hendrik Metzen
committed
REFACTOR Replacing all calls to deepcopy() by clone()
Jan Hendrik Metzen
committed
TST Replace call to deepcopy() by clone()
Jan Hendrik Metzen
committed
ENH Added Exponentiation kernel (DotProduct no longer has internal degree)
Jan Hendrik Metzen
committed
TST test_kernels tests Exponentiation kernel
Jan Hendrik Metzen
committed
FIX Using numpy-downward compatible ones_lile() instead of full_like()
Jan Hendrik Metzen
committed
DOC Reorganizing gaussian_process examples
Jan Hendrik Metzen
committed
FIX Minor bugfixes and downward-compatibility/python3 issue
Jan Hendrik Metzen
committed
REFACTOR Rename GPR method sample to sample_y
Jan Hendrik Metzen
committed
REFACTOR Compute standard deviation of predictive distribution at n points in O(n)
Jan Hendrik Metzen
committed
FIX GPC uses LabelEncoder in fit() and returns (n_samples, n_classes) array in predict_proba()
Jan Hendrik Metzen
committed
FIX Adapted examples and tests to GPC.predict_proba() adapted return shape
Jan Hendrik Metzen
committed
FIX GaussianProcessRegressor inherits from RegressorMixin
Jan Hendrik Metzen
committed
ENH Reducing memory-consumption of GPC.predict_proba()
Jan Hendrik Metzen
committed
FIX Numpy downward-compatible way of computing sum over two axis
Jan Hendrik Metzen
committed
ENH GPC can use warm-starts in _posterior_mode()
Jan Hendrik Metzen
committed
ENH Adding diag(X) method to GP kernels.
Jan Hendrik Metzen
committed
DOC Adding GPC to plot_classifier_comparison.py script and enabling warm_start
Jan Hendrik Metzen
committed
DOC gpr module fully documented
Jan Hendrik Metzen
committed
DOC gpc module fully documented
Jan Hendrik Metzen
committed
REFACTOR Kernel hyperparamters are now set via their name
Jan Hendrik Metzen
committed
TST Adapted tests to changes in kernels and added additional test for theta property
Jan Hendrik Metzen
committed
DOC Adapted GP examples to changes in kernel module
Jan Hendrik Metzen
committed
FIX Kernel property theta can deal with anisotropic RBF kernel
Jan Hendrik Metzen
committed
FIX Dealing with anisotropic length-scales correctly
Jan Hendrik Metzen
committed
DOC Revised docstrings of kernels.py
Jan Hendrik Metzen
committed
REFACTOR Removed literal form of specifying bounds for ConstantKernel
Jan Hendrik Metzen
committed
REFACTOR Changed order of l and alpha in RationalQuadratic kernel
Jan Hendrik Metzen
committed
ENH GPR can use individual noise-levels for each datapoint
Jan Hendrik Metzen
committed
ENH GaussianProcessRegressor supports multiple restarts of the optimizer
Jan Hendrik Metzen
committed
TST An increasing number of random-starts of GPR only increases the LML of the chosen theta.
Jan Hendrik Metzen
committed
DOC Adapted old example of fitting with noisy targets to new GPR class
Jan Hendrik Metzen
committed
ENH GaussianProcessClassifier supports multiple restarts of the optimizer
Jan Hendrik Metzen
committed
TST An increasing number of random-starts of GPC only increases the LML of the chosen theta.
Jan Hendrik Metzen
committed
FIX Fetching CO2 data from mldata
Jan Hendrik Metzen
committed
FIX GP kernels check for type equality before checking attribute equality
Jan Hendrik Metzen
committed
FIX cross_validation and pairwise_kernels treat GP kernels as special case
Jan Hendrik Metzen
committed
DOC Adding example comparing kernel ridge with GPR.
Jan Hendrik Metzen
committed
DOC Revised plot_gpc example
Jan Hendrik Metzen
committed
DOC Adding documentation for GP kernels
Jan Hendrik Metzen
committed
DOC Documentation of the CO2 example
Jan Hendrik Metzen
committed
DOC Backported documentation to kernels module
Jan Hendrik Metzen
committed
DOC Extending further the GP documentation
Jan Hendrik Metzen
committed
ENH GPR supports normalization of target values
Jan Hendrik Metzen
committed
TST Testing normalization of output values in GPR
Jan Hendrik Metzen
committed
ENH GPR supports target value with more than one dimension
Jan Hendrik Metzen
committed
TST Testing multi-out GPR
Jan Hendrik Metzen
committed
FIX Not performing hyperparamter-tuning in GPR if there are no hyperparameters
Jan Hendrik Metzen
committed
ENH GPR can use an externally defined optimizer for hyperparameter tuning
Jan Hendrik Metzen
committed
TST Testing GPR with custom external optimizer
Jan Hendrik Metzen
committed
REFACTOR GP kernel hyperparameters theta and their bounds are log-tranformed
Jan Hendrik Metzen
committed
TST Adapting GP tests to changed hyperparameter representation
Jan Hendrik Metzen
committed
DOCAdapting GP examples to changed hyperparameter representation
Jan Hendrik Metzen
committed
REFACTOR Remove theta_ attribute from GPR and GPC
Jan Hendrik Metzen
committed
ENH GPC can use an externally defined optimizer for hyperparameter tuning
Jan Hendrik Metzen
committed
TST Testing GPC with custom external optimizer
Jan Hendrik Metzen
committed
REFACTOR normalize_y in GPR does not modify standard-deviation
Jan Hendrik Metzen
committed
FIX Sampling theta_intial uniformly from log-bounds in GPR
Jan Hendrik Metzen
committed
FIX Correcly composing theta vector in GP kernels
Jan Hendrik Metzen
committed
FIX Ensuring that predicted variances are never negative
Jan Hendrik Metzen
committed
ENH Adding Matern kernel for Gaussian Processes
Jan Hendrik Metzen
committed
DOC Adding documentation on Matern kernel
Jan Hendrik Metzen
committed
TST Added test checking that GPR can deal with different outputs for same input
Jan Hendrik Metzen
committed
FIX Checking for correct dimensionality of anisotropic kernels
Jan Hendrik Metzen
committed
FIX RBF and Matern kernel handle 1d length 1 length-scales correctly
Jan Hendrik Metzen
committed
DOC Added further doc for GPC
Jan Hendrik Metzen
committed
ADD CompoundKernel for multi-class GP classification
Jan Hendrik Metzen
committed
FIX GPR's random_state is cloned correctly
Jan Hendrik Metzen
committed
ENH Support for multi-class GPC added based on OneVsRestClassifier
Jan Hendrik Metzen
committed
DOC Adding GPC to plot_classification_probability.py example
Jan Hendrik Metzen
committed
TST Testing GPC for multi-class classification problems
Jan Hendrik Metzen
committed
FIX Several minor changes/fixes in gaussian_process module
Jan Hendrik Metzen
committed
FIX Using six.string_types instead of basestring (python3 compatibility)
Jan Hendrik Metzen
committed
FIX Excluding WhiteKernel from one unit-test
Jan Hendrik Metzen
committed
TST Extending testing of multi-class GP
Jan Hendrik Metzen
committed
DOC Updated documentation of Gaussian process classifier
Jan Hendrik Metzen
committed
ADD Example illustrating Gaussian process classifier on iris dataset
Jan Hendrik Metzen
committed
TST Fixing some GP-releated issues in estimator checks
Jan Hendrik Metzen
committed
TST Fix GPC and GPR such that all tests are ok
Jan Hendrik Metzen
committed
TST Further GP-related fixes
Jan Hendrik Metzen
committed
FIX Hiding method decision_function in GaussianProcessClassifier
Jan Hendrik Metzen
committed
TST GaussianProcessRegressor added to MULTI_OUTPUT estimators
Jan Hendrik Metzen
committed
PEP8 Fixing PEP8 issues in gaussian_process package
Jan Hendrik Metzen
committed
ENH X_train_ can be copied if requested
Jan Hendrik Metzen
committed
ENH Adding max_iter parameter for Newton iteration in GPC
Jan Hendrik Metzen
committed
COS Removing keyword magic from PairwiseKernel
Jan Hendrik Metzen
committed
FIX Added missing import of ConstantKernel in gpc.py
Jan Hendrik Metzen
committed
TST Excluded GPC from test_non_transformer_estimators_n_iter()
Jan Hendrik Metzen
committed

Commits on Aug 5, 2015

MISC Deprecating legacy GP implementation
Jan Hendrik Metzen
committed
COS Making BinaryGaussianProcessClassifierLaplace private
Jan Hendrik Metzen
committed
DOC Cleaning up documenation of legacy GPs
Jan Hendrik Metzen
committed
DOC Updating GPC XOR example (showing also results for DotProduct kernel)
Jan Hendrik Metzen
committed
FIX Remove BinaryGaussianProcessClassifierLaplace from packages __all__ list
Jan Hendrik Metzen
committed
FIX Undoing regression_model and correlation_model deprecation
Jan Hendrik Metzen
committed

Commits on Aug 6, 2015

TST Testing n_jobs option of GaussianProcessClassifier
Jan Hendrik Metzen
committed
REF Realizing multi-class GPC bei delegating to OneVsRest or OneVsOne
Jan Hendrik Metzen
committed
DOC Updating documentation of multi-class GPC
Jan Hendrik Metzen
committed

Commits on Aug 10, 2015

REF Several small changes based on @eickenberg's suggestions
Jan Hendrik Metzen
committed

Commits on Aug 11, 2015

ENH More efficient computation of K_gradient for anisotropic Matern kernel
Jan Hendrik Metzen
committed
REF Renaming sigma_squared_n to alpha (consistency with Ridge)
Jan Hendrik Metzen
committed

Commits on Aug 12, 2015

ENH Full support of (deep) get_params() and set_params() in GP kernels
Jan Hendrik Metzen
committed

Commits on Aug 15, 2015

DOC Revising GP documentation based on @kastnerkyle comments
Jan Hendrik Metzen
committed

Commits on Aug 17, 2015

FIX theta_vars correctly supported in KernelOperators
Jan Hendrik Metzen
committed
TST Adding two unittests for testing GP kernels
Jan Hendrik Metzen
committed
REF Removing theta_vars attribute from kernels and using Hyperparameter instances instead
Jan Hendrik Metzen
committed

Commits on Aug 19, 2015

DEL Removing bounds.setter in kernels.py
Jan Hendrik Metzen
committed
REF Renaming max_iter to max_iter_predict in GaussianProcessClassifier
Jan Hendrik Metzen
committed
DOC Documenting GP kernel API
Jan Hendrik Metzen
committed
DOC Fixing some minor issues in narrative doc of GPs
Jan Hendrik Metzen
committed
FIX Fix failing tests (doctest, order of hyperparameters, pairwise_kernels)
Jan Hendrik Metzen
committed
FIX python3 compatibility in doctest
Jan Hendrik Metzen
committed
FIX Fixing test_kernel_theta
Jan Hendrik Metzen
committed
FIX Further python3 related fixes
Jan Hendrik Metzen
committed
FIX Fixing a python2.6-related issue
Jan Hendrik Metzen
committed

Commits on Aug 20, 2015

FIX Changing bounds in GP doctest
Jan Hendrik Metzen
committed
MISC copy_X_train defaults to True in GPR and GPC
Jan Hendrik Metzen
committed
FIX Fix message of ValueError for wrong shape of alpha in GPR
Jan Hendrik Metzen
committed

Commits on Aug 30, 2015

PEP8 Making pep8 and pyflakes happy
Jan Hendrik Metzen
committed

Commits on Aug 31, 2015

REF More meaningful names for hyperparameters of GP kernels
Jan Hendrik Metzen
committed
MISC Changing default for copy_X_train to True in GaussianProcessClassifier
Jan Hendrik Metzen
committed

Commits on Sep 7, 2015

FIX String comparison via equality and not identity in Hyperparameter.__init__
Jan Hendrik Metzen
committed
ENH GPR.log_marginal_likelihood() returns the current log-likelihood if no theta vector is provided
Jan Hendrik Metzen
committed
FIX Enforcing y to be numeric in GPR and fixing python3 issue with maps
Jan Hendrik Metzen
committed
ADD Mixins for normalized and stationary kernels
Jan Hendrik Metzen
committed

Commits on Sep 14, 2015

TST test_lml_precomputed() checks only for equality in first 7 digits
Jan Hendrik Metzen
committed