Skip to content

[MRG]Allow Gaussian process kernels on structured data #13936 #13954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

yhtang
Copy link
Contributor

@yhtang yhtang commented May 26, 2019

Fixes #13936.

Made the Gaussian process regressor and classifier compatible with either structure- or vector-based kernels. Specifically:

  1. Added an vector_X_only() abstract method to the GP kernel base class.
  2. Added the VectorOnlyKernelMixin and StructureOrGenericKernelMixin base classes to overload the vector_X_only() method for kernels that can only operate on fixed-length feature vectors and that can work on structured and/or generic data, respectively.
  3. Let the GP regressor and classifier use different check_array and check_X_y parameters for vector/structured input. I.e., do not force the X and Y array to be at least 2D and numeric if the kernel can operate on structured input. Updated documentations accordingly.'
  4. Provided a minimal example of GP regression and classification using a sequence kernel on variable-length strings at plot_gpr_on_structured_data.py.

@jnothman
Copy link
Member

jnothman commented May 26, 2019 via email

@yhtang yhtang force-pushed the dev-GP-on-structured-data branch 3 times, most recently from ae89b25 to 8fb6e85 Compare May 27, 2019 06:40
@yhtang yhtang changed the title Allow Gaussian process kernels on structured data #13936 [WIP]Allow Gaussian process kernels on structured data #13936 May 27, 2019
@yhtang yhtang force-pushed the dev-GP-on-structured-data branch 5 times, most recently from 43a7d2b to 21e3645 Compare May 28, 2019 03:13
@yhtang yhtang changed the title [WIP]Allow Gaussian process kernels on structured data #13936 [MGR]Allow Gaussian process kernels on structured data #13936 May 28, 2019
@yhtang yhtang changed the title [MGR]Allow Gaussian process kernels on structured data #13936 [MRG]Allow Gaussian process kernels on structured data #13936 May 28, 2019
@yhtang
Copy link
Contributor Author

yhtang commented May 28, 2019

Thanks for the pr. I don't think we currently have any examples stored in ipynb, even if they export to ipynb. Apart from anything else, it is a nuisance with version control. Please look at our other examples and format similarly.

Done. Thanks for the comment!

@yhtang yhtang force-pushed the dev-GP-on-structured-data branch from 21e3645 to 3da4e6f Compare May 28, 2019 21:42
@jnothman
Copy link
Member

Please avoid force-pushing. It makes it quite difficult to see changes, and risks overwriting core devs' tweaks to your work. Instead, always append commits, and we will squash upon merge.

@yhtang
Copy link
Contributor Author

yhtang commented May 29, 2019

Please avoid force-pushing. It makes it quite difficult to see changes, and risks overwriting core devs' tweaks to your work. Instead, always append commits, and we will squash upon merge.

I see. Got it.

@yhtang
Copy link
Contributor Author

yhtang commented May 30, 2019

@jnothman Could you please review it and let me know if there are anything to be resolved?

@jnothman
Copy link
Member

Could you please review it and let me know if there are anything to be resolved?

When I find time for it, yes!

Copy link
Member

@jnothman jnothman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. I find this quite nice, thank you.

Please add an entry to the change log at doc/whats_new/v0.22.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:

feature vector form. This is enabled through the use of kernel functions
that can directly operate on discrete structures such as variable-length
sequences, trees, and graphs.
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would help to briefly describe the kernel, and to also describe the plots. It's unclear, for instance, that you do not include two samples in training in the Regression example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -355,6 +356,10 @@ def diag(self, X):
def is_stationary(self):
"""Returns whether the kernel is stationary. """

@abstractmethod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely comfortable doing this as it will break any custom kernels users have created... they will not be able to construct their kernel.

Could we instead default to true for the sake of backwards compatibility?

Also, should this be a method or a property?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it defaults to True, and removed VectorKernelMixin because it is now redundant. I think it is fine either as a method or property, but what confuses me is that is_stationary is a method instead of a property. Is there some particular thought behind this?

kernel.kernel.requires_vector_input())
if isinstance(kernel, KernelOperator):
assert_equal(kernel.requires_vector_input(),
np.any([kernel.k1.requires_vector_input(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just use or instead of np.any??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

@@ -636,6 +671,11 @@ def is_stationary(self):
"""Returns whether the kernel is stationary. """
return self.k1.is_stationary() and self.k2.is_stationary()

def requires_vector_input(self):
"""Returns whether the kernel is stationary. """
return np.any([self.k1.requires_vector_input(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use or instead of np.any?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

@jnothman
Copy link
Member

jnothman commented Aug 5, 2019

Please merge in master and resolve conflicts with it. And ask for help if you're having trouble

@yhtang
Copy link
Contributor Author

yhtang commented Aug 6, 2019 via email

yhtang and others added 5 commits November 6, 2019 10:25
…ctured data in addition to fixed-length feature vectors. Made the Gaussian process regressor and classifier compatible with either structure- or vector-based kernels.
convention-conforming variable names

Co-Authored-By: Joel Nothman <joel.nothman@gmail.com>
convention-conforming variable names

Co-Authored-By: Joel Nothman <joel.nothman@gmail.com>
Co-Authored-By: Joel Nothman <joel.nothman@gmail.com>
@yhtang yhtang force-pushed the dev-GP-on-structured-data branch from 3d4503b to 29978c3 Compare November 6, 2019 20:32
@yhtang
Copy link
Contributor Author

yhtang commented Nov 7, 2019

It's been a while since last time and it made rebasing somehow difficult. As such, I've started from scratch on a different branch and created a new PR #15557 to keep things clean. @jnothman I've incorporated all your suggestions in the new branch/PR. Could we keep subsequent discussions in the new PR? Thanks!

@yhtang yhtang closed this Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GaussianProcessRegressor unnecessarily forcing training samples to be array_like
2 participants