Skip to content

new feature: add LOBPCG as an SVD solver in PCA #12079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
lobpcg opened this issue Sep 14, 2018 · 6 comments
Closed

new feature: add LOBPCG as an SVD solver in PCA #12079

lobpcg opened this issue Sep 14, 2018 · 6 comments

Comments

@lobpcg
Copy link
Contributor

lobpcg commented Sep 14, 2018

Description

The code https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/decomposition/pca.py currently supports only

svd_solver : string {'auto', 'full', 'arpack', 'randomized'}

while LOBPCG https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lobpcg.html is already used in http://scikit-learn.org/stable/modules/generated/sklearn.manifold.SpectralEmbedding.html and http://scikit-learn.org/stable/modules/generated/sklearn.cluster.spectral_clustering.html

Steps/Code to Reproduce

N/A

Expected Results

LOBPCG is expected to outperform all currently available SVD solvers for PCA for large problems, e.g., see comments at https://www.mathworks.com/matlabcentral/fileexchange/48-lobpcg-m

Actual Results

Not supported

Versions

All
@aishgrt1
Copy link
Contributor

aishgrt1 commented Sep 27, 2018 •
edited
Loading

@jnothman @amueller Shall I take this?

@lobpcg
Copy link
Contributor Author

lobpcg commented Oct 3, 2018 •
edited
Loading

@jnothman @amueller @aishgrt1 Please see PR #12319 Comments and help are welcome and needed!

scipy/scipy#9352 will address the issue that
https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lobpcg.html itself needs editing to properly work in float32 used in the example plot_faces_decomposition, so this example is not included in the PR #12319.

I am unsure if the transpose='auto' option in lobpcg_svd, which is copy/pasted from randomised_svd , actually makes the eigenvalue problem smaller, not larger, but it a common issue with randomised_svd, so it needs to be addressed separately from this lobpcg_svd add-on.

This was referenced Oct 4, 2018
Closed
Closed
Closed
Closed
@lobpcg
Copy link
Contributor Author

lobpcg commented Oct 12, 2018

@aishgrt1 @jnothman @amueller
Our coding for this issues and #12080 has been completed, please see PR #12319 and react.

@jnothman
Copy link
Member

jnothman commented Oct 14, 2018 via email

@lobpcg
Copy link
Contributor Author

lobpcg commented Oct 30, 2021

scipy/scipy#10830 has implemented LOBPCG solver in svds in scipy while this PR being stalled. Which allows using lobpcg svd_solver to PCA and TruncatedSVD simply by calling https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html with solver=‘lobpcg’

@lobpcg lobpcg closed this as completed Oct 30, 2021
@flying-sheep
Copy link

Can this be reopened? sklearn.decomposition.PCA still doesn’t support svd_solver='lobpcg'.

Should be a simple case of updating the validators and passing this on:

scikit-learn/sklearn/decomposition/_pca.py

Line 750 in e91011b

U, S, Vt = svds(X_centered, k=n_components, tol=self.tol, v0=v0)

@flying-sheep flying-sheep mentioned this issue Oct 15, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@jnothman @flying-sheep @aishgrt1 @lobpcg