Skip to content

Add non-symmetric eigenvalue solvers, eig / eivals, to the linalg extension #935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ev-br opened this issue Apr 29, 2025 · 0 comments
Open
Labels
RFC Request for comments. Feature requests and proposed changes.

Comments

@ev-br
Copy link
Member

ev-br commented Apr 29, 2025

Currently the linalg extension only contains symmetric/hermitian eigenvalue routines, eigh and eigvalsh. The general/non-symmetric versions are missing.
IIUC one reason for the omission is that the low-level routines were not universally available on CUDA or were of allegedly questionable quality [1].
Now that the needed LAPACK-like routine was added to cuSolver 12.6 update 2 [2], the low-level blocker is no more.

Availability:

  • numpy

np.linalg.eig, np.linalg.eigvals are available: https://numpy.org/doc/2.1/reference/generated/numpy.linalg.eig.html

  • torch

torch.linalg.eig, torch.linalg.eigvals are available: https://pytorch.org/docs/stable/generated/torch.linalg.eig.html

  • jax

jax.numpy.linalg.eig, jax.numpy.linalg.eigvals are available: https://docs.jax.dev/en/latest/_autosummary/jax.numpy.linalg.eig.html

Under the hood, it uses the same MAGMA kernels as pytorch, so [1] applies. However, jax-ml/jax#27265 tracks a potential update to use the new routines from [2].

  • cupy

Not available in CuPy 13.4 yet; There is a pull request to add it: cupy/cupy#8980, hopefully to CuPy 14.x.

Proposed API

The API should closely follow eigh and eigvalsh:

  • eigvals(x, /) should return an array of eigenvalues in an unspecified order;

  • eig(x, /) should return a namedtuple (eigenvalues, right eigenvectors). The order of eigenvalues and eigenvectors is unspecified, and the only requirement is that eigenvalues[i] corresponds to eigenvectors[:, i].

Whether the eigenvectors have a unit norm is unspecified and thus implementation-defined.

An option to return left eigenvectors may be added in the future if the array libraries add this option (currently, neither numpy nor torch provide this option).

[1] cupy/cupy#6359 (comment)
[2] https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

@ev-br ev-br added the RFC Request for comments. Feature requests and proposed changes. label Apr 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
RFC Request for comments. Feature requests and proposed changes.
Projects
None yet
Development

No branches or pull requests

1 participant