Skip to content

Update guidance for linalg rtol tolerance cutoff to accommodate non-SVD-like algorithms #304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 4, 2021

Conversation

kgryte
Copy link
Contributor

@kgryte kgryte commented Nov 4, 2021

This PR

  • resolves Definition of rtol in matrix_rank, lstsq and pinv #216 by updating the guidance for rtol to accommodate non-SVD-like algorithms. Namely, instead of a strict cutoff of less than or equal to, this PR updates the guidance to say "approximately less than or equal to" to allow conforming implementations some wiggle room should they choose a non-SVD-like algorithm for implementing pinv. For matrix_rank, the API is already defined in terms of singular values, so, in practice, an implementation is likely to be SVD-based; in which case, the more strict guidance is likely to hold.
  • in general, the previous guidance should hold true, given the prevalent usage of LAPACK; however, array libraries should be able to use alternative algorithms in accordance with the design principle to be "implementation agnostic". A future refinement could be to define the tolerance in terms of condition numbers, but that is left to future consideration.

@kgryte kgryte added Maintenance Bug fix, typo fix, or general maintenance. topic: Linear Algebra Linear algebra. labels Nov 4, 2021
@kgryte kgryte added this to the v2021 milestone Nov 4, 2021
@kgryte
Copy link
Contributor Author

kgryte commented Nov 4, 2021

Thanks @leofang!

@kgryte kgryte merged commit 52905bf into main Nov 4, 2021
@kgryte kgryte deleted the rtol-approx branch November 4, 2021 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Bug fix, typo fix, or general maintenance. topic: Linear Algebra Linear algebra.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Definition of rtol in matrix_rank, lstsq and pinv
2 participants