Skip to content

The covariance matrix is incorrect in BayesianRidge #31093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
antoinebaker opened this issue Mar 27, 2025 · 1 comment · Fixed by #31094
Closed

The covariance matrix is incorrect in BayesianRidge #31093

antoinebaker opened this issue Mar 27, 2025 · 1 comment · Fixed by #31094
Labels

Comments

@antoinebaker
Copy link
Contributor

Describe the bug

The posterior covariance matrix in BayesianRidge, attribute sigma_, is incorrect when n_features > n_samples. This is because the posterior covariance requires the full svd, while the current code uses the reduced svd.

Steps/Code to Reproduce

import numpy as np
from sklearn.linear_model import BayesianRidge
from sklearn import datasets

# on main
X, y = datasets.make_regression(n_samples=10, n_features=20)
n_features = X.shape[1]
reg = BayesianRidge(fit_intercept=False).fit(X, y)
covariance_matrix = np.linalg.inv(
    reg.lambda_ * np.identity(n_features) + reg.alpha_ * np.dot(X.T, X)
)
np.allclose(reg.sigma_, covariance_matrix)

Expected Results

True

Actual Results

False

Versions

1.7.dev0
@antoinebaker antoinebaker added Bug Needs Triage Issue requires triage labels Mar 27, 2025
@glemaitre glemaitre removed the Needs Triage Issue requires triage label Mar 27, 2025
@glemaitre
Copy link
Member

Thanks @antoinebaker. Looks like a bug indeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants