[Feature Request] Regularized linear regression model with a weight prior #31160
cod3licious
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I have a use case where I want to train a linear regression model (could be any of ridge, lasso, elastic net) but instead of regularizing the weights such that the coefficients tend to zero, I would like to penalize coefficients for deviating from prior weights that I've passed when initializing the model.
From theoretical knowledge and similar datasets, I already know roughly what the weights should be (e.g., which variables should be positively and which negatively correlated), so I would like the model to use these weights unless the given dataset strongly suggests that a coefficient should be adapted.
Basically, instead of doing
$$\min_w |w|^2 = |w-0|^2$$
for the regularization, I would like to do
$$\min_w |w-w_{prior}|^2$$
To encourage the weights to stay close to the$w_{prior}$ that I have given.
Beta Was this translation helpful? Give feedback.
All reactions