Skip to content

Allow GeneralizedLinearRegressor to use parameter level regularization parameter instead of a scalar for all parameters #22350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xiaowei1234 opened this issue Jan 31, 2022 · 1 comment
Labels
Needs Triage Issue requires triage New Feature

Comments

@xiaowei1234
Copy link

Describe the workflow you want to enable

Currently the GeneralizedLinearRegressor module only allows a scalar alpha regularization parameter to be used for all parameters. However, sometimes the modeler would like to input different amounts of regularization on certain parameters, either for interpretation or some other prior knowledge.

Describe your proposed solution

The alpha parameter can take either a scalar value or an iterable of the same length as the number of parameters in the input matrix.

The optimization algorithm will then flow as normal with the line below in glm.py naturally doing a vector to vector dot product instead of a scalar vector multiplication.

coef_scaled = alpha * coef[offset:]

I believe the alpha parameter passed to scipy.optimize.minimize in the args parameter isn't being used anywhere.

For QA, I plan to output regression results in R or statsmodels for the same data and model specs

Describe alternatives you've considered, if relevant

No response

Additional context

I modified this as part of my dayjob and also doing this as part of my programming work at my university. This is my first PR so please let me know if I'm missing anything.

@xiaowei1234 xiaowei1234 added Needs Triage Issue requires triage New Feature labels Jan 31, 2022
@lorentzenchr
Copy link
Member

@xiaowei1234 Thanks for opening this issue. It is a duplicate of #11566, so I'm closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Triage Issue requires triage New Feature
Projects
None yet
Development

No branches or pull requests

2 participants