Skip to content

RidgeCV and Ridge produce different results when fitted with sample_weight #5364

Closed
@mtrbean

Description

@mtrbean
import numpy as np
from sklearn.linear_model import RidgeCV, Ridge
from sklearn.datasets import load_boston
from sklearn.preprocessing import scale

boston = scale(load_boston().data)
target = load_boston().target
alphas = np.linspace(0.1, 200)
weight = np.logspace(0, -2, len(target))

print "RidgeCV(eigen)"
fit0 = RidgeCV(alphas=alphas, store_cv_values=True, gcv_mode='eigen').fit(boston, target, sample_weight=weight)    
print "alpha:", fit0.alpha_
print "cv:", fit0.cv_values_[0,0:5]
print "coef:", fit0.coef_[:5]

print "RidgeCV(svd)"
fit1 = RidgeCV(alphas=alphas, store_cv_values=True, gcv_mode='svd').fit(boston, target, sample_weight=weight)    
print "alpha:", fit1.alpha_
print "cv:", fit1.cv_values_[0,0:5]
print "coef:", fit1.coef_[:5]

print "Ridge"
fit2 = Ridge(fit0.alpha_).fit(boston, target, sample_weight=weight)
print "coef:", fit2.coef_[:5]

gives:

RidgeCV(eigen)
alpha: 167.363265306
cv: [ 1.63311277  1.58236816  1.53267447  1.48401844  1.43638695]
coef: [-0.90546722  1.0438252   0.09577594  0.68459321 -2.00971137]

RidgeCV(svd)
alpha: 167.363265306
cv: [ 1.63311277  1.58236816  1.53267447  1.48401844  1.43638695]
coef: [-0.90546722  1.0438252   0.09577594  0.68459321 -2.00971137]

Ridge
coef: [-0.11587241  0.37139676 -0.35468307  0.26534304 -0.29640185]

If sample_weight is None, all three models give the same coefficients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions