Example 3:Ridge Regression

This example try to show how to use our PLQ Composite Decomposition tool to perform Ridge Regression.

1. Data Generation

from plqcom import PLQLoss, plq_to_rehloss, affine_transformation
import numpy as np
from rehline import ReHLine

n_samples, n_features = 1000, 5
rng = np.random.RandomState(0)
X = rng.randn(n_samples, n_features)
beta = rng.randn(n_features)
y = np.dot(X, beta) + rng.normal(scale=0.1, size=n_samples)

2.Create and Decompose the PLQ Loss

plqloss = PLQLoss(quad_coef={'a': np.array([1.]), 'b': np.array([0.]), 'c': np.array([0.])},
                  cutpoints=np.array([]))

rehloss = plq_to_rehloss(plqloss)

rehloss.rehu_cut, rehloss.rehu_coef, rehloss.rehu_intercept

(array([[inf],
        [inf]]),
 array([[-1.41421356],
        [ 1.41421356]]),
 array([[-0.],
        [ 0.]]))

3. Broadcast to all samples

rehloss = affine_transformation(rehloss, n=X.shape[0], c=1, p=-1, q=y)

4. Use the ReHLine to solve the problem

clf = ReHLine()
C = 1
clf.Tau, clf.S, clf.T, clf.C = rehloss.rehu_cut, rehloss.rehu_coef, rehloss.rehu_intercept, C
clf.fit(X=X)
print('sol privided by rehline: %s' % clf.coef_)

sol privided by rehline: [ 0.31029299 -0.73855141 -1.53883908 -0.56076169 -1.6047202 ]

Compare with the solution provided by sklearn

from sklearn.linear_model import Ridge
clf1 = Ridge(alpha=0.5)
clf1.fit(X, y)
print('sol privided by sklearn: %s' % clf1.coef_)

sol privided by sklearn: [ 0.31017419 -0.73849146 -1.53893293 -0.56084128 -1.60476756]