plqcom¶

Submodules¶

Classes¶

PLQLoss

PLQLoss is a class represents a continuous convex piecewise quandratic loss function, which adopts three types of

Functions¶

`max_to_plq`(quad_coef)
`is_continuous`(plq_loss)	Check whether a PLQ loss function is continuous
`is_convex`(plq_loss)	Check whether a PLQ loss function is convex
`check_cutoff`(plq_loss)	Check whether there exists a cutoff between the knots, if so, add the cutoff to the knot list and update
`find_min`(plq_loss)	Find the minimum knots and value of the PLQ function, if the minimum value is greater than zero
`plq_to_rehloss`(plq_loss)	convert the PLQLoss function to a ReHLoss function piece by piece.
`affine_transformation`(rehloss[, n, c, p, q, form, y])	Since composite ReLU-ReHU function is closure under affine transformation,

Package Contents¶

class plqcom.PLQLoss(quad_coef=None, form='plq', cutpoints=np.empty(shape=(0,)), points=np.empty(shape=(0, 2)))¶

Bases: object

PLQLoss is a class represents a continuous convex piecewise quandratic loss function, which adopts three types of input forms: ‘plq’, ‘max’ and ‘points’.

Parameters:

quad_coef{dict-like} of {‘a’: [], ‘b’: [], ‘c’: []}

The quandratic coefficients in pieces of the PLQLoss. The i-th piece Q is: a[i]* x**2 + b[i] * x + c[i]

formstr, optional, default: ‘plq’

The form of the input PLQ function.

‘plq’ for the PLQ form: In this form, cutpoints must be given explicitly.
‘max’ for the max form: The max form is a special form of the PLQ function, which is the pointwise maximum of several linear or quadratic functions. The cutpoints are not necessary in this form, since they will be automatically calculated.
‘points’ for the piecewise linear form based on given points: The piecewise linear form is a special form of the PLQ function, which is the piecewise linear function. The function will connect the given points to form a piecewise linear loss. For the first piece and the last piece related to infinity, they will be the same as their adjacent piece.

cutpoints{array-like} of float, optional, default: None

cutpoints of the PLQLoss, except -np.inf and np.inf

if the form is ‘max’ or ‘points’, the cutpoints is not necessary

if the form is ‘plq’, the cutpoints is necessary

points{array-like} of (x,y) pairs [(x1, y1), (x2, y2), … (xn, yn)]

or {dict-like} of {‘x’: [x1, x2, …, xn], ‘y’: [y1, y2, … yn]} or {2d-array-like} of [[x1, x2, …, xn], [y1, y2, … yn]] optional, default: None

Points coordinates of the piecewise linear form of the PLQLoss. The PLQLoss will be constructed by straight lines between each two adjcent points according to their x coordinates. Two points with the same x coordinates will be rejected.

if the form is ‘points’, the points is necessary

if the form is ‘max’ or ‘plq’, the points is not necessary

Examples

>>> import numpy as np
>>> from plqcom import PLQLoss
>>> cutpoints = [0., 1.]
>>> quad_coef = {'a': np.array([0., .5, 0.]), 'b': np.array([-1, 0., 1]), 'c': np.array([0., 0., -.5])}
>>> random_loss = PLQLoss(quad_coef, cutpoints=cutpoints)
>>> x = np.arange(-2,2,.05)
>>> random_loss(x)

cutpoints¶

min_val¶

min_knot¶

__call__(x)¶

Evaluation of PLQLoss function.

Parameters:

x{array-like} of shape {n_samples}
Training vector, where `n_samples` is the number of samples.

Returns:

y{array-like} of shape {n_samples}: The values of the PLQLoss function on each x y[j] = quad_coef[‘a’][i]*x[j]**2 + quad_coef[‘b’][i]*x[j] + quad_coef[‘c’][i], if cutpoints[i] < x[j] < cutpoints[i+1]

plqcom.max_to_plq(quad_coef)¶

plqcom.is_continuous(plq_loss)¶

Check whether a PLQ loss function is continuous

Parameters:

plq_lossPLQLoss: A PLQLoss object

Returns:

bool: Whether the PLQ function is continuous, True for continuous, False for not continuous

Examples

>>> from plqcom import PLQLoss, is_continuous
>>> plq_loss = PLQLoss(cutpoints=np.array([0.]),quad_coef={'a': np.array([0, 0]), 'b': np.array([-1, 1]), 'c': np.array([1, 1])})
>>> is_continuous(plq_loss)
True

plqcom.is_convex(plq_loss)¶

Check whether a PLQ loss function is convex

Parameters:

plq_lossPLQLoss: A PLQLoss object

Returns:

bool: Whether the PLQ function is convex, True for convex, False for not convex

Examples

>>> from plqcom import PLQLoss, is_convex
>>> plq_loss = PLQLoss(cutpoints=np.array([0.]),quad_coef={'a': np.array([0, 0]), 'b': np.array([-1, 1]), 'c': np.array([1, 1])})
>>> is_convex(plq_loss)
True

plqcom.check_cutoff(plq_loss)¶

Check whether there exists a cutoff between the knots, if so, add the cutoff to the knot list and update the coefficients

Parameters:

plq_lossPLQLoss: A PLQLoss object

Examples

>>> from plqcom import PLQLoss, check_cutoff
>>> plq_loss = PLQLoss(cutpoints=np.array([0.]),quad_coef={'a': np.array([0, 0]), 'b': np.array([-1, 1]), 'c': np.array([1, 1])})
>>> check_cutoff(plq_loss)
>>> plq_loss.cutpoints
array([-inf,   0.,  inf])

plqcom.find_min(plq_loss)¶

Find the minimum knots and value of the PLQ function, if the minimum value is greater than zero record the minimum value and knot, remove the minimum value from the PLQ function and update the coefficients

Parameters:

plq_lossPLQLoss: A PLQLoss object

Examples

>>> from plqcom import PLQLoss, find_min
>>> plq_loss = PLQLoss(cutpoints=np.array([0]),quad_coef={'a': np.array([0, 0]), 'b': np.array([-1, 1]), 'c': np.array([1, 1])})
>>> find_min(plq_loss)
>>> plq_loss.min_val
1.0

plqcom.plq_to_rehloss(plq_loss)¶

convert the PLQLoss function to a ReHLoss function piece by piece.: The details are described in the technical details.

Parameters:

plq_lossPLQLoss: A PLQLoss object

Returns:

an object of ReHLoss

Examples

>>> from plqcom import PLQLoss, plq_to_rehloss
>>> plq_loss = PLQLoss(cutpoints=np.array([0]),quad_coef={'a': np.array([0, 0]), 'b': np.array([-1, 1]), 'c': np.array([1, 1])})
>>> reh_loss = plq_to_rehloss(plq_loss)

plqcom.affine_transformation(rehloss: rehline._loss.ReHLoss, n=1, c=1, p=1, q=0, form='custom', y=1)¶

Since composite ReLU-ReHU function is closure under affine transformation, this function perform affine transformation on the PLQ object.

Parameters:

rehlossReHLoss

A ReHLoss object

c: a number or {array_like} of shape (n_samples,), default=1

scale parameter on loss function and require c > 0

p: a number or {array_like} of shape (n_samples,),default=1

scale parameter on z

q: a number or {array_like} of shape (n_samples,),default=0

shift parameter on z

n: int, default=1

number of samples

form: str, default=’custom’

the form of affine transformation ‘custom’ for custom form

In this form, the c, p, q can be either a number or an array

‘classification’ for classification form: In this form, $L_i = c_iL(y_i z_i)$, i.e. p=y_i, q=0
‘regression’ for regression form: In this form, $L_i = c_iL(y_i - z_i)$, i.e. p=-1, q=y should be very careful when specify the original L and parameters

y: {array_like} of shape (n_samples,), default=None, only required when form is ‘classification’ or ‘regression’

the label of the samples

Returns:

ReHLoss: A ReHLoss object after affine transformation

Examples

>>> from plqcom import PLQLoss, affine_transformation, plq_to_rehloss
>>> import numpy as np
>>> from rehline import ReHLine
>>> n, d, C = 1000, 3, 0.5
>>> np.random.seed(1024)
>>> X = np.random.randn(1000, 3)
>>> beta0 = np.random.randn(3)
>>> y = np.sign(X.dot(beta0) + np.random.randn(n))
>>> plqloss = PLQLoss(quad_coef={'a': np.array([0., 0.]), 'b': np.array([0., 1.]), 'c': np.array([0., 0.])}, cutpoints=np.array([0]))
>>> rehloss = plq_to_rehloss(plqloss)
>>> rehloss = affine_transformation(rehloss, n=X.shape[0], c=C, p=-y, q=1)