Module Ridge

operalib.ridge implements Operator-Valued Kernel ridge regression.

class operalib.ridge.Ridge(kernel='DGauss', lbda=1e-05, A=None, gamma=1.0, theta=0.7, period='autocorr', autocorr_params=None, solver=<function fmin_l_bfgs_b>, solver_params=None, kernel_params=None)

Operator-Valued kernel ridge regression.

Operator-Valued kernel ridge regression (OVKRR) combines ridge regression (linear least squares with l2-norm regularization) with the (OV)kernel trick. It thus learns a linear function in the space induced by the respective kernel and the data. For non-linear kernels, this corresponds to a non-linear function in the original space.

The form of the model learned by OVKRR is identical to support vector regression (SVR). However, different loss functions are used: OVKRR uses squared error loss while support vector regression uses epsilon-insensitive loss, both combined with l2 regularization. In contrast to SVR, fitting a OVKRR model can be done in closed-form and is typically faster for medium-sized datasets. On the other hand, the learned model is non-sparse and thus slower than SVR, which learns a sparse model for epsilon > 0, at prediction-time.

See also

sklearn.Ridge
Linear ridge regression.
sklearn.KernelRidge
Kernel ridge regression.
sklearn.SVR
Support Vector Regression implemented using libsvm.

References

  • Kevin P. Murphy “Machine Learning: A Probabilistic Perspective”, The MIT Press chapter 14.4.3, pp. 492-493

Examples

>>> import operalib as ovk
>>> import numpy as np
>>> n_samples, n_features, n_targets = 10, 5, 5
>>> rng = np.random.RandomState(0)
>>> y = rng.randn(n_samples, n_targets)
>>> X = rng.randn(n_samples, n_features)
>>> clf = ovk.Ridge('DGauss', lbda=1.0)
>>> clf.fit(X, y)

Attributes

dual_coef_ (array, shape = [n_features] or [n_targets, n_features]) Weight vector(s) in kernel space
self.linop_ (callable) Callable which associate to the training points X the Gram matrix (the Gram matrix being a LinearOperator)
A_ (array, shape = [n_targets, n_targets]) Set when Linear operator used by the decomposable kernel is default or None.
period_ (float) Set when period used by the First periodic kernel is ‘autocorr’.
solver_res_ (any) Raw results returned by the solver.

Methods

fit(X, y) Fit OVK ridge regression model.
get_params([deep]) Get parameters for this estimator.
predict(X) Predict using the OVK ridge model.
score(X, y[, sample_weight]) Returns the coefficient of determination R^2 of the prediction.
set_params(**params) Set the parameters of this estimator.
__init__(kernel='DGauss', lbda=1e-05, A=None, gamma=1.0, theta=0.7, period='autocorr', autocorr_params=None, solver=<function fmin_l_bfgs_b>, solver_params=None, kernel_params=None)

Initialize OVK ridge regression model.

Parameters:

kernel : {string, callable}, default=’DGauss’

Kernel mapping used internally. A callable should accept two arguments and the keyword arguments passed to this object as kernel_params, and should return a LinearOperator.

lbda : {float}, default=1e-5

Small positive values of lbda improve the conditioning of the problem and reduce the variance of the estimates. Lbda corresponds to (2*C)^-1 in other linear models such as LogisticRegression or LinearSVC.

A : {LinearOperator, array-like, sparse matrix}, default=None

Linear operator used by the decomposable kernel. If default is None, wich is set to identity matrix of size y.shape[1] when fitting.

gamma : {float}, default=1.

Sigma parameter for the Decomposable Gaussian kernel. Ignored by other kernels.

theta : {float}, default=.7

Theta parameter for the Decomposable First Periodic kernel. Ignored by other kernels.

period : {float}, default=default_period

Period parameter for the First periodic kernel. If optional modules have been imported then default_period is 2 * pi. Otherwise it uses autocorrelation methods to determine the period.

solver : {callable}, default=scipy.optimize.fmin_l_bfgs_b

Solver able to find the minimum of the ridge problem. scipy.optimize.fmin_l_bfgs_b(*solver_params)[0] must return the optimal solution.

autocorr_params : {mapping of string to any}

Additional parameters (keyword arguments) for the period detection for periodic kernels. If None, parameter choice is left to the period detection method.

solver_params : {mapping of string to any}, optional

Additional parameters (keyword arguments) for solver function passed as callable object.

kernel_params : {mapping of string to any}, optional

Additional parameters (keyword arguments) for kernel function passed as callable object.

fit(X, y)

Fit OVK ridge regression model.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Training data.

y : {array-like}, shape = [n_samples] or [n_samples, n_targets]

Target values.

Returns:

self : returns an instance of self.

predict(X)

Predict using the OVK ridge model.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Samples.

Returns:

C : {array}, shape = [n_samples] or [n_samples, n_targets]

Returns predicted values.