Kernels

Available kernels

class DLL.MachineLearning.SupervisedLearning.Kernels.RBF(sigma=1, correlation_length=1, train_sigma=True, train_correlation_length=True)[source]

The commonly used radial basis function (rbf) kernel. Yields high values for samples close to one another. The used equation is:

\[k(x_i, x_j) = \sigma^2 \exp\left(-\frac{(x_i - x_j)^2}{2 l^2}\right),\]

where \(d\) is the the Euclidian metric and \(\sigma\) and \(l\) are the sigma and the correlation_length parameters respectively.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
correlation_length (float | torch.Tensor, optional) – The length scale of the kernel. Determines how quickly the similarity decays as points become further apart. Must be a positive real number or a torch.Tensor of shape (n_features,). Defaults to 1.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.
train_correlation_length (bool, optional) – Determines, wheter or not the correlation_length parameter should be changed during training the kernel. Defaults to True.

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “rbf_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

class DLL.MachineLearning.SupervisedLearning.Kernels.Linear(sigma=1, sigma_bias=0, train_sigma=True, train_sigma_bias=True)[source]

The linear kernel, often used as a baseline in kernel-based learning methods, representing a linear relationship between inputs. The used equation is:

\[k(x_i, x_j) = \sigma^2 x_ix_j+\sigma_{bias},\]

where \(\sigma\) and \(\sigma_{bias}\) are the sigma and the sigma_bias parameters respectively.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
sigma_bias (float, optional) – The constant term of the kernel, sometimes called the bias or intercept. It allows the kernel function to handle non-zero means. Must be a real number. Defaults to 0.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.
train_sigma_bias (bool, optional) – Determines, wheter or not the sigma_bias parameter should be changed during training the kernel. Defaults to True.

Example

The commonly used polynomial kernel can be used as follows:

from DLL.MachineLearning.SupervisedLearning.Kernels import Linear

linear_kernel = Linear()
polynomial_kernel_degree_d = linear_kernel ** d

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “linear_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

class DLL.MachineLearning.SupervisedLearning.Kernels.Periodic(sigma=1, correlation_length=1, period=1, train_sigma=True, train_correlation_length=True, train_period=True)[source]

The periodic kernel, commonly used to capture periodic relationships in data, such as seasonal patterns or repeating cycles. The used equation is:

\[k(x_i, x_j) = \sigma^2 \exp\left(-\frac{2\sin^2(\frac{\pi d(x_i, x_j)}{p})}{l^2}\right),\]

where \(d\) is the Euclidian metric and \(\sigma\), \(l\) and \(p\) are the sigma, the correlation_length and the period parameters respectively.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
correlation_length (float, optional) – Controls how quickly the similarity decays as points move further apart in the input space. Must be a positive real number. Defaults to 1.
period (float, optional) – The period of the kernel, indicating the distance over which the function repeats. Must be a positive real number. Defaults to 1.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.
train_correlation_length (bool, optional) – Determines, wheter or not the correlation_length parameter should be changed during training the kernel. Defaults to True.
train_period (bool, optional) – Determines, wheter or not the period parameter should be changed during training the kernel. Defaults to True.

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “periodic_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

class DLL.MachineLearning.SupervisedLearning.Kernels.WhiteGaussian(sigma=1, train_sigma=True)[source]

The white Gaussian kernel, commonly used to capture Gaussian noise in data. This kernel models purely random noise without dependencies on input values. The used equation is:

\[k(x_i, x_j) = \sigma^2 \mathbb{1}\{x_i = x_j\},\]

where \(\mathbb{1}\) is the indicator function and \(\sigma\) is the sigma parameter.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “white_gaussian_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

class DLL.MachineLearning.SupervisedLearning.Kernels.RationalQuadratic(sigma=1, correlation_length=1, alpha=1, train_sigma=True, train_correlation_length=True, train_alpha=True)[source]

The rational quadratic kernel, a versatile kernel often used in Gaussian Processes for modeling data with varying degrees of smoothness. It can be seen as a scale mixture of the squared exponential kernel, allowing flexibility between linear and non-linear relationships. The used equation is:

\[k(x_i, x_j) = \sigma^2 \left(1 + \frac{d(x_i, x_j)^2}{2\alpha l^2} \right)^{-\alpha},\]

where \(d\) is the Euclidian metric and \(\sigma\), \(l\) and \(\alpha\) are the sigma, correlation_length and alpha parameters respectively.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
correlation_length (float, optional) – Controls how quickly the similarity decays as points move further apart in the input space. Must be a positive real number. Defaults to 1.
alpha (float, optional) – Controls the relative weighting of large-scale and small-scale variations. Higher values make the kernel behave more like a squared exponential (Gaussian) kernel, while lower values allow for more flexibility. Must be a positive real number. Defaults to 1.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.
train_correlation_length (bool, optional) – Determines, wheter or not the correlation_length parameter should be changed during training the kernel. Defaults to True.
train_alpha (bool, optional) – Determines, wheter or not the alpha parameter should be changed during training the kernel. Defaults to True.

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “rational_quadratic_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

class DLL.MachineLearning.SupervisedLearning.Kernels.Matern(sigma=1, correlation_length=1, nu=1.5, train_sigma=True, train_correlation_length=True)[source]

The Matern kernel, a versatile kernel often used in Gaussian Processes for modeling data with varying degrees of smoothness. Is a generalization of the RBF kernel with varying levels of smoothness controlled by nu. The used equation is:

\[k(x_i, x_j) = \sigma^2 \frac{2^{1-\nu}}{\Gamma(\nu)} \left(\sqrt{2\nu}\frac{d(x_i, x_j)}{l}\right)^\nu K_\nu\left(\sqrt{2\nu}\frac{d(x_i, x_j)}{l}\right),\]

where \(d\) is the Euclidian metric, \(\Gamma\) is the gamma function, \(K_\nu\) is the modified Bessel function of the second kind and \(\sigma\), \(l\) and \(\nu\) are the sigma, correlation_length and nu parameters respectively.

Parameters:

sigma (float, optional) – The overall scale factor of the variance. Controls the amplitude of the kernel. Must be a positive real number. Defaults to 1.
correlation_length (float, optional) – Controls how quickly the similarity decays as points move further apart in the input space. Must be a positive real number. Defaults to 1.
nu (float, optional) – Controls the smoothness of the kernel. Important values of nu include 0.5 for the rbf kernel with the l1 norm modelling non differentiable functions, 1.5 for once differentiable functions and 2.5 for twice differentiable functions. If is set to float(“inf”), the kernel is equivalent to the RBF kernel. It is not possible to train nu. Must be a positive real number. Defaults to 1.5.
train_sigma (bool, optional) – Determines, wheter or not the sigma parameter should be changed during training the kernel. Defaults to True.
train_correlation_length (bool, optional) – Determines, wheter or not the correlation_length parameter should be changed during training the kernel. Defaults to True.

__call__(X1, X2)[source]

Yields the kernel matrix between two vectors.

Parameters:

X1 (torch.Tensor of shape (n_samples_1, n_features))
X2 (torch.Tensor of shape (n_samples_2, n_features))

Returns:

The pairwise kernel values between samples from X1 and X2.

Return type:

kernel_matrix (torch.Tensor of shape (n_samples_1, n_samples_2))

Raises:

TypeError – If the input matricies are not a PyTorch tensors.
ValueError – If the input matricies are not the correct shape.

parameters()[source]

Yields the parameters of the kernel as a dictionary. If one uses a combination of the kernels, the parameters of each of the child kernels are returned.

Returns:: The parameters as a dictionary. The key of the parameter is eg. “matern_sigma_1”.
Return type:: parameters (dict[str, torch.Tensor])

Combining kernels

Kernels can be combined in many ways. These ways are showcased in the example below:

from DLL.MachineLearning.SupervisedLearning.Kernels import Linear, RBF

linear_kernel = Linear()
rbf_kernel = RBF()

d = 2

# new kernels include:
sum_kernel = linear_kernel + rbf_kernel  # sums
product_kernel = linear_kernel * rbf_kernel  # products
exponent_kernel = linear_kernel ** d  # exponents