Gaussian processes

GaussianProcessRegressor

class DLL.MachineLearning.SupervisedLearning.GaussianProcesses.GaussianProcessRegressor(covariance_function, noise=0, epsilon=1e-05, device=device(type='cpu'))[source]

Bases: object

Implements the Gaussian process regression model.

Parameters:

  • covariance_function (Kernels, optional) – The kernel function expressing how similar are different samples.

  • noise (int | float, optional) – The artificially added noise to the model. Is added as variance to each sample. Must be non-negative. Defaults to 0.

  • epsilon (float, optional) – Implemented similarly to noise. Makes sure the covariance matrix is positive definite and hence invertible. Must be positive. Defaults to 1e-5. If one gets a RunTimeError for a matrix not being invertible, one should increase this parameter.

  • device (torch.device, optional) – The device of all matrices. Defaults to torch.device(“cpu”).

n_features

The number of features. Available after fitting.

Type:

int

fit(X, y)[source]

Fits the GaussianProcessRegressor model to the input data.

Parameters:

  • X (torch.Tensor of shape (n_samples, n_features)) – The input data, where each row is a sample and each column is a feature.

  • y (torch.Tensor of shape (n_samples,)) – The target values corresponding to each sample. Should be normalized to zero mean and one variance.

Returns:

None

Raises:

  • TypeError – If the input matrix or the target matrix is not a PyTorch tensor.

  • ValueError – If the input matrix or the target matrix is not the correct shape.

log_marginal_likelihood()[source]

Computes the log marginal likelihood of the current model. This value is used to optimize hyperparameters.

Returns:

The log marginal likelihood of the current model.

Return type:

log marginal likelihood (float)

predict(X)[source]

Applies the fitted GaussianProcessRegressor model to the input data, predicting the correct values.

Parameters:

X (torch.Tensor of shape (n_samples, n_features)) – The input data to be regressed.

Returns:

A tuple containing the posterior mean and posterior covariance. As the prediction, one should use the mean.

Return type:

mean, covariance (tuple[torch.Tensor of shape (n_samples,), torch.Tensor of shape (n_samples, n_samples))

Raises:

  • NotFittedError – If the GaussianProcessRegressor model has not been fitted before predicting.

  • TypeError – If the input matrix is not a PyTorch tensor.

  • ValueError – If the input matrix is not the correct shape.

train_kernel(epochs=10, optimiser=None, callback_frequency=1, verbose=False)[source]

Trains the current covariance function parameters by maximizing the log marginal likelihood.

Parameters:

  • epochs (int, optional) – The number of optimisation rounds. Must be a positive integer. Defaults to 10.

  • optimiser (Optimisers | None, optional) – The optimiser used for training the model. If None, the Adam optimiser is used.

  • callback_frequency (int, optional) – The number of iterations between printing info from training. Must be a positive integer. Defaults to 1, which means that every iteration, info is printed assuming verbose=True.

  • verbose (bool, optional) – If True, prints the log marginal likelihood of the model during training. Defaults to False.

Returns:

A dictionary tracking the evolution of the log marginal likelihood at intervals defined by callback_frequency. The tensor can be accessed with history[“log marginal likelihood”].

Return type:

history (dict[str, torch.Tensor], the tensor is floor(epochs / callback_frequency) long.)

Raises:

  • NotFittedError – If the GaussianProcessRegressor model has not been fitted before training the kernel.

  • TypeError – If the parameters are of wrong type.

  • ValueError – If epochs is not a positive integer.

GaussianProcessClassifier

class DLL.MachineLearning.SupervisedLearning.GaussianProcesses.GaussianProcessClassifier(covariance_function, noise=0, n_iter_laplace_mode=100, epsilon=1e-05, device=device(type='cpu'))[source]

Bases: object

Implements the Gaussian process classification model for binary classes. This model can be extended to multiclass classification by OvO or OvR. This implementation is adapted from the sklearn implementation and is based chapters 3 and 5 of on this book.

Parameters:

  • covariance_function (Kernels, optional) – The kernel function expressing how similar are different samples.

  • noise (int | float, optional) – The artificially added noise to the model. Is added as variance to each sample. Must be non-negative. Defaults to 0.

  • n_iter_laplace_mode (int, optional) – The max amount of Newton iterations used to find the mode of the Laplace approximation. Must be a positive integer. Defaults to 100.

  • epsilon (float, optional) – Implemented similarly to noise. Makes sure the covariance matrix is positive definite and hence invertible. Must be positive. Defaults to 1e-5. If one gets a RunTimeError for a matrix not being invertible, one should increase this parameter.

  • device (torch.device, optional) – The device of all matrices. Defaults to torch.device(“cpu”).

n_features

The number of features. Available after fitting.

Type:

int

fit(X, y)[source]

Fits the GaussianProcessClassifier model to the input data.

Parameters:

  • X (torch.Tensor of shape (n_samples, n_features)) – The input data, where each row is a sample and each column is a feature.

  • y (torch.Tensor of shape (n_samples,)) – The target values corresponding to each sample. Should be normalized to zero mean and one variance. Every element must be in [0, 1].

Returns:

None

Raises:

  • TypeError – If the input matrix or the target matrix is not a PyTorch tensor.

  • ValueError – If the input matrix or the target matrix is not the correct shape.

log_marginal_likelihood()[source]

Computes the log marginal likelihood of the current model. This value is used to optimize hyperparameters.

Returns:

The log marginal likelihood of the current model.

Return type:

log marginal likelihood (float)

predict(X)[source]

Applies the fitted GaussianProcessClassifier model to the input data, predicting the correct values.

Parameters:

X (torch.Tensor of shape (n_samples, n_features)) – The input data to be regressed.

Returns:

A tuple containing the posterior mean and posterior covariance. As the prediction, one should use the mean.

Return type:

mean, covariance (tuple[torch.Tensor of shape (n_samples,), torch.Tensor of shape (n_samples, n_samples))

Raises:

  • NotFittedError – If the GaussianProcessClassifier model has not been fitted before predicting.

  • TypeError – If the input matrix is not a PyTorch tensor.

  • ValueError – If the input matrix is not the correct shape.

predict_proba(X)[source]

Applies the fitted GaussianProcessClassifier model to the input data, predicting the probabilities of each class.

Parameters:

X (torch.Tensor of shape (n_samples, n_features)) – The input data to be regressed.

Returns:

The probabilities that the class belongs to class 1.

Return type:

probabilities (tuple[torch.Tensor of shape (n_samples,))

Raises:

  • NotFittedError – If the GaussianProcessClassifier model has not been fitted before predicting.

  • TypeError – If the input matrix is not a PyTorch tensor.

  • ValueError – If the input matrix is not the correct shape.

train_kernel(epochs=10, optimiser=None, callback_frequency=1, verbose=False)[source]

Trains the current covariance function parameters by maximizing the log marginal likelihood.

Parameters:

  • epochs (int, optional) – The number of optimisation rounds. Must be a positive integer. Defaults to 10.

  • optimiser (Optimisers | None, optional) – The optimiser used for training the model. If None, the Adam optimiser is used.

  • callback_frequency (int, optional) – The number of iterations between printing info from training. Must be a positive integer. Defaults to 1, which means that every iteration, info is printed assuming verbose=True.

  • verbose (bool, optional) – If True, prints the log marginal likelihood of the model during training. Defaults to False.

Returns:

A dictionary tracking the evolution of the log marginal likelihood at intervals defined by callback_frequency. The tensor can be accessed with history[“log marginal likelihood”].

Return type:

history (dict[str, torch.Tensor], the tensor is floor(epochs / callback_frequency) long.)

Raises:

  • NotFittedError – If the GaussianProcessClassifier model has not been fitted before training the kernel.

  • TypeError – If the parameters are of wrong type.

  • ValueError – If epochs is not a positive integer.