Multidimensional Gaussian Process Regression (GPR)

This script demonstrates the use of a Gaussian Process Regressor (GPR) with a Radial Basis Function (RBF) kernel in a multidimensional setting. The example involves training a GPR model on 2D input data and predicting the outputs on a test set.

/home/runner/work/deep-learning-library/deep-learning-library/DLL/MachineLearning/SupervisedLearning/GaussianProcesses/_GaussianProcessRegressor.py:68: UserWarning: y should have zero mean for optimal results. Use DLL.Data.Preprocessing.StandardScaler to scale the data.
  warn("y should have zero mean for optimal results. Use DLL.Data.Preprocessing.StandardScaler to scale the data.")
Epoch: 1 - Log marginal likelihood: -961.1761474609375 - Parameters: {'rbf_sigma_3': 1.01, 'rbf_corr_len_3_1': 9.99, 'rbf_corr_len_3_2': 9.99, 'white_gaussian_sigma_1': 0.99}
Epoch: 101 - Log marginal likelihood: -856.370849609375 - Parameters: {'rbf_sigma_3': 1.921, 'rbf_corr_len_3_1': 8.993, 'rbf_corr_len_3_2': 8.602, 'white_gaussian_sigma_1': 0.736}
Epoch: 201 - Log marginal likelihood: -740.5985107421875 - Parameters: {'rbf_sigma_3': 2.762, 'rbf_corr_len_3_1': 8.265, 'rbf_corr_len_3_2': 6.518, 'white_gaussian_sigma_1': 0.552}
Epoch: 301 - Log marginal likelihood: -207.18463134765625 - Parameters: {'rbf_sigma_3': 4.162, 'rbf_corr_len_3_1': 7.353, 'rbf_corr_len_3_2': 4.429, 'white_gaussian_sigma_1': 0.266}
Epoch: 401 - Log marginal likelihood: -97.6556396484375 - Parameters: {'rbf_sigma_3': 4.679, 'rbf_corr_len_3_1': 5.535, 'rbf_corr_len_3_2': 3.693, 'white_gaussian_sigma_1': 0.24}
Epoch: 501 - Log marginal likelihood: -8.6304931640625 - Parameters: {'rbf_sigma_3': 5.046, 'rbf_corr_len_3_1': 3.62, 'rbf_corr_len_3_2': 3.28, 'white_gaussian_sigma_1': 0.223}
Epoch: 601 - Log marginal likelihood: 15.4263916015625 - Parameters: {'rbf_sigma_3': 5.224, 'rbf_corr_len_3_1': 2.716, 'rbf_corr_len_3_2': 3.001, 'white_gaussian_sigma_1': 0.223}
Epoch: 701 - Log marginal likelihood: 21.6832275390625 - Parameters: {'rbf_sigma_3': 5.322, 'rbf_corr_len_3_1': 2.31, 'rbf_corr_len_3_2': 2.795, 'white_gaussian_sigma_1': 0.223}
Epoch: 801 - Log marginal likelihood: 23.8365478515625 - Parameters: {'rbf_sigma_3': 5.388, 'rbf_corr_len_3_1': 2.122, 'rbf_corr_len_3_2': 2.639, 'white_gaussian_sigma_1': 0.223}
Epoch: 901 - Log marginal likelihood: 24.797119140625 - Parameters: {'rbf_sigma_3': 5.439, 'rbf_corr_len_3_1': 2.039, 'rbf_corr_len_3_2': 2.523, 'white_gaussian_sigma_1': 0.223}

import torch
import matplotlib.pyplot as plt

from DLL.MachineLearning.SupervisedLearning.GaussianProcesses import GaussianProcessRegressor
from DLL.MachineLearning.SupervisedLearning.Kernels import RBF, WhiteGaussian
from DLL.Data.Preprocessing import data_split, StandardScaler
from DLL.DeepLearning.Optimisers import ADAM


n = 30
X, Y = torch.meshgrid(torch.linspace(0, 1, n, dtype=torch.float32), torch.linspace(-1, 1, n, dtype=torch.float32), indexing="xy")
x = torch.stack((X.flatten(), Y.flatten()), dim=1)
y = X.flatten() ** 2 + Y.flatten() ** 2 + 0.1 * torch.randn(size=Y.flatten().size()) - 5
x_train, y_train, _, _, x_test, y_test = data_split(x, y, train_split=0.8, validation_split=0.0)
transformer = StandardScaler()
y_train = transformer.fit_transform(y_train)
y_test = transformer.transform(y_test)


model = GaussianProcessRegressor(RBF(correlation_length=torch.tensor([10.0, 10.0],)) + WhiteGaussian())
model.fit(x_train, y_train)
optimizer = ADAM(0.01)
lml = model.train_kernel(epochs=1000, optimiser=optimizer, callback_frequency=100, verbose=True)["log marginal likelihood"]
mean, _ = model.predict(x_test)
z = transformer.inverse_transform(mean)

fig = plt.figure(figsize=(14, 6))
ax = fig.add_subplot(121, projection='3d')
surf = ax.scatter(x_test[:, 0], x_test[:, 1], z, color="blue", label="prediction")
surf = ax.scatter(x_test[:, 0], x_test[:, 1], transformer.inverse_transform(y_test), color="red", label="true value")
ax.legend()

ax = fig.add_subplot(122)
ax.plot(lml)
ax.grid()
ax.set_xlabel("Epoch")
ax.set_ylabel("Log marginal likelihood")

plt.show()