Losses

BCE

class DLL.DeepLearning.Losses.BCE(reduction='mean')

Bases: BaseLoss

The binary cross entropy loss. Used in binary classification. Identical to categorical cross entropy with 2 classes.

Parameters:

reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”. Defaults to “mean”.

gradient(prediction, true_output)

Calculates the gradient of the binary categorical cross entropy.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the binary categorical cross entropy.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the binary categorical cross entropy with the equations:

\[\begin{split}\begin{align*} l_i &= y_i\cdot\text{ln}(f(x_i)) + (1 - y_i)\cdot\text{ln}(1 - f(x_i)),\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor

CCE

class DLL.DeepLearning.Losses.CCE(reduction='mean')

Bases: BaseLoss

The categorical cross entropy loss. Used in multi-class classification.

Parameters:

reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”. Defaults to “mean”.

gradient(prediction, true_output)

Calculates the gradient of the categorical categorical cross entropy.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values as a probability distribution. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A one-hot encoded tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the categorical cross entropy loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the categorical categorical cross entropy with the equations:

\[\begin{split}\begin{align*} l_i &= y_i\cdot\text{ln}(f(x_i)),\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values as a probability distribution. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A one-hot encoded tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor

Exponential

class DLL.DeepLearning.Losses.Exponential(reduction='mean')

Bases: BaseLoss

The exponential loss. Used for binary classification.

Parameters:

reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”. Defaults to “mean”.

gradient(prediction, true_output)

Calculates the gradient of the exponential loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the exponential loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values in range [0, 1]. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values labeled with 0 or 1. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the exponential loss with the equations:

\[\begin{split}\begin{align*} l_i &= y_i\cdot e^{-f(x_i)} + (1 - y_i)\cdot e^{f(x_i)},\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values as a probability distribution. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A one-hot encoded tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor

MSE

class DLL.DeepLearning.Losses.MSE(reduction='mean')

Bases: BaseLoss

The squared error loss. Used for regression.

Parameters:

reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”.

gradient(prediction, true_output)

Calculates the gradient of the squared error loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the squared error loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the squared error loss with the equations:

\[\begin{split}\begin{align*} l_i &= (y_i - f(x_i))^2,\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor

MAE

class DLL.DeepLearning.Losses.MAE(reduction='mean')

Bases: BaseLoss

The absolute error loss. Used for regression.

Parameters:

reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”.

gradient(prediction, true_output)

Calculates the gradient of the absolute error loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the absolute error loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the absolute error loss with the equations:

\[\begin{split}\begin{align*} l_i &= |y_i - f(x_i)|,\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor

Huber

class DLL.DeepLearning.Losses.Huber(delta=1.0, reduction='mean')

Bases: BaseLoss

The huber loss. Used for regression. Is a combination of squared error and absolute error.

Parameters:

• delta (int | float, optional) – The radius around the true value that uses the squared error. If the difference is larger than delta, the absolute error is used. Must be a positive real number. Defaults to 1.

• reduction (str, optional) – The reduction method. Must be one of “mean” or “sum”.

gradient(prediction, true_output)

Calculates the gradient of the huber loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the gradients.

Return type:

torch.Tensor

hessian(prediction, true_output)

Calculates the diagonal of the hessian matrix of the huber loss.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor of the same shape as the inputs containing the diagonal of the hessian matrix.

Return type:

torch.Tensor

loss(prediction, true_output)

Calculates the huber loss with the equations:

\[\begin{split}\begin{align*} l_i &= \begin{cases} \frac{1}{2}(y_i - f(x_i))^2 & \text{if } |y_i - f(x_i)| \leq \delta,\\ \delta|y_i - f(x_i)| - \frac{1}{2}\delta^2 & \text{otherwise}, \end{cases}\\ L_{sum} &= \sum_{i=1}^n l_i \text{ or } L_{mean} = \frac{1}{n}\sum_{i=1}^n l_i, \end{align*}\end{split}\]

where \(f(x_i)\) is the predicted value and \(y_i\) is the true value.

Parameters:

• prediction (torch.Tensor) – A tensor of predicted values. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values. Must be the same shape as the prediction.

Returns:

A tensor containing a single value with the loss.

Return type:

torch.Tensor