Metrics

DLL.Data.Metrics.accuracy(predictions, true_output)

Calculates the accuracy of predictions.

Parameters:

• predictions (torch.Tensor) – A torch tensor of either predicted labels or probabilities. If is given probabilities, calculates the corresponding predictions.

• true_output (torch.Tensor) – A torch tensor of either true labels or one-hot encoded values. If is given a one-hot encoded tensor, calculates the corresponding labels.

Returns:

the accuracy of the predictions.

Return type:

float

DLL.Data.Metrics.adjusted_r2_score(predictions, true_output, n_features)

Calculates the adjusted coefficient of determination of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

• n_features (int) – The number of features in the original data. Must be a positive integer.

Returns:

The adjusted coefficient of determination of the predictions.

Return type:

float

DLL.Data.Metrics.auc(fpr, tpr)

Calculates area under the roc curve using the trapezoidal rule. The problem must be binary classification.

Parameters:

• fpr (torch.Tensor of shape (n_thresholds,)) – A torch tensor containing the false positive rates.

• tpr (torch.Tensor of shape (n_thresholds,)) – A torch tensor containing the true positive rates.

Returns:

the area under the roc curve.

Return type:

float

DLL.Data.Metrics.binary_cross_entropy(predictions, true_output)

Calculates the binary cross entropy of the predictions. The problem must be binary classification.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted probabilities as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true labels as a vector. Must be the same shape as the predictions.

Returns:

The binary cross entropy of the predictions.

Return type:

float

DLL.Data.Metrics.calculate_metrics(data, metrics, loss=None, validation=False)

Calculates the values of different metrics based on training predictions and true values.

Parameters:

• data (tuple[torch.Tensor, torch.Tensor]) – A tuple of predictions and the true outputs of a model.

• metrics (tuple | list) – A list metric names to calculate. Each element must be in [“loss”, “accuracy”, “precision”, “recall”, “f1_score”, “rmse”, “mae”, “mse”, “bce”, “cce”, “huber”, “median_absolute”].

• loss (Callable[[predictions, true values], float], optional) – The wanted loss function. If Defaults to None.

• validation (bool, optional) – Determines if “val_” is appended before each metric. If true, the each element of the metrics must be for instance “val_loss” or “val_mse”. Defaults to False.

Returns:

A dictionary with metric name as the key and the metric as the value.

Return type:

dict[str, float]

DLL.Data.Metrics.calibration_curve(y_true, y_prob, n_bins=5, strategy='quantile')

Computes a calibration curve.

Parameters:

• y_true (torch.Tensor of shape (n_samples,)) – True binary labels.

• y_prob (torch.Tensor of shape (n_samples,)) – Predicted probabilities.

• n_bins (int, optional) – Number of bins to discretize predictions. Must be a positive integer. Defaults to 5.

• strategy (str, optional) – The binning strategy. Must be in [“uniform”, “quantile”]. Defaults to “quantile”.

Returns:

A tuple of the fraction of positives in each bin and the mean predicted probability in each bin.

Return type:

prob_true, prob_pred (tuple[torch.Tensor, torch.Tensor])

DLL.Data.Metrics.categorical_cross_entropy(predictions, true_output)

Calculates the categorical cross entropy of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted values as a probability distribution. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A one-hot encoded tensor of true values. Must be the same shape as the prediction.

Returns:

The categorical cross entropy of the predictions.

Return type:

float

DLL.Data.Metrics.confusion_matrix(predictions, true_output)

The confusion matrix. At the moment, the problem must be binary classification. The element at (i, j) represents the number of observations in class i predicted to be class j.

Parameters:

• predictions (torch.Tensor) – A torch tensor of either predicted labels or probabilities. If is given probabilities, calculates the corresponding predictions.

• true_output (torch.Tensor) – A torch tensor of either true labels or one-hot encoded values. If is given a one-hot encoded tensor, calculates the corresponding labels.

Returns:

The confusion matrix.

Return type:

torch.Tensor of shape (n_classes, n_classes)

DLL.Data.Metrics.exponential_loss(predictions, true_output)

Calculates the exponential loss of the predictions. The problem must be binary classification.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted probabilities as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true labels as a vector. Must be the same shape as the predictions.

Returns:

The exponential loss of the predictions.

Return type:

float

DLL.Data.Metrics.f1_score(predictions, true_output)

Calculates the f1 score of the predictions. The problem must be binary classification.

Parameters:

• predictions (torch.Tensor of shape (n_samples,)) – A torch tensor of either predicted labels or probabilities. If is given probabilities, calculates the corresponding predictions.

• true_output (torch.Tensor of shape (n_samples,)) – A torch tensor of true labels.

Returns:

the f1 score of the predictions.

Return type:

float

DLL.Data.Metrics.huber_loss(predictions, true_output)

Calculates the huber loss of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

Returns:

The huber loss of the predictions.

Return type:

float

DLL.Data.Metrics.mean_absolute_error(predictions, true_output)

Calculates the mean absolute error of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

Returns:

The mean absolute error of the predictions.

Return type:

float

DLL.Data.Metrics.mean_squared_error(predictions, true_output)

Calculates the mean squared error of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the predictions.

Returns:

The mean squared error of the predictions.

Return type:

float

DLL.Data.Metrics.median_absolute_error(predictions, true_output)

Calculates the median absoalute error of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

Returns:

The median absolute error of the predictions.

Return type:

float

DLL.Data.Metrics.precision(predictions, true_output)

Calculates the precision or the positive predictive value of the predictions. The problem must be binary classification to be able to calculate the precision.

Parameters:

• predictions (torch.Tensor of shape (n_samples,)) – A torch tensor of either predicted labels or probabilities. If is given probabilities, calculates the corresponding predictions.

• true_output (torch.Tensor of shape (n_samples,)) – A torch tensor of true labels.

Returns:

the precision of the predictions.

Return type:

float

DLL.Data.Metrics.prob_to_pred(probabilities)

Converts probabilities to predicted labels.

Parameters:

probabilities (torch.Tensor with 1 or 2 dimensions) – A tensor of probabilities. Must either be 1 or 2 dimensional.

DLL.Data.Metrics.r2_score(predictions, true_output)

Calculates the coefficient of determination of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

Returns:

The coefficient of determination of the predictions.

Return type:

float

DLL.Data.Metrics.recall(predictions, true_output)

Calculates the recall or the sensitivity of the predictions. The problem must be binary classification to be able to calculate the recall.

Parameters:

• predictions (torch.Tensor of shape (n_samples,)) – A torch tensor of either predicted labels or probabilities. If is given probabilities, calculates the corresponding predictions.

• true_output (torch.Tensor of shape (n_samples,)) – A torch tensor of true labels.

Returns:

the recall of the predictions.

Return type:

float

DLL.Data.Metrics.roc_auc(probabilities, true_output, thresholds)

Calculates area under the roc curve. The problem must be binary classification.

Parameters:

• probabilities (torch.Tensor of shape (n_samples,)) – A torch tensor of probabilities.

• true_output (torch.Tensor of shape (n_samples,)) – A torch tensor of true labels.

• thresholds (torch.Tensor of shape (n_thresholds,)) – A tensor of thresholds. Every value must be between 0 and 1.

Returns:

the area under the roc curve.

Return type:

float

DLL.Data.Metrics.roc_curve(probabilities, true_output, thresholds)

Calculates receiver operating characteristic curve. The problem must be binary classification.

Parameters:

• probabilities (torch.Tensor of shape (n_samples,)) – A torch tensor of probabilities.

• true_output (torch.Tensor of shape (n_samples,)) – A torch tensor of true labels.

• thresholds (torch.Tensor of shape (n_thresholds,)) – A tensor of thresholds. Every value must be between 0 and 1.

Returns:

A tuple of false-positive-rate and true-positive-rate evaluated at the given thresholds.

Return type:

tuple[torch.Tensor, torch.Tensor]

DLL.Data.Metrics.root_mean_squared_error(predictions, true_output)

Calculates the root mean squared error of the predictions.

Parameters:

• predictions (torch.Tensor) – A tensor of predicted as a vector. Must be the same shape as the true_output.

• true_output (torch.Tensor) – A tensor of true values as a vector. Must be the same shape as the prediction.

Returns:

The root mean squared error of the predictions.

Return type:

float

DLL.Data.Metrics.silhouette_score(X, y, return_samples=False)

Computes the silhouette score of a clustering algorithm.

Parameters:

• X (torch.Tensor) – The input data of the model.

• y (torch.Tensor) – The output classes.

• return_samples (bool, optional) – Determines if silhouette score is returned separately or is averaged accross every sample. Defaults to False, i.e. by default returns the average value.

Returns:

if return_samples, returns a 1 dimensional torch tensor of values and if false, returns the average silhouette score.

Return type:

float | torch.Tensor