metrics

Metrics with backend dispatch for NumPy, PyTorch, and JAX.

Submodules

jax

JAX implementation of Metrics.

numpy

NumPy implementation of Metrics.

torch

PyTorch implementation of Metrics.

Functions

accuracy

Compute top-1 classification accuracy.

auc

Compute area under a curve using the trapezoid rule.

average_interval_width

Compute the mean width of per-class probability intervals.

average_precision_score

Compute average precision (area under the precision-recall curve).

brier_loss

Compute the mean Brier loss of the predictions, the average of brier_loss_per_instance().

brier_loss_per_instance

Compute the Brier loss sum_k (p_k - [k == y])^2 of each prediction.

classwise_ece

Compute the classwise expected calibration error (classwise-ECE) [KPerelloNKangsepp+19].

convex_hull_coverage

Empirical convex-hull coverage for distribution-valued targets.

coverage

Compute the empirical coverage of a predicted set against true labels.

efficiency

Compute the average size of a predicted set.

expected_calibration_error

Compute the confidence expected calibration error (ECE) [GPSW17].

false_negative_rate

Compute the false negative rate FN / (FN + TP) of binary predictions.

false_positive_rate

Compute the false positive rate FP / (FP + TN) of binary predictions.

log_loss

Compute the mean log loss of the predictions, the average of log_loss_per_instance().

log_loss_per_instance

Compute the log loss -log(p_y) of each prediction.

precision_recall_curve

Compute precision-recall curve.

roc_auc_score

Compute area under the ROC curve.

roc_curve

Compute receiver operating characteristic (ROC) curve.

spherical_loss

Compute the mean spherical loss of the predictions, the average of spherical_loss_per_instance().

spherical_loss_per_instance

Compute the spherical loss 1 - p_y / ||p||_2 of each prediction.

zero_one_loss

Compute the mean zero-one loss of the predictions, the average of zero_one_loss_per_instance().

zero_one_loss_per_instance

Compute the zero-one loss 1 - [argmax_k p_k == y] of each prediction.