probly.evaluation.ood.out_of_distribution_detection_fpr_at_x_tpr¶

probly.evaluation.ood.out_of_distribution_detection_fpr_at_x_tpr(in_distribution: ndarray, out_distribution: ndarray, tpr_target: float = 0.95) → float[source]¶

Perform out-of-distribution detection using false positive rate at a given true positive rate.

If no thresholds are specified, the default tpr_target is 0.95.

This can be epistemic uncertainty, as is common, but also e.g. softmax confidence.

Parameters:

in_distribution – Scores for in-distribution samples.
out_distribution – Scores for out-of-distribution samples.
tpr_target – Target TPR value in (0, 1].

Returns:

False positive rate at the first threshold where TPR >= tpr_target.

Raises:

ValueError – If tpr_target is not in (0, 1] or cannot be achieved.

Note

Assumes that larger scores correspond to the positive class (out-of-distribution).