probly.conformal_prediction.utils.metrics

Utility functions for evaluating conformal prediction metrics.

Functions

average_set_size(prediction_sets)	Compute average size of prediction sets.
empirical_coverage(prediction_sets, true_labels)	Compute empirical coverage of prediction sets.

probly.conformal_prediction.utils.metrics.average_set_size(prediction_sets)

Compute average size of prediction sets.

Parameters:

prediction_sets (ndarray[Any, dtype[bool]]) – A (n_instances, n_labels) boolean array indicating which labels are included in each prediction set.

Returns:

Average size of the prediction sets as a float.

Return type:

float

probly.conformal_prediction.utils.metrics.empirical_coverage(prediction_sets, true_labels)

Compute empirical coverage of prediction sets.

Parameters:

• prediction_sets (ndarray[Any, dtype[bool]]) – A (n_instances, n_labels) boolean array indicating which labels are included in each prediction set.

• true_labels (ndarray[Any, dtype[integer]]) – A sequence of true labels for each instance.

Returns:

Empirical coverage as a float.

Raises:

ValueError – If the number of instances in prediction_sets and true_labels do not match.

Return type:

float