efficiency¶
- probly.metrics.efficiency = <flextype.singledispatch.Flexdispatch object>¶
Compute the average size of a predicted set.
Efficiency reports how compact the predicted set is on average. Smaller is better. The exact semantics depend on the dispatched type:
One-hot conformal sets: average cardinality (number of selected classes).
Interval conformal sets: average interval width (
upper - lower).Singleton credal sets:
1by definition (one distribution yields one argmax class per sample).Discrete credal sets: average number of distinct argmax classes across the vertex set. Differs from the convex hull’s interval-dominance cardinality (which is always
>=the discrete count).Convex / DistanceBased / ProbabilityIntervals / DirichletLevelSet credal sets: cardinality of the interval-dominance prediction set.
- Parameters:
y_pred – A predicted-set representation. See module docstring for the registered types.
- Returns:
The mean set size as a float.
- Raises:
NotImplementedError – If no implementation is registered for the type of
y_pred.
Examples using probly.metrics.efficiency¶
How Conformal Sets Change with the Score and with Alpha
Adaptive Conformal Intervals on Heteroscedastic Data
Quantile Regression Conformal Prediction — sklearn
Quantile Regression Conformal Prediction — PyTorch