efficiency

probly.metrics.efficiency = <flextype.singledispatch.Flexdispatch object>

Compute the average size of a predicted set.

Efficiency reports how compact the predicted set is on average. Smaller is better. The exact semantics depend on the dispatched type:

  • One-hot conformal sets: average cardinality (number of selected classes).

  • Interval conformal sets: average interval width (upper - lower).

  • Singleton credal sets: 1 by definition (one distribution yields one argmax class per sample).

  • Discrete credal sets: average number of distinct argmax classes across the vertex set. Differs from the convex hull’s interval-dominance cardinality (which is always >= the discrete count).

  • Convex / DistanceBased / ProbabilityIntervals / DirichletLevelSet credal sets: cardinality of the interval-dominance prediction set.

Parameters:

y_pred – A predicted-set representation. See module docstring for the registered types.

Returns:

The mean set size as a float.

Raises:

NotImplementedError – If no implementation is registered for the type of y_pred.

Examples using probly.metrics.efficiency

Classification Conformal Prediction — sklearn

Classification Conformal Prediction — sklearn

Classification Conformal Prediction — PyTorch

Classification Conformal Prediction — PyTorch

How Conformal Sets Change with the Score and with Alpha

How Conformal Sets Change with the Score and with Alpha

A Gentle Introduction to Conformal Prediction

A Gentle Introduction to Conformal Prediction

Adaptive Conformal Intervals on Heteroscedastic Data

Adaptive Conformal Intervals on Heteroscedastic Data

Quantile Regression Conformal Prediction — sklearn

Quantile Regression Conformal Prediction — sklearn

Quantile Regression Conformal Prediction — PyTorch

Quantile Regression Conformal Prediction — PyTorch

Regression Conformal Prediction — sklearn

Regression Conformal Prediction — sklearn

Regression Conformal Prediction — PyTorch

Regression Conformal Prediction — PyTorch

Mixing torch-uncertainty and probly

Mixing torch-uncertainty and probly

Cyclone forecast uncertainty with WeatherNext

Cyclone forecast uncertainty with WeatherNext