expected_calibration_error¶
- probly.metrics.expected_calibration_error = <flextype.singledispatch.Flexdispatch object>¶
Compute the confidence expected calibration error (ECE) [GPSW17].
Predictions are grouped into
num_binsequal-width bins by confidence, i.e. the maximum predicted probability, and the ECE is the frequency-weighted mean absolute gap between accuracy and mean confidence across bins. In contrast toclasswise_ece(), only the winning class probability enters.- Parameters:
y_prob – Predicted class probabilities of shape
(n, k), or a categorical distribution.y_true – Integer ground-truth class labels of shape
(n,).num_bins – Number of equal-width confidence bins.
- Returns:
The expected calibration error in
[0, 1].- Raises:
NotImplementedError – If no implementation is registered for the type of
y_prob.