classwise_ece¶
- probly.metrics.classwise_ece = <flextype.singledispatch.Flexdispatch object>¶
Compute the classwise expected calibration error (classwise-ECE) [KPerelloNKangsepp+19].
For every class
j, the predicted probabilitiesp_jare grouped intonum_binsequal-width bins; the per-class error is the bin-size-weighted mean absolute difference between the empirical frequency of classjand its mean predicted probability within each bin, and the classwise-ECE averages these errors over all classes. In contrast to the confidence-basedexpected_calibration_error(), it is also sensitive to miscalibration on non-maximal class probabilities.- Parameters:
y_prob – Predicted class probabilities of shape
(n, k), or a categorical distribution.y_true – Integer ground-truth class labels of shape
(n,).num_bins – Number of equal-width probability bins per class.
- Returns:
The classwise expected calibration error in
[0, 1].- Raises:
NotImplementedError – If no implementation is registered for the type of
y_prob.