probly.calibration.visualization.reliability_diagram

Implementation for Reliability Diagrams.

Functions

compute_reliability_diagram(probabilities, ...)	Calculates the bins for a reliability diagram.
plot_reliability_diagram(diagram[, title])	Plots the diagram calculated with compute_reliability_diagram.

probly.calibration.visualization.reliability_diagram.compute_reliability_diagram(probabilities, labels, n_bins=10)

Calculates the bins for a reliability diagram.

Parameters:

• probabilities (ndarray) – The probabilities from the model

• labels (ndarray) – The labels corresponding to the probabilities

• n_bins (int) – The number of bins the intervall [0, 1] should be divided into

Returns:

An dict object of the form [“n_bins”][“bin_accuracies”][“bin_confidences”][“bin_counts”] where

• [“n_bins contains”] the number of bins

• [“bin_accuracies”] contains the mean accuracies for all bins

• [“bin_confidences”] contains the mean confidences for all bins

• [“bin_counts”] contains the number of predictions in every bin

Return type:

diagram

probly.calibration.visualization.reliability_diagram.plot_reliability_diagram(diagram, title='Model Calibration')

Plots the diagram calculated with compute_reliability_diagram.

Parameters:

• diagram (dict) – A dictionary containing the fields [“n_bins”][“bin_accuracies”][“bin_confidences”][“bin_counts”]

• title (str) – The title the diagram should have

Returns:

The matplotlib figure ax: The matplotlib axis

Return type:

fig