Interpretation techniques

Introduction

Quantifying a model’s uncertainty is only the first step; the true value comes from interpreting what that uncertainty means. A single score tells us if a model is uncertain, but a deeper analysis can tell us why. The probly.quantification module provides a suite of functions designed to facilitate this interpretation.

The key technique probly supports is decomposition of uncertainty. For sampling-based methods (like MC-Dropout or Ensembles), the library provides functions to separate total predictive uncertainty into two components:

• Aleatoric uncertainty – captures the inherent noise and ambiguity in the data. Can be calculated with conditional_entropy (classification) or expected_conditional_variance (regression).

• Epistemic uncertainty – represents the model’s own ignorance or lack of knowledge. Can be calculated with mutual_information (both tasks).

Distinguishing these is critical for debugging and building trust. High aleatoric uncertainty may be irreducible due to noisy data, while high epistemic uncertainty signals the model is “out of its depth” on unfamiliar inputs. This information can guide out-of-distribution detection or active learning, leading to more robust and reliable systems.