1.2. Where Uncertainty Arises¶
1.2.1. Aleatoric Uncertainty¶
return to case a: the blurry photo, or the one on which humans disagree
given these pixels, the outcome is genuinely not determined, and the world itself holds a 51/49 mix
consequently, even the best possible model on these features outputs 0.51, and the resulting error rate cannot be trained away
roughly speaking, this is aleatoric uncertainty, from alea, the dice, also called data uncertainty [HullermeierW21]
it resides in the data-generating process: class overlap, label noise, measurement noise
annotator disagreement makes it observable: when many humans label the same image, the split of their votes estimates the aleatoric part [PBGR19]
more data sharpens the estimate of the odds, but it does not remove the odds
1.2.2. Epistemic Uncertainty¶
return to case b: an animal the model was never trained on
here the world is not mixed, the model is, so the 0.51 originates in the model and not in the image
roughly speaking, this is uncertainty about the model itself, caused by too little data in this region or by a model class that cannot represent the true relation
it is called epistemic uncertainty, from episteme, knowledge, also called model uncertainty [KG17]
it is reducible in principle: the right data, such as labeled examples near this input, moves it
this is precisely the claim for which, as Why One Number Is Not Enough argued, a single distribution has no slot
it shows up as disagreement between plausible models, which is why it requires the richer objects of representing uncertainty
1.2.3. The Reducibility Test¶
a single question separates the two: would more data move this number?
in case a it would not, since more blurry photos only re-estimate the same 51/49, so the uncertainty is aleatoric
in case b it would, since labeled examples of the new animal move the prediction, so the uncertainty is epistemic
note that “more data” is ambiguous: more samples of the same kind, targeted labels near this input, and new features are three different interventions
the answer determines the response: epistemic uncertainty justifies collecting data or routing to a human, whereas aleatoric uncertainty says stop collecting and either accept the odds or improve the features
in other words, the split is exactly the information that the two identical 0.51s could not carry
it is also computable, in the sense that total uncertainty can be decomposed into an aleatoric and an epistemic part, see decomposing the total
1.2.4. Why the Split Is Relative¶
the last option, improving the features, already hints that the split is not a property of the world but of the modeling setup [HullermeierW21]
it is relative to the feature set: blurry dog versus fox is aleatoric for a pixel classifier, yet with a sharper sensor or a second view, part of the noise becomes signal that is merely missing, that is, epistemic
it is relative to the model class: what a linear model must write off as noise, a richer model may resolve
it is relative to the context: a coin flip is aleatoric to the bettor, but epistemic to the physicist measuring the throw
“irreducible” therefore always means irreducible given this feature set and this model class
as a result, aleatoric and epistemic numbers obtained from different setups are not comparable
1.2.5. Distribution Shift¶
everything so far assumed that training and deployment data are drawn from the same distribution
deployment tends to break that assumption, through a new sensor, a new season, or a new population
shift is case b at scale: not one unfamiliar animal, but a whole regime without training density
it is thus arguably the main source of epistemic uncertainty at deployment time
two kinds are worth distinguishing: under covariate shift the inputs move, under concept drift the input-outcome relation moves, so that even the aleatoric odds change
whether an uncertainty estimate holds up under shift is an empirical question, and the answer is often negative [SOF+19]
out-of-distribution inputs lie at the far end of shift, and whether a score separates them from familiar ones is one of the checks in evaluating uncertainty