SecondOrderScoringRuleDecomposition¶
- class probly.quantification.decomposition.scoring_rule.SecondOrderScoringRuleDecomposition(distribution: SecondOrderDistributionLike, scoring_rule: ScoringRule)[source]¶
Bases:
AdditiveDecomposition,GenericTotal/aleatoric/epistemic decomposition induced by a per-instance scoring rule.
The scoring rule is a function such as
probly.metrics.log_loss_per_instance(). For a second-order categorical distributionQwith Bayesian model averagetheta_bar = E[theta]and generalized entropyG(theta) = scoring_rule(theta, theta), the expected loss ofthetaunder itself:Total uncertainty:
G(theta_bar).Aleatoric uncertainty:
E_{theta ~ Q}[G(theta)].Epistemic uncertainty:
total - aleatoric(a Jensen gap, non-negative when the rule is proper).
The decomposition is additive:
total == aleatoric + epistemic.Example
A custom pseudo-spherical rule of order 3, with
G(theta) = 1 - ||theta||_3, only needs to accept target probabilities:def pseudo_spherical_loss(y_prob, y_true): norm = torch.sum(y_prob**3, dim=-1, keepdim=True) ** (1 / 3) return torch.sum(y_true * (1.0 - y_prob**2 / norm**2), dim=-1) decomposition = SecondOrderScoringRuleDecomposition(sample, pseudo_spherical_loss)
- property aleatoric: AU¶
The aleatoric uncertainty of the decomposition.
- canonical_notion[source]¶
alias of
TotalUncertainty
- distribution: SecondOrderDistributionLike¶
- property epistemic: EU¶
The epistemic uncertainty of the decomposition.
- get(k[, d]) D[k] if k in D, else d. d defaults to None.¶
- items() a set-like object providing a view on D's items¶
- keys() a set-like object providing a view on D's keys¶
- scoring_rule: ScoringRule¶
- property total: TU¶
The total uncertainty of the decomposition.
- values() an object providing a view on D's values¶