SecondOrderScoringRuleDecomposition

class probly.quantification.decomposition.scoring_rule.SecondOrderScoringRuleDecomposition(distribution: SecondOrderDistributionLike, scoring_rule: ScoringRule)[source]

Bases: AdditiveDecomposition, Generic

Total/aleatoric/epistemic decomposition induced by a per-instance scoring rule.

The scoring rule is a function such as probly.metrics.log_loss_per_instance(). For a second-order categorical distribution Q with Bayesian model average theta_bar = E[theta] and generalized entropy G(theta) = scoring_rule(theta, theta), the expected loss of theta under itself:

  • Total uncertainty: G(theta_bar).

  • Aleatoric uncertainty: E_{theta ~ Q}[G(theta)].

  • Epistemic uncertainty: total - aleatoric (a Jensen gap, non-negative when the rule is proper).

The decomposition is additive: total == aleatoric + epistemic.

Example

A custom pseudo-spherical rule of order 3, with G(theta) = 1 - ||theta||_3, only needs to accept target probabilities:

def pseudo_spherical_loss(y_prob, y_true):
    norm = torch.sum(y_prob**3, dim=-1, keepdim=True) ** (1 / 3)
    return torch.sum(y_true * (1.0 - y_prob**2 / norm**2), dim=-1)

decomposition = SecondOrderScoringRuleDecomposition(sample, pseudo_spherical_loss)
property aleatoric: AU

The aleatoric uncertainty of the decomposition.

canonical_notion[source]

alias of TotalUncertainty

property components: list[type[Notion]]

The components of the decomposition.

distribution: SecondOrderDistributionLike
property epistemic: EU

The epistemic uncertainty of the decomposition.

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.
get_canonical() → Notion[source]

Return the canonical notion of the decomposition.

get_notion(notion: type[N]) → N[source]

Return the component corresponding to the given notion.

items() → a set-like object providing a view on D's items
keys() → a set-like object providing a view on D's keys
scoring_rule: ScoringRule
property total: TU

The total uncertainty of the decomposition.

values() → an object providing a view on D's values