deup¶
- probly.method.deup.deup(base: Predictor[In, Out], hidden_size: int = 1024, n_hidden_layers: int = 5, stationarizing_features: Sequence[str | dict[str, Any]] | None = None, sn_coeff: float | None = None) DEUPPredictor[In, Out][source]¶
Transform a model for Direct Epistemic Uncertainty Prediction [LJN+23].
Strips the final classification head from
baseto obtain a feature encoder, then attaches two heads:classification_head: the original lastnn.Linearlayer, kept for phase-1 cross-entropy training.error_head: a small MLP of depthn_hidden_layerswithhidden_sizeunits and ReLU activations, mapping encoder features to a scalar predicted per-sample cross-entropy.
Training proceeds in two phases:
Train
encoderandclassification_headjointly with standard cross-entropy, identical to a plain classifier.Freeze
encoderandclassification_head; trainerror_headon held-out data (e.g. the validation set) with MSE between the predicted scalar and the actual per-sample cross-entropy of the frozen main model.
Uncertainty decomposition at inference (classification setup of [LJN+23]):
total =
error_score(output oferror_head),aleatoric = 0 (ground-truth label entropy, unavailable without replicates),
epistemic =
error_score(the reducible part of the loss).
Note
The default stationarizing features (
log_maf_density+log_due_variance) match the paper’s CIFAR-10 setup in spirit: a normalizing-flow density estimate \(\log \hat{q}(z|D)\) in encoder feature space, and a DUE posterior variance \(\log \hat{V}(x)\) from a spectrally-normalized encoder + SVGP. The paper uses a MAF on raw pixels and DUE with full GP; here both operate on encoder features for scalability.- Parameters:
base – Base classification model to be transformed.
hidden_size – Width of each hidden layer in the error head.
n_hidden_layers – Number of hidden layers in the error head (minimum 1).
stationarizing_features – Sequence of stationarizing feature specifications (see [LJN+23]). Each entry is either a registry name (e.g.
"log_maf_density","log_due_variance") or a mapping{"name": ..., **kwargs}. The error head receives only these features — encoder features are excluded. At least one provider is required.sn_coeff – Lipschitz coefficient for spectral normalization of the encoder. Defaults to
None(no spectral norm). Automatically set to 3.0 when a provider that requires spectral norm is used (e.g.log_due_variance), matching the DUE / DDU convention.
- Returns:
The transformed DEUP predictor.