Note
Go to the end to download the full example code.
Het-Net on Two Moons¶
Het-Net augments a standard classifier with a learnable heteroscedastic noise head that draws Monte Carlo samples in logit space. The representer reuses the same noise mechanism at inference time to estimate per-sample aleatoric uncertainty, which is helpful on data with input-dependent label noise such as this noisy Two Moons setup.
from __future__ import annotations
from sklearn.datasets import make_moons
import torch
import torch.nn.functional as F
import numpy as np
from probly.method.het_net import het_net
from probly.representer import representer
from examples.utils.model import SequentialModel
from examples.utils.plotting import plot_example_uncertainty
Setup¶
Two Moons with input-dependent (heteroscedastic) Gaussian noise: points further from each class’ core get a larger noise scale, mimicking spatially varying label/observation noise.
X, y = make_moons(n_samples=500, noise=0.0, random_state=0)
rng = np.random.default_rng(0)
noise_scale = np.zeros_like(X[:, 0])
x0 = X[y == 0, 0]
noise_scale[y == 0] = 0.05 + 0.4 * (np.max(x0) - x0) / (np.max(x0) - np.min(x0))
x1 = X[y == 1, 0]
noise_scale[y == 1] = 0.05 + 0.4 * (x1 - np.min(x1)) / (np.max(x1) - np.min(x1))
X += rng.normal(scale=np.expand_dims(noise_scale, 1), size=X.shape)
X_tensor = torch.from_numpy(X).float()
y_tensor = torch.from_numpy(y).long()
Model¶
Het-Net wraps the backbone with a HeteroscedasticLayer that learns a
per-sample noise distribution over the logits. The noise head is co-trained
with the backbone and captures input-dependent (aleatoric) uncertainty.
base_model = SequentialModel()
het_net_model = het_net(base_model, num_samples=100, predictor_type="logit_classifier")
Training¶
The heteroscedastic head draws num_samples noise samples per input in a
single vectorized forward pass and returns the log of the averaged
probabilities, which cross-entropy turns into the negative log-likelihood.
opt = torch.optim.Adam(het_net_model.parameters(), lr=1e-3)
het_net_model.train()
for _epoch in range(500):
opt.zero_grad()
log_probs = het_net_model(X_tensor)
loss = F.cross_entropy(log_probs, y_tensor)
loss.backward()
opt.step()
Uncertainty Evaluation¶
het_net_model.eval()
rep = representer(het_net_model)
plot = plot_example_uncertainty(X, y, rep, title="HET-Net Predictive Uncertainty", notion="aleatoric")
plot.show()

Total running time of the script: (0 minutes 6.731 seconds)