active_learning

Active learning module with composable pool, strategies, and iterator.

Typical workflow:

from probly.evaluation.active_learning import (
    from_dataset, MarginSampling, active_learning_steps, compute_accuracy,
)

pool = from_dataset(x_train, y_train, x_test, y_test, initial_size=100, seed=42)
for state in active_learning_steps(pool, estimator, MarginSampling(), query_size=50):
    acc = compute_accuracy(state.estimator.predict(state.pool.x_test), state.pool.y_test)

Pass numpy arrays for a numpy-backed pipeline or torch tensors for a torch-backed pipeline. Each component dispatches independently on the array type it receives.

Submodules

loop

Active learning step iterator.

metrics

Evaluation metrics for active learning experiments.

pool

Active learning pool with backend dispatch for NumPy and PyTorch.

strategies

Active learning query strategies with backend dispatch for NumPy and PyTorch.

torch_metrics

Torch implementations of active learning metrics.

Functions

active_learning_steps

Yield AL state after initial training and each query-retrain cycle.

compute_accuracy

Compute classification accuracy.

compute_ece

Compute Expected Calibration Error.

compute_nauc

Compute normalized area under the score curve.

from_dataset

Create a pool by randomly splitting training data into labeled and unlabeled subsets.