Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
probly 0.9.0 documentation
Light Logo Dark Logo

Table of Contents

  • Introduction
  • Installation
  • User Guide
    • 1. Transformation
    • 2. Representation
    • 3. Quantification
    • 4. Evaluation
    • 5. Visualization
  • Examples and Tutorials
    • Examples
      • Active Learning
        • Active Learning with sklearn - Margin vs Random
        • Active Learning with PyTorch - BADGE Selection
      • Conformal Prediction
        • Regression Conformal Prediction — sklearn
        • Classification Conformal Prediction — sklearn
        • Quantile Regression Conformal Prediction — sklearn
        • Classification Conformal Prediction — PyTorch
        • Regression Conformal Prediction — PyTorch
        • Conformalized Credal Set Prediction — PyTorch.
        • Quantile Regression Conformal Prediction — PyTorch
      • Integrations
        • Mixing torch-uncertainty and probly
      • Method
        • DARE on Two Moons
        • Laplace Approximation on Two Moons
        • Mahalanobis OOD on Two Moons
        • DUQ on Two Moons
        • DDU on Two Moons
        • Het-Net on Two Moons
        • Credal Net Visualization
        • Evidential on Two Moons
        • Credal Wrapper Output Visualization
        • Credal Ensembling Visualization
        • Credal BNN Visualization
        • DARE on MNIST
        • SNGP Distance Awareness on 2D Toys
        • Laplace on MNIST
        • Credal BNN on MNIST
        • Credal Wrapper on MNIST
        • Credal Ensembling on MNIST
        • Credal Net on MNIST
        • SNGP on MNIST
        • Mahalanobis OOD on MNIST
        • Het-Net on MNIST
        • DUQ on MNIST
        • DDU on MNIST
        • Credal Relative Likelihood Visualization
        • Evidential on MNIST
        • Credal Relative Likelihood on MNIST
        • DEUP on Two Moons
        • DEUP on MNIST
      • Plot
        • Visualising OOD detection results
        • Plotting credal sets on the simplex
        • Plotting binary credal sets on an interval
        • Plotting credal sets on a spider (radar) chart
      • Pytraverse
        • A Brief Introduction to PyTraverse
      • Quantification
        • Uncertainty Quantification
        • Ensemble Regression Uncertainty
        • Ensemble Ordinal Classification Uncertainty
      • Release Highlights
      • Representation
        • Singleton credal set
        • Convex credal set
        • Discrete credal set
        • Distance-based credal set
        • Probability-intervals credal set
        • Working with ArraySample
      • Streaming
        • Streaming uncertainty with ARFRegressor
        • Streaming uncertainty with ARFClassifier
        • MC-Dropout uncertainty on a 2-D stream
      • Transformation
        • DropConnect on Two Moons
        • Deep Ensemble on Two Moons
        • MC Dropout on Two Moons
        • Natural Posterior Network on Two Moons
        • Bayesian Neural Network on Two Moons
        • Posterior Network on Two Moons
        • Sub-Ensemble on Two Moons
        • Bayesian Ensemble on Two Moons
        • BatchEnsemble on Two Moons
        • MC Dropout on MNIST
        • DropConnect on MNIST
        • Deep Ensemble on MNIST
        • Bayesian Neural Network on MNIST
        • Natural Posterior Network on MNIST
        • Bayesian Ensemble on MNIST
        • Posterior Network on MNIST
        • Sub-Ensemble on MNIST
        • BatchEnsemble on MNIST
  • API Reference
    • probly
      • calibrator
      • conformal_scores
        • absolute_error
          • absolute_error.jax
          • absolute_error.torch
            • compute_absolute_error_score_torch
            • compute_absolute_error_score_torch_sample
        • aps
          • aps.flax
          • aps.torch
            • compute_aps_score_torch
        • cqr
          • cqr.jax
          • cqr.torch
            • compute_cqr_score_torch
        • cqr_r
          • cqr_r.jax
          • cqr_r.torch
            • compute_cqr_r_score_torch
        • dirichlet_relative_likelihood
          • dirichlet_relative_likelihood.torch
            • compute_dirichlet_rl_score_torch
            • compute_dirichlet_rl_score_torch_dirichlet
        • inner_product
          • inner_product.torch
            • compute_inner_product_score_torch
        • kullback_leibler
          • kullback_leibler.torch
            • compute_kl_divergence_score_torch
        • lac
          • lac.jax
            • compute_lac_score_jax
          • lac.torch
            • compute_lac_score_torch
            • compute_lac_score_torch_categorical
            • compute_lac_score_torch_sample
        • raps
          • raps.jax
          • raps.torch
            • compute_raps_score_torch
        • saps
          • saps.jax
          • saps.torch
            • compute_saps_score_torch
        • total_variation
          • total_variation.torch
            • compute_tv_score_torch
        • uacqr
          • uacqr.jax
          • uacqr.torch
            • compute_uacqr_score_func_torch
        • wasserstein_distance
          • wasserstein_distance.torch
            • compute_wasserstein_distance_score_torch
      • datasets
        • torch
          • Benthic
          • CIFAR10C
          • CIFAR10H
          • CIFAR10HDCIC
          • DCICDataset
          • ImageNetReaL
          • MiceBone
          • Pig
          • Plankton
          • QualityMRI
          • Synthetic
          • Treeversity1
          • Treeversity6
          • Turkey
      • decider
        • categorical_distribution
          • categorical_distribution.array
          • categorical_distribution.torch
      • evaluation
        • active_learning
          • active_learning.loop
            • ALState
            • active_learning_steps
          • active_learning.metrics
            • compute_nauc
          • active_learning.pool
            • active_learning.pool.array
              • NumpyActiveLearningPool
            • active_learning.pool.torch
              • TorchActiveLearningPool
          • active_learning.strategies
            • active_learning.strategies.array
            • active_learning.strategies.torch
        • ood
          • evaluate_ood
          • out_of_distribution_detection_aupr
          • out_of_distribution_detection_auroc
          • out_of_distribution_detection_fnr_at_x_tpr
          • out_of_distribution_detection_fpr_at_x_tpr
          • parse_dynamic_metric
        • tasks
          • selective_prediction
      • integrations
        • torch_uncertainty
          • TorchUncertaintyCalibratedLogitRepresenter
          • TorchUncertaintyConformalRepresenter
          • TorchUncertaintySampleRepresenter
      • layers
        • flax
          • BatchEnsembleConv
          • BatchEnsembleLinear
          • DropConnectLinear
        • torch
          • BatchEnsembleConv2d
          • BatchEnsembleLinear
          • BayesConv2d
          • BayesLinear
          • DropConnectLinear
          • GaussianMixtureHead
          • HeteroscedasticLayer
          • IRDHead
          • IntBatchNorm1d
          • IntBatchNorm2d
          • IntConv2d
          • IntLinear
          • IntSoftmax
          • MahalanobisHead
          • NatPNClassHead
          • NatPNRegHead
          • NormalInverseGammaLinear
          • RadialNormalizingFlow
          • RadialNormalizingFlowStack
          • RegressionHead
          • SNCoeffParametrization
          • SNGPLayer
          • SharedMaskDropout
          • apply_spectral_norm_to_encoder
          • pack_interval
          • unpack_interval
      • lazy_types
      • method
        • batchensemble
        • bayesian
        • calibration
        • cast
        • conformal
        • conformal_credal_set
        • credal_bnn
        • credal_ensembling
        • credal_net
          • CredalNetPredictor
        • credal_relative_likelihood
        • credal_wrapper
        • dare
        • ddu
        • deup
          • deup.torch
            • ErrorPredictionHead
            • LogDUEVariance
            • LogGMMDensity
            • LogMAFDensity
            • LogMCDropoutVariance
            • StationarizingFeatureProvider
            • TorchDEUPPredictor
            • TorchDEUPRepresentation
        • dropconnect
        • dropout
        • duq
        • efficient_credal_prediction
        • ensemble
        • evidential
          • evidential.classification
          • evidential.regression
        • graph_posterior_network
        • het_net
        • laplace
          • laplace.torch
            • LaplaceRepresenter
        • mahalanobis
        • natural_posterior_network
        • posterior_network
        • prior_network
          • PriorNetworkPredictor
        • sngp
        • subensemble
      • metrics
        • array
          • auc_numpy
          • average_precision_score_numpy
          • precision_recall_curve_numpy
          • roc_auc_score_numpy
          • roc_curve_numpy
        • jax
          • auc_jax
          • average_precision_score_jax
          • precision_recall_curve_jax
          • roc_auc_score_jax
          • roc_curve_jax
        • torch
          • auc_torch
          • average_precision_score_torch
          • precision_recall_curve_torch
          • roc_auc_score_torch
          • roc_curve_torch
      • plot
        • config
          • PlotConfig
        • credal
          • credal.plot
            • plot_credal_set
        • ood
          • plot_histogram
          • plot_pr_curve
          • plot_roc_curve
        • utils
      • predictor
        • laplace
        • sklearn
          • predict_sklearn
      • quantification
        • decomposition
          • decomposition.decomposition
            • AdditiveDecomposition
            • AleatoricDecomposition
            • AleatoricEpistemicDecomposition
            • AleatoricEpistemicTotalDecomposition
            • AleatoricTotalDecomposition
            • CachingDecomposition
            • ConstantTotalDecomposition
            • Decomposer
            • Decomposition
            • EpistemicDecomposition
            • EpistemicTotalDecomposition
            • TotalDecomposition
          • decomposition.entropy
          • decomposition.ordinal
          • decomposition.scoring_rule
          • decomposition.spectral
            • decomposition.spectral.torch
              • SpectralDecomposition
              • spectral_decomposition
              • torch_embedding_sample_sample_spectral_decomposition
          • decomposition.variance
          • decomposition.wasserstein
          • decomposition.zero_one
        • measure
        • notion
          • AleatoricUncertainty
          • EpistemicUncertainty
          • Notion
          • TotalUncertainty
        • scoring_rule
          • scoring_rule.array
            • array_brier_loss_vector
            • array_log_loss_vector
            • array_spherical_loss_vector
            • array_zero_one_loss_vector
          • scoring_rule.torch
            • torch_brier_loss_vector
            • torch_log_loss_vector
            • torch_spherical_loss_vector
            • torch_zero_one_loss_vector
      • representation
        • array_like
          • ArrayFlagsLike
          • ArrayLike
          • NumpyArrayLike
          • NumpyArrayLikeConvertible
          • NumpyArrayLikeImplementation
        • conformal_set
          • conformal_set.array
            • ArrayIntervalConformalSet
            • ArrayOneHotConformalSet
          • conformal_set.torch
            • TorchIntervalConformalSet
            • TorchOneHotConformalSet
        • credal_set
          • credal_set.array
            • ArrayCategoricalCredalSet
            • ArrayConvexCredalSet
            • ArrayDiscreteCredalSet
            • ArrayDistanceBasedCredalSet
            • ArrayProbabilityIntervalsCredalSet
            • ArraySingletonCredalSet
          • credal_set.array_functions
          • credal_set.torch
            • TorchCategoricalCredalSet
            • TorchConvexCredalSet
            • TorchDirichletLevelSetCredalSet
            • TorchDistanceBasedCredalSet
            • TorchProbabilityIntervalsCredalSet
        • distribution
          • distribution.array_bernoulli
            • ArrayBernoulliDistribution
            • ArrayBernoulliDistributionSample
            • ArrayLogitBernoulliDistribution
            • ArrayProbabilityBernoulliDistribution
          • distribution.array_categorical
            • ArrayCategoricalDistribution
            • ArrayCategoricalDistributionSample
            • ArrayLogitCategoricalDistribution
            • ArrayProbabilityCategoricalDistribution
          • distribution.array_dirichlet
            • ArrayDirichletDistribution
          • distribution.array_gaussian
            • ArrayGaussianDistribution
            • ArrayGaussianDistributionSample
          • distribution.torch_bernoulli
            • TorchBernoulliDistribution
            • TorchBernoulliDistributionSample
            • TorchLogitBernoulliDistribution
            • TorchProbabilityBernoulliDistribution
          • distribution.torch_categorical
            • TorchCategoricalDistribution
            • TorchCategoricalDistributionSample
            • TorchLogitCategoricalDistribution
            • TorchProbabilityCategoricalDistribution
          • distribution.torch_dirichlet
            • TorchDirichletDistribution
          • distribution.torch_gaussian
            • TorchGaussianDistribution
            • TorchGaussianDistributionSample
          • distribution.torch_mixture
            • TorchDirichletMixtureDistribution
            • TorchMixtureDistribution
          • distribution.torch_sparse_log_categorical
            • TorchSparseLogCategoricalDistribution
            • TorchSparseLogCategoricalDistributionSample
        • embedding
          • embedding.torch
            • TorchEmbedding
            • TorchEmbeddingSample
            • TorchEmbeddingSampleSample
        • jax_like
          • JaxLikeImplementation
        • representation
          • Representation
        • sample
          • sample.array
            • ArraySample
          • sample.array_functions
            • ArraySampleCreator
            • ArraySampleInternals
            • array_apply_along_axis_function
            • array_concatenate_function
            • array_copy_function
            • array_expand_dims_function
            • array_function_override
            • array_internals_override
            • array_matrix_transpose
            • array_reduction_function
            • array_reshape_function
            • array_sample_axis_preserving_function
            • array_sample_internals
            • array_squeeze_function
            • array_stack_function
            • array_swapaxes_function
            • array_transpose
            • track_sample_axis_after_reduction
          • sample.axis_tracking
            • ArrayIndex
            • AxisTrackingResult
            • track_axis
          • sample.jax
            • JaxArraySample
          • sample.jax_axis_tracking
          • sample.torch
            • TorchSample
          • sample.torch_axis_tracking
          • sample.torch_functions
            • TorchSampleCreator
            • TorchSampleInternals
            • torch_adjoint_function
            • torch_cat_function
            • torch_function_override
            • torch_internals_override
            • torch_movedim_function
            • torch_permute_function
            • torch_reduction_function
            • torch_sample_internals
            • torch_stack_function
            • torch_transpose_function
            • track_sample_dim_after_reduction
        • text_generation
          • text_generation.torch
            • TorchTextGeneration
            • TorchTextGenerationSample
            • TorchTextGenerationSampleSample
            • TorchTokenGeneration
        • torch_functions
          • torch_average
        • torch_like
          • TorchLike
          • TorchLikeConvertible
          • TorchLikeImplementation
      • representer
        • clarifier
          • clarifier.huggingface
            • HFQuestionClarifier
        • credal_set
          • ConvexCredalSetRepresenter
          • ProbabilityIntervalsRepresenter
          • RepresentativeConvexCredalSetRepresenter
          • SampleMeanConvexCredalSetRepresenter
        • credal_set_torch
          • torch_compute_representative_sample
        • embedder
          • embedder.huggingface
            • HFTextEmbedder
            • SentenceTransformerLike
        • sampler
          • sampler.flax
            • register_forced_train_mode
          • sampler.huggingface
            • HFTextGenerationSampler
            • load_model
          • sampler.sklearn
            • register_forced_fitted_already_mode
          • sampler.torch
            • register_forced_train_mode
        • semantic_clustering
          • semantic_clustering.huggingface
            • HFGreedySemanticClusterer
            • HFSemanticClusterer
      • train
        • bayesian
          • bayesian.torch
            • ELBOLoss
            • collect_kl_divergence
        • calibration
          • calibration.torch
            • ExpectedCalibrationError
            • FocalLoss
            • LabelRelaxationLoss
            • LabelSmoothingLoss
        • credal
          • credal.torch
            • intersection_probability_ce_loss
        • dare
          • dare.torch
            • dare_regularizer
        • evidential
          • evidential.common
          • evidential.torch
            • der_loss
            • dirichlet_entropy
            • evidential_ce_loss
            • evidential_kl_divergence
            • evidential_log_loss
            • evidential_mse_loss
            • evidential_nignll_loss
            • evidential_regression_regularization
            • ird_loss
            • kl_dirichlet
            • lop_gpn_loss
            • lp_fn
            • make_in_domain_target_alpha
            • make_ood_target_alpha
            • mixture_uce_loss
            • natpn_loss
            • normal_wishart_log_prob
            • pn_loss
            • postnet_loss
            • predictive_probs
            • regularization_fn
            • rpn_distillation_loss
            • rpn_loss
            • rpn_ng_kl
            • rpn_prior
            • unified_evidential_train
      • transformation
        • batchensemble
        • bayesian
        • bayesian_ensemble
        • calibration
          • calibration.sklearn
            • SklearnIdentityLogitEstimator
            • SklearnVectorScalingPredictor
            • calibrate_sklearn_calibrated_classifier_cv
            • generate_sklearn_scaling_calibrator
          • calibration.torch
            • TorchAffineLogitCalibrationPredictor
            • TorchIdentityLogitModel
            • TorchIsotonicCalibrationPredictor
            • generate_torch_scaling_calibrator
          • sklearn_identity_logit_estimator
          • torch_identity_logit_model
        • cast
        • class_bias_ensemble
        • conformal
          • conformal.flax
            • FlaxConformalSetPredictor
          • conformal.sklearn
            • SklearnConformalSetPredictor
          • conformal.torch
            • TorchConformalSetPredictor
        • conformal_credal_set
          • conformal_credal_set.torch
            • TorchConformalCredalSetPredictor
        • dirichlet_clipped_exp_one_activation
        • dirichlet_exp_activation
        • dirichlet_softplus_activation
        • dropconnect
        • dropout
        • ensemble
        • interval_classifier
        • natural_posterior_network
          • natural_posterior_network.torch
            • TorchNaturalPosteriorNetwork
        • normal_inverse_gamma_head
        • posterior_network
        • subensemble
        • transformation
          • PredictorTransformation
          • PredictorTransformationDecorator
          • predictor_transformation
      • traverse_nn
        • flax
        • reset_traverser
        • torch
        • torch_utils
        • utils
          • get_output_dim
      • utils
        • iterable
          • first_element
        • quantile
          • quantile.jax
          • quantile.torch
        • switchdispatch
        • torch
          • intersection_probability
          • temperature_softmax
          • torch_collect_outputs
          • torch_entropy
          • torch_reset_all_parameters
  • References and Further Reading
  • FAQ and Troubleshooting
GitHub Logo
Back to top

Active Learning¶

Examples concerning the probly.evaluation.active_learning module.

Active Learning with sklearn - Margin vs Random

Active Learning with sklearn - Margin vs Random

Active Learning with PyTorch - BADGE Selection

Active Learning with PyTorch - BADGE Selection
Next
Active Learning with sklearn - Margin vs Random
Previous
Examples
Copyright © 2025, probly team
Made with Sphinx and @pradyunsg's Furo