Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
probly 0.3.1 documentation
Light Logo Dark Logo

Table of Contents

  • Introduction
  • Installation
  • User Guide
    • 1. Transformation
    • 2. Representation
    • 3. Quantification
    • 4. Evaluation
    • 5. Visualization
  • Examples and Tutorials
    • Examples
      • Conformal Prediction
        • Regression Conformal Prediction — sklearn
        • Classification Conformal Prediction — sklearn
        • Quantile Regression Conformal Prediction — sklearn
        • Regression Conformal Prediction — PyTorch
        • Classification Conformal Prediction — PyTorch
        • Quantile Regression Conformal Prediction — PyTorch
      • Plot
        • Visualising OOD detection results
        • Plotting credal sets on the simplex
        • Plotting binary credal sets on an interval
        • Plotting credal sets on a spider (radar) chart
      • Pytraverse
        • A Brief Introduction to PyTraverse
      • Quantification
        • Uncertainty Quantification
      • Release Highlights
      • Representation
        • Singleton credal set
        • Convex credal set
        • Discrete credal set
        • Distance-based credal set
        • Probability-intervals credal set
        • Working with ArraySample
      • Streaming
        • Streaming uncertainty with ARFRegressor
        • Streaming uncertainty with ARFClassifier
        • MC-Dropout uncertainty on a 2-D stream
      • Transformation
        • Applying the Dropout Transformation
        • SNGP Distance Awareness on Two Moons
  • API Reference
    • probly
      • probly.calibrator
      • probly.conformal_scores
        • probly.conformal_scores.absolute_error
          • probly.conformal_scores.absolute_error.jax
          • probly.conformal_scores.absolute_error.torch
            • probly.conformal_scores.absolute_error.torch.compute_absolute_error_score_torch
            • probly.conformal_scores.absolute_error.torch.compute_absolute_error_score_torch_sample
        • probly.conformal_scores.aps
          • probly.conformal_scores.aps.flax
          • probly.conformal_scores.aps.torch
            • probly.conformal_scores.aps.torch.compute_aps_score_torch
        • probly.conformal_scores.cqr
          • probly.conformal_scores.cqr.jax
          • probly.conformal_scores.cqr.torch
            • probly.conformal_scores.cqr.torch.compute_cqr_score_torch
        • probly.conformal_scores.cqr_r
          • probly.conformal_scores.cqr_r.jax
          • probly.conformal_scores.cqr_r.torch
            • probly.conformal_scores.cqr_r.torch.compute_cqr_r_score_torch
        • probly.conformal_scores.lac
          • probly.conformal_scores.lac.jax
            • probly.conformal_scores.lac.jax.compute_lac_score_jax
          • probly.conformal_scores.lac.torch
            • probly.conformal_scores.lac.torch.compute_lac_score_torch
            • probly.conformal_scores.lac.torch.compute_lac_score_torch_categorical
            • probly.conformal_scores.lac.torch.compute_lac_score_torch_sample
        • probly.conformal_scores.raps
          • probly.conformal_scores.raps.jax
          • probly.conformal_scores.raps.torch
            • probly.conformal_scores.raps.torch.compute_raps_score_torch
        • probly.conformal_scores.saps
          • probly.conformal_scores.saps.jax
          • probly.conformal_scores.saps.torch
            • probly.conformal_scores.saps.torch.compute_saps_score_torch
        • probly.conformal_scores.uacqr
          • probly.conformal_scores.uacqr.jax
          • probly.conformal_scores.uacqr.torch
            • probly.conformal_scores.uacqr.torch.compute_uacqr_score_func_torch
      • probly.datasets
        • probly.datasets.torch
          • probly.datasets.torch.Benthic
          • probly.datasets.torch.CIFAR10H
          • probly.datasets.torch.DCICDataset
          • probly.datasets.torch.ImageNetReaL
          • probly.datasets.torch.Plankton
          • probly.datasets.torch.QualityMRI
          • probly.datasets.torch.Treeversity1
          • probly.datasets.torch.Treeversity6
      • probly.decider
        • probly.decider.categorical_distribution
          • probly.decider.categorical_distribution.array
          • probly.decider.categorical_distribution.torch
      • probly.evaluation
        • probly.evaluation.active_learning
          • probly.evaluation.active_learning.loop
            • probly.evaluation.active_learning.loop.active_learning_loop
        • probly.evaluation.ood
          • probly.evaluation.ood.evaluate_ood
          • probly.evaluation.ood.out_of_distribution_detection_aupr
          • probly.evaluation.ood.out_of_distribution_detection_auroc
          • probly.evaluation.ood.out_of_distribution_detection_fnr_at_x_tpr
          • probly.evaluation.ood.out_of_distribution_detection_fpr_at_x_tpr
          • probly.evaluation.ood.parse_dynamic_metric
          • probly.evaluation.ood.visualize_ood
        • probly.evaluation.tasks
          • probly.evaluation.tasks.selective_prediction
      • probly.layers
        • probly.layers.flax
          • probly.layers.flax.BatchEnsembleConv
          • probly.layers.flax.BatchEnsembleLinear
          • probly.layers.flax.DropConnectLinear
        • probly.layers.torch
          • probly.layers.torch.BatchEnsembleConv2d
          • probly.layers.torch.BatchEnsembleLinear
          • probly.layers.torch.BayesConv2d
          • probly.layers.torch.BayesLinear
          • probly.layers.torch.DropConnectLinear
          • probly.layers.torch.HeteroscedasticLayer
          • probly.layers.torch.IRDHead
          • probly.layers.torch.IntBatchNorm1d
          • probly.layers.torch.IntBatchNorm2d
          • probly.layers.torch.IntConv2d
          • probly.layers.torch.IntLinear
          • probly.layers.torch.IntSoftmax
          • probly.layers.torch.NatPNClassHead
          • probly.layers.torch.NatPNRegHead
          • probly.layers.torch.NormalInverseGammaLinear
          • probly.layers.torch.RadialNormalizingFlow
          • probly.layers.torch.RadialNormalizingFlowStack
          • probly.layers.torch.RegressionHead
          • probly.layers.torch.SNGPLayer
          • probly.layers.torch.SpectralNormWithMultiplier
          • probly.layers.torch.pack_interval
          • probly.layers.torch.unpack_interval
      • probly.lazy_types
      • probly.method
        • probly.method.batchensemble
        • probly.method.bayesian
        • probly.method.calibration
          • probly.method.calibration.sklearn
            • probly.method.calibration.sklearn.SklearnIdentityLogitEstimator
            • probly.method.calibration.sklearn.SklearnVectorScalingPredictor
            • probly.method.calibration.sklearn.calibrate_sklearn_calibrated_classifier_cv
            • probly.method.calibration.sklearn.generate_sklearn_scaling_calibrator
          • probly.method.calibration.torch
            • probly.method.calibration.torch.TorchAffineLogitCalibrationPredictor
            • probly.method.calibration.torch.TorchIdentityLogitModel
            • probly.method.calibration.torch.TorchIsotonicCalibrationPredictor
            • probly.method.calibration.torch.generate_torch_scaling_calibrator
          • probly.method.calibration.sklearn_identity_logit_estimator
          • probly.method.calibration.torch_identity_logit_model
        • probly.method.cast
        • probly.method.conformal
          • probly.method.conformal.flax
            • probly.method.conformal.flax.FlaxConformalSetPredictor
          • probly.method.conformal.sklearn
            • probly.method.conformal.sklearn.SklearnConformalSetPredictor
          • probly.method.conformal.torch
            • probly.method.conformal.torch.TorchConformalSetPredictor
        • probly.method.credal_bnn
        • probly.method.credal_ensembling
        • probly.method.credal_net
          • probly.method.credal_net.torch
        • probly.method.credal_relative_likelihood
          • probly.method.credal_relative_likelihood.torch
        • probly.method.credal_wrapper
        • probly.method.dare
        • probly.method.ddu
        • probly.method.dropconnect
        • probly.method.dropout
        • probly.method.duq
        • probly.method.efficient_credal_prediction
        • probly.method.ensemble
        • probly.method.evidential
          • probly.method.evidential.classification
            • probly.method.evidential.classification.common
              • probly.method.evidential.classification.common.EvidentialClassificationPredictor
              • probly.method.evidential.classification.common.evidential_classification
              • probly.method.evidential.classification.common.register
            • probly.method.evidential.classification.torch
              • probly.method.evidential.classification.torch.append_activation_torch
          • probly.method.evidential.regression
            • probly.method.evidential.regression.common
              • probly.method.evidential.regression.common.evidential_regression
              • probly.method.evidential.regression.common.register
            • probly.method.evidential.regression.torch
              • probly.method.evidential.regression.torch.replace_last_torch_nig
        • probly.method.het_nets
          • probly.method.het_nets.torch
            • probly.method.het_nets.torch.drop_in_place_het_layer
            • probly.method.het_nets.torch.remove_layer
            • probly.method.het_nets.torch.skip_layer
        • probly.method.method
          • probly.method.method.PredictorTransformationMethod
          • probly.method.method.predictor_transformation
        • probly.method.natural_posterior_network
          • probly.method.natural_posterior_network.torch
            • probly.method.natural_posterior_network.torch.TorchNaturalPosteriorNetwork
        • probly.method.posterior_network
          • probly.method.posterior_network.torch
            • probly.method.posterior_network.torch.TorchPosteriorNetwork
        • probly.method.prior_network
          • probly.method.prior_network.torch
            • probly.method.prior_network.torch.append_activation_torch
        • probly.method.sngp
        • probly.method.subensemble
      • probly.metrics
        • probly.metrics.array
          • probly.metrics.array.auc_numpy
          • probly.metrics.array.average_precision_score_numpy
          • probly.metrics.array.precision_recall_curve_numpy
          • probly.metrics.array.roc_auc_score_numpy
          • probly.metrics.array.roc_curve_numpy
        • probly.metrics.jax
          • probly.metrics.jax.auc_jax
          • probly.metrics.jax.average_precision_score_jax
          • probly.metrics.jax.precision_recall_curve_jax
          • probly.metrics.jax.roc_auc_score_jax
          • probly.metrics.jax.roc_curve_jax
        • probly.metrics.torch
          • probly.metrics.torch.auc_torch
          • probly.metrics.torch.average_precision_score_torch
          • probly.metrics.torch.precision_recall_curve_torch
          • probly.metrics.torch.roc_auc_score_torch
          • probly.metrics.torch.roc_curve_torch
      • probly.plot
        • probly.plot.config
          • probly.plot.config.PlotConfig
        • probly.plot.credal
          • probly.plot.credal.plot
            • probly.plot.credal.plot.plot_credal_set
        • probly.plot.ood
          • probly.plot.ood.plot_histogram
          • probly.plot.ood.plot_pr_curve
          • probly.plot.ood.plot_roc_curve
        • probly.plot.utils
      • probly.predictor
        • probly.predictor.sklearn
          • probly.predictor.sklearn.predict_sklearn
      • probly.quantification
        • probly.quantification.decomposition
          • probly.quantification.decomposition.ddu
            • probly.quantification.decomposition.ddu.torch
              • probly.quantification.decomposition.ddu.torch.torch_negative_log_density
          • probly.quantification.decomposition.decomposition
            • probly.quantification.decomposition.decomposition.AdditiveDecomposition
            • probly.quantification.decomposition.decomposition.AleatoricDecomposition
            • probly.quantification.decomposition.decomposition.AleatoricEpistemicDecomposition
            • probly.quantification.decomposition.decomposition.AleatoricEpistemicTotalDecomposition
            • probly.quantification.decomposition.decomposition.AleatoricTotalDecomposition
            • probly.quantification.decomposition.decomposition.CachingDecomposition
            • probly.quantification.decomposition.decomposition.ConstantTotalDecomposition
            • probly.quantification.decomposition.decomposition.Decomposer
            • probly.quantification.decomposition.decomposition.Decomposition
            • probly.quantification.decomposition.decomposition.EpistemicDecomposition
            • probly.quantification.decomposition.decomposition.EpistemicTotalDecomposition
            • probly.quantification.decomposition.decomposition.TotalDecomposition
          • probly.quantification.decomposition.duq
            • probly.quantification.decomposition.duq.torch
              • probly.quantification.decomposition.duq.torch.torch_duq_uncertainty
          • probly.quantification.decomposition.entropy
          • probly.quantification.decomposition.zero_one
        • probly.quantification.measure
        • probly.quantification.notion
          • probly.quantification.notion.AleatoricUncertainty
          • probly.quantification.notion.EpistemicUncertainty
          • probly.quantification.notion.Notion
          • probly.quantification.notion.TotalUncertainty
      • probly.representation
        • probly.representation.array_like
          • probly.representation.array_like.ArrayFlagsLike
          • probly.representation.array_like.ArrayLike
          • probly.representation.array_like.NumpyArrayLike
          • probly.representation.array_like.NumpyArrayLikeConvertible
          • probly.representation.array_like.NumpyArrayLikeImplementation
        • probly.representation.conformal_set
          • probly.representation.conformal_set.array
            • probly.representation.conformal_set.array.ArrayIntervalConformalSet
            • probly.representation.conformal_set.array.ArrayOneHotConformalSet
          • probly.representation.conformal_set.torch
            • probly.representation.conformal_set.torch.TorchIntervalConformalSet
            • probly.representation.conformal_set.torch.TorchOneHotConformalSet
        • probly.representation.credal_set
          • probly.representation.credal_set.array
            • probly.representation.credal_set.array.ArrayCategoricalCredalSet
            • probly.representation.credal_set.array.ArrayConvexCredalSet
            • probly.representation.credal_set.array.ArrayDiscreteCredalSet
            • probly.representation.credal_set.array.ArrayDistanceBasedCredalSet
            • probly.representation.credal_set.array.ArrayProbabilityIntervalsCredalSet
            • probly.representation.credal_set.array.ArraySingletonCredalSet
          • probly.representation.credal_set.array_functions
          • probly.representation.credal_set.torch
            • probly.representation.credal_set.torch.TorchCategoricalCredalSet
            • probly.representation.credal_set.torch.TorchConvexCredalSet
            • probly.representation.credal_set.torch.TorchProbabilityIntervalsCredalSet
        • probly.representation.ddu
          • probly.representation.ddu.torch
            • probly.representation.ddu.torch.TorchDDURepresentation
        • probly.representation.distribution
          • probly.representation.distribution.array_categorical
            • probly.representation.distribution.array_categorical.ArrayCategoricalDistribution
            • probly.representation.distribution.array_categorical.ArrayCategoricalDistributionSample
          • probly.representation.distribution.array_dirichlet
            • probly.representation.distribution.array_dirichlet.ArrayDirichletDistribution
          • probly.representation.distribution.array_gaussian
            • probly.representation.distribution.array_gaussian.ArrayGaussianDistribution
            • probly.representation.distribution.array_gaussian.ArrayGaussianDistributionSample
          • probly.representation.distribution.torch_categorical
            • probly.representation.distribution.torch_categorical.TorchCategoricalDistribution
            • probly.representation.distribution.torch_categorical.TorchCategoricalDistributionSample
          • probly.representation.distribution.torch_dirichlet
            • probly.representation.distribution.torch_dirichlet.TorchDirichletDistribution
          • probly.representation.distribution.torch_gaussian
            • probly.representation.distribution.torch_gaussian.TorchGaussianDistribution
            • probly.representation.distribution.torch_gaussian.TorchGaussianDistributionSample
        • probly.representation.duq
          • probly.representation.duq.torch
            • probly.representation.duq.torch.TorchDUQRepresentation
        • probly.representation.het_nets
        • probly.representation.jax_like
          • probly.representation.jax_like.JaxLikeImplementation
        • probly.representation.representation
          • probly.representation.representation.Representation
        • probly.representation.sample
          • probly.representation.sample.array
            • probly.representation.sample.array.ArraySample
          • probly.representation.sample.array_functions
            • probly.representation.sample.array_functions.ArraySampleCreator
            • probly.representation.sample.array_functions.ArraySampleInternals
            • probly.representation.sample.array_functions.array_apply_along_axis_function
            • probly.representation.sample.array_functions.array_concatenate_function
            • probly.representation.sample.array_functions.array_copy_function
            • probly.representation.sample.array_functions.array_expand_dims_function
            • probly.representation.sample.array_functions.array_function_override
            • probly.representation.sample.array_functions.array_internals_override
            • probly.representation.sample.array_functions.array_matrix_transpose
            • probly.representation.sample.array_functions.array_reduction_function
            • probly.representation.sample.array_functions.array_reshape_function
            • probly.representation.sample.array_functions.array_sample_axis_preserving_function
            • probly.representation.sample.array_functions.array_sample_internals
            • probly.representation.sample.array_functions.array_squeeze_function
            • probly.representation.sample.array_functions.array_stack_function
            • probly.representation.sample.array_functions.array_swapaxes_function
            • probly.representation.sample.array_functions.array_transpose
            • probly.representation.sample.array_functions.track_sample_axis_after_reduction
          • probly.representation.sample.axis_tracking
            • probly.representation.sample.axis_tracking.ArrayIndex
            • probly.representation.sample.axis_tracking.AxisTrackingResult
            • probly.representation.sample.axis_tracking.track_axis
          • probly.representation.sample.jax
            • probly.representation.sample.jax.JaxArraySample
          • probly.representation.sample.jax_axis_tracking
          • probly.representation.sample.torch
            • probly.representation.sample.torch.TorchSample
          • probly.representation.sample.torch_axis_tracking
          • probly.representation.sample.torch_functions
            • probly.representation.sample.torch_functions.TorchSampleCreator
            • probly.representation.sample.torch_functions.TorchSampleInternals
            • probly.representation.sample.torch_functions.torch_adjoint_function
            • probly.representation.sample.torch_functions.torch_cat_function
            • probly.representation.sample.torch_functions.torch_function_override
            • probly.representation.sample.torch_functions.torch_internals_override
            • probly.representation.sample.torch_functions.torch_permute_function
            • probly.representation.sample.torch_functions.torch_reduction_function
            • probly.representation.sample.torch_functions.torch_sample_internals
            • probly.representation.sample.torch_functions.torch_stack_function
            • probly.representation.sample.torch_functions.torch_transpose_function
            • probly.representation.sample.torch_functions.track_sample_dim_after_reduction
        • probly.representation.torch_functions
          • probly.representation.torch_functions.torch_average
        • probly.representation.torch_like
          • probly.representation.torch_like.TorchLike
          • probly.representation.torch_like.TorchLikeConvertible
          • probly.representation.torch_like.TorchLikeImplementation
      • probly.representer
        • probly.representer.credal_ensembler
          • probly.representer.credal_ensembler.torch
            • probly.representer.credal_ensembler.torch.torch_compute_credal_ensembling_set
            • probly.representer.credal_ensembler.torch.torch_compute_credal_net_set
        • probly.representer.efficient_credal_prediction
          • probly.representer.efficient_credal_prediction.torch
        • probly.representer.het_nets
        • probly.representer.sampler
          • probly.representer.sampler.flax
            • probly.representer.sampler.flax.register_forced_train_mode
          • probly.representer.sampler.sklearn
            • probly.representer.sampler.sklearn.register_forced_fitted_already_mode
          • probly.representer.sampler.torch
            • probly.representer.sampler.torch.register_forced_train_mode
        • probly.representer.sngp
          • probly.representer.sngp.torch
            • probly.representer.sngp.torch.torch_compute_categorical_sample_from_logits
      • probly.train
        • probly.train.bayesian
          • probly.train.bayesian.torch
            • probly.train.bayesian.torch.ELBOLoss
            • probly.train.bayesian.torch.collect_kl_divergence
        • probly.train.calibration
          • probly.train.calibration.torch
            • probly.train.calibration.torch.ExpectedCalibrationError
            • probly.train.calibration.torch.FocalLoss
            • probly.train.calibration.torch.LabelRelaxationLoss
            • probly.train.calibration.torch.LabelSmoothingLoss
        • probly.train.credal
          • probly.train.credal.torch
            • probly.train.credal.torch.intersection_probability_ce_loss
        • probly.train.dare
          • probly.train.dare.torch
            • probly.train.dare.torch.dare_regularizer
        • probly.train.evidential
          • probly.train.evidential.common
          • probly.train.evidential.torch
            • probly.train.evidential.torch.der_loss
            • probly.train.evidential.torch.dirichlet_entropy
            • probly.train.evidential.torch.evidential_ce_loss
            • probly.train.evidential.torch.evidential_kl_divergence
            • probly.train.evidential.torch.evidential_log_loss
            • probly.train.evidential.torch.evidential_mse_loss
            • probly.train.evidential.torch.evidential_nignll_loss
            • probly.train.evidential.torch.evidential_regression_regularization
            • probly.train.evidential.torch.ird_loss
            • probly.train.evidential.torch.kl_dirichlet
            • probly.train.evidential.torch.lp_fn
            • probly.train.evidential.torch.make_in_domain_target_alpha
            • probly.train.evidential.torch.make_ood_target_alpha
            • probly.train.evidential.torch.natpn_loss
            • probly.train.evidential.torch.normal_wishart_log_prob
            • probly.train.evidential.torch.pn_loss
            • probly.train.evidential.torch.postnet_loss
            • probly.train.evidential.torch.predictive_probs
            • probly.train.evidential.torch.regularization_fn
            • probly.train.evidential.torch.rpn_distillation_loss
            • probly.train.evidential.torch.rpn_loss
            • probly.train.evidential.torch.rpn_ng_kl
            • probly.train.evidential.torch.rpn_prior
            • probly.train.evidential.torch.unified_evidential_train
      • probly.traverse_nn
        • probly.traverse_nn.flax
        • probly.traverse_nn.reset_traverser
        • probly.traverse_nn.torch
        • probly.traverse_nn.torch_utils
        • probly.traverse_nn.utils
          • probly.traverse_nn.utils.get_output_dim
      • probly.utils
        • probly.utils.iterable
          • probly.utils.iterable.first_element
        • probly.utils.quantile
          • probly.utils.quantile.jax
          • probly.utils.quantile.torch
        • probly.utils.switchdispatch
        • probly.utils.torch
          • probly.utils.torch.intersection_probability
          • probly.utils.torch.temperature_softmax
          • probly.utils.torch.torch_collect_outputs
          • probly.utils.torch.torch_entropy
          • probly.utils.torch.torch_reset_all_parameters
  • References and Further Reading
  • FAQ and Troubleshooting
GitHub Logo
Back to top

Quantification¶

Examples concerning the probly.quantification module.

Uncertainty Quantification

Uncertainty Quantification
Next
Uncertainty Quantification
Previous
A Brief Introduction to PyTraverse
Copyright © 2025, probly team
Made with Sphinx and @pradyunsg's Furo