probly.evaluation.ood.parse_dynamic_metric

probly.evaluation.ood.parse_dynamic_metric(spec: str) → tuple[str, float]

Parse dynamic metric specification.

Parameters:

spec – Metric specification string (e.g., ‘fpr@0.8’, ‘fnr@95%’, ‘fpr’).

Returns:

• base: The base metric name (‘fpr’ or ‘fnr’).

• threshold: The threshold value. Defaults to 0.95 if not specified.

Return type:

A tuple containing

Raises:

ValueError – If specification is invalid.

Example

>>> parse_dynamic_metric('fpr@0.8')
('fpr', 0.8)
>>> parse_dynamic_metric('fnr@95%')
('fnr', 0.95)
>>> parse_dynamic_metric('fpr')
('fpr', 0.95)