TorchTokenGeneration

class probly.representation.text_generation.torch.TorchTokenGeneration(sequences: Tensor, transition_scores: Tensor)

Bases: TorchAxisProtected[Tensor]

Generated token sequences and token transition scores.

Shape: batch_shape. sequences stores token ids with a protected trailing sequence axis. transition_scores stores generated-token log probabilities with a protected trailing generated-token axis.

property T: Self

Inverts the order of the dimensions of the underlying array.

clone(*, memory_format: memory_format = torch.preserve_format) → Self

Return a copy of the array.

cpu(memory_format: memory_format = torch.preserve_format) → Self

Move the array to the CPU.

cuda(device: device | str | None = None, non_blocking: bool = False, memory_format: memory_format = torch.preserve_format) → Self

Move the array to the GPU.

detach() → Self

Return a detached version of the array.

property device: device

Device of the array.

property dtype: dtype

Data type of the array.

gather(dim: int, index: Tensor) → Self

Return a copy with gathered protected values along a batch dimension.

property mH: Self

The adjoint (conjugate) transposed version of the underlying array.

property mT: Self

The transposed version of the underlying array.

property ndim: int

Number of dimensions.

numpy(*, force: bool = False) → ndarray

Convert to a numpy array.

permitted_functions = {}

permute(*dims: Size | int | tuple[int] | list[int]) → Self

Return a permuted version of the array.

classmethod primary_protected_name() → str

Return the first protected field (dict order).

protected_axes: ClassVar[dict[str, int]] = {'sequences': 1, 'transition_scores': 1}

property protected_shape: tuple[int, ...]

Protected trailing shape of the primary field.

protected_value() → TorchProtectedValue

Return the primary protected value.

protected_values(func: Callable | None = None) → dict[str, TorchProtectedValue] | None

Return all protected field values as-is.

Optionally takes the torch function that triggered the call for context. This can be used to conditionally modify the returned values or prevent them from being accessed.

reshape(*shape: int | tuple[int, ...]) → Self

Return a copy with reshaped protected values.

resolve_conj() → Self

Return a version of the array with any conjugate operations resolved.

resolve_neg() → Self

Return a version of the array with any negation operations resolved.

sequences: torch.Tensor

property shape: tuple[int, ...]

Shape of the array.

size(dim: int | None = None) → int | Size

Return the size of the array along the given dimension.

to(*args: Any, **kwargs: Any) → Self

Move and/or cast the tensor, mirroring torch.Tensor.to.

to_device(device: Literal['cpu'], /, *, stream: int | Any | None = None) → Self

Move the array to a device.

to_text(tokenizer: PreTrainedTokenizerBase, *, length_normalization: bool = False, stop_token_ids: set[int] | None = None, skip_special_tokens: bool = True, **decode_kwargs: Any) → TorchTextGeneration

Decode token sequences and sum transition scores into log-likelihoods.

Parameters:

• tokenizer – Tokenizer exposing batch_decode.

• length_normalization – Whether log likelihoods should be averaged over scored tokens.

• stop_token_ids – Explicit stop token ids. If omitted, tokenizer EOS and padding ids are used.

• skip_special_tokens – Whether special tokens should be omitted while decoding.

• **decode_kwargs – Additional keyword arguments forwarded to batch_decode.

Returns:

The decoded text generation representation.

transition_scores: torch.Tensor

transpose(dim0: int, dim1: int) → Self

Return a transposed version of the array.

with_protected_values(values: dict[str, TorchProtectedValue], func: Callable | None = None) → TorchAxisProtected[T]

Return a copy with updated protected field values.