TorchTokenGeneration¶
- class probly.representation.text_generation.torch.TorchTokenGeneration(sequences: Tensor, transition_scores: Tensor)[source]¶
Bases:
TorchAxisProtected[TorchTokenGeneration,TorchTokenGeneration,TorchTokenGeneration,Never,TorchTokenGeneration,Tensor,TorchTokenGeneration]Generated token sequences and token transition scores.
Shape:
batch_shape.sequencesstores token ids with a protected trailing sequence axis.transition_scoresstores generated-token log probabilities with a protected trailing generated-token axis.- property T: ShapeResultT¶
Inverts the order of the dimensions of the underlying array.
- clone(*, memory_format: torch.memory_format = torch.preserve_format) OwnerT[source]¶
Return a copy of the array.
- cpu(memory_format: memory_format = torch.preserve_format) Self[source]¶
Move the array to the CPU.
- cuda(device: device | str | None = None, non_blocking: bool = False, memory_format: memory_format = torch.preserve_format) Self[source]¶
Move the array to the GPU.
- gather(dim: int, index: torch.Tensor) ShapeResultT[source]¶
Return a copy with gathered protected values along a batch dimension.
- property mH: ShapeResultT¶
The adjoint (conjugate) transposed version of the underlying array.
- property mT: ShapeResultT¶
The transposed version of the underlying array.
- permitted_functions = {}¶
- permute(*dims: torch.Size | int | tuple[int] | list[int]) ShapeResultT[source]¶
Return a permuted version of the array.
- primary_protected_name = 'sequences'¶
- protected_values(func: Callable | None = None) dict[str, V] | None[source]¶
Read effective values as-is, optionally checking function permissions.
- reshape(*shape: int | tuple[int, ...]) ShapeResultT[source]¶
Return a copy with reshaped protected values.
- resolve_conj() OpResultT[source]¶
Return a version of the array with any conjugate operations resolved.
- resolve_neg() OpResultT[source]¶
Return a version of the array with any negation operations resolved.
- sequences: torch.Tensor¶
- property shape: tuple[int, ...]¶
Read the logical shape without collecting fields unless all broadcast.
- size(dim: int | None = None) int | Size[source]¶
Return the size of the array along the given dimension.
- to_device(device: device | str | None = None, /, *, stream: int | Any | None = None) Self[source]¶
Move the array to the given device.
- Parameters:
device – Target device, or None to retain the current device.
stream – Unsupported; must be None.
- Returns:
The moved array representation.
- Raises:
NotImplementedError – If a stream is provided.
- to_text(tokenizer: PreTrainedTokenizerBase, *, length_normalization: bool = False, stop_token_ids: set[int] | None = None, skip_special_tokens: bool = True, **decode_kwargs: Any) TorchTextGeneration[source]¶
Decode token sequences and sum transition scores into log-likelihoods.
- Parameters:
tokenizer – Tokenizer exposing
batch_decode.length_normalization – Whether log likelihoods should be averaged over scored tokens.
stop_token_ids – Explicit stop token ids. If omitted, tokenizer EOS and padding ids are used.
skip_special_tokens – Whether special tokens should be omitted while decoding.
**decode_kwargs – Additional keyword arguments forwarded to
batch_decode.
- Returns:
The decoded text generation representation.
- transition_scores: torch.Tensor¶