hub / github.com/cedrickchee/transformers-llama / BatchEncoding

Class BatchEncoding

src/transformers/tokenization_utils_base.py:163–761 · view source on GitHub ↗

Holds the output of the [`~tokenization_utils_base.PreTrainedTokenizerBase.__call__`], [`~tokenization_utils_base.PreTrainedTokenizerBase.encode_plus`] and [`~tokenization_utils_base.PreTrainedTokenizerBase.batch_encode_plus`] methods (tokens, attention_masks, etc). This class is d

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 15

test_batch_encoding_with_labelsMethod · 0.90

test_batch_encoding_with_labels_ptMethod · 0.90

test_batch_encoding_with_labels_tfMethod · 0.90

test_batch_encoding_with_labels_jaxMethod · 0.90

test_group_by_length_with_batch_encodingMethod · 0.90

test_decode_best_spansMethod · 0.90

_generative_stepMethod · 0.90

__call__Method · 0.90

_batch_encode_plusMethod · 0.85

_encode_plusMethod · 0.85

Calls

no outgoing calls

Tested by 6

test_batch_encoding_with_labelsMethod · 0.72

test_batch_encoding_with_labels_ptMethod · 0.72

test_batch_encoding_with_labels_tfMethod · 0.72

test_batch_encoding_with_labels_jaxMethod · 0.72

test_group_by_length_with_batch_encodingMethod · 0.72

test_decode_best_spansMethod · 0.72