MCPcopy Create free account
hub / github.com/YunwenTechnology/QueryGeneration / convert_by_vocab

Function convert_by_vocab

tokenization/tokenization.py:123–128  ·  view source on GitHub ↗

Converts a sequence of [tokens|ids] using the vocab.

(vocab, items)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

Callers 4

convert_tokens_to_idsFunction · 0.85
convert_ids_to_tokensFunction · 0.85
convert_tokens_to_idsMethod · 0.85
convert_ids_to_tokensMethod · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected