MCPcopy Create free account
hub / github.com/google-research/language / vocab_size

Method vocab_size

language/serene/preprocessing.py:630–631  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

628 raise NotImplementedError()
629
630 def vocab_size(self):
631 return len(self._tokenizer.vocab)
632
633 def save_to_file(self, filename_prefix):
634 vocab_file = f'{filename_prefix}.vocab'

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected