MCPcopy Create free account
hub / github.com/SooLab/CGFormer / get_vocab

Method get_vocab

bert/tokenization_bert.py:202–203  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

200 return len(self.vocab)
201
202 def get_vocab(self):
203 return dict(self.vocab, **self.added_tokens_encoder)
204
205 def _tokenize(self, text):
206 split_tokens = []

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected