Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/SooLab/CGFormer
/ get_vocab
Method
get_vocab
bert/tokenization_bert.py:202–203 ·
view source on GitHub ↗
(self)
Source
from the content-addressed store, hash-verified
200
return
len(self.vocab)
201
202
def
get_vocab(self):
203
return
dict(self.vocab, **self.added_tokens_encoder)
204
205
def
_tokenize(self, text):
206
split_tokens = []
Callers
nothing calls this directly
Calls
no outgoing calls
Tested by
no test coverage detected