MCPcopy Create free account
hub / github.com/google-research/language / tokens

Method tokens

language/serene/preprocessing.py:280–281  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

278
279 @property
280 def tokens(self):
281 return list(self._vocab_list)
282
283 @property
284 def oov_token(self):

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected