Convert an Id to Token. Reverse lookup of self.vocab
(self, Id)
| 527 | raise NotImplementedError('TextTokenizer EncodeAsTokens not implemented') |
| 528 | |
| 529 | def IdToToken(self, Id): |
| 530 | """Convert an Id to Token. Reverse lookup of self.vocab""" |
| 531 | raise NotImplementedError('TextTokenizer IdToToken not implemented') |
| 532 | |
| 533 | def TokenToId(self, token): |
| 534 | """Convert a Token to Id. Lookup of self.vocab""" |
nothing calls this directly
no outgoing calls
no test coverage detected