Returns the vocabulary as a dict of {token: index} pairs. `tokenizer.get_vocab()[token]` is equivalent to `tokenizer.convert_tokens_to_ids(token)` when `token` is in the vocab.
(self)
| 172 | raise NotImplementedError |
| 173 | |
| 174 | def get_vocab(self): |
| 175 | """ Returns the vocabulary as a dict of {token: index} pairs. `tokenizer.get_vocab()[token]` is equivalent to `tokenizer.convert_tokens_to_ids(token)` when `token` is in the vocab. """ |
| 176 | raise NotImplementedError() |
| 177 | |
| 178 | def get_added_vocab(self) -> Dict[str, int]: |
| 179 | return self.added_tokens_encoder |
nothing calls this directly
no outgoing calls
no test coverage detected