MCPcopy Index your code
hub / github.com/zai-org/CodeGeeX / map_token

Method map_token

codegeex/mindspore/src/code_tokenizer.py:118–121  ·  view source on GitHub ↗
(self, token: Union[int, str])

Source from the content-addressed store, hash-verified

116 return self._symbols[idx]
117
118 def map_token(self, token: Union[int, str]):
119 if isinstance(token, int):
120 token = str(token)
121 return self.index(token)
122
123 def map_tokens(self, tokens):
124 return [self.map_token(token) for token in tokens]

Callers 1

map_tokensMethod · 0.95

Calls 1

indexMethod · 0.95

Tested by

no test coverage detected