MCPcopy Create free account
hub / github.com/THUDM/GLM / _encode

Method _encode

data_utils/tokenization.py:396–397  ·  view source on GitHub ↗
(self, text)

Source from the content-addressed store, hash-verified

394 return tokenization
395
396 def _encode(self, text):
397 raise NotImplementedError
398
399 def EncodeAsTokens(self, text, process_fn=None):
400 """

Callers 1

split_on_tokensMethod · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected