MCPcopy Create free account
hub / github.com/THUDM/GLM / __call__

Method __call__

data_utils/tokenization.py:490–491  ·  view source on GitHub ↗
(self, text, process_fn=None)

Source from the content-addressed store, hash-verified

488 self.num_tokens = self.num_text_tokens
489
490 def __call__(self, text, process_fn=None):
491 return self.EncodeAsIds(text, process_fn)
492
493 def __len__(self):
494 return self.num_text_tokens

Callers

nothing calls this directly

Calls 1

EncodeAsIdsMethod · 0.95

Tested by

no test coverage detected