MCPcopy Create free account
hub / github.com/THUDM/GLM / __init__

Method __init__

data_utils/tokenization.py:68–78  ·  view source on GitHub ↗
(self, tokenization, text=None, original_text=None, command_tokens=None, asIds=True)

Source from the content-addressed store, hash-verified

66 """
67
68 def __init__(self, tokenization, text=None, original_text=None, command_tokens=None, asIds=True):
69 self.tokenization = tokenization
70 self.text = text
71 if self.text is None:
72 self.text = self.tokenization
73 self.original_text = original_text
74 if self.original_text is None:
75 self.original_text = self.text
76 self.command_tokens = command_tokens
77 self.asIds = asIds
78 self.parse_command_tokens()
79
80 def set_command_tokens(self, command_tokens):
81 self.command_tokens = command_tokens

Callers

nothing calls this directly

Calls 1

parse_command_tokensMethod · 0.95

Tested by

no test coverage detected