MCPcopy Create free account
hub / github.com/THUDM/GLM / count_tokens

Function count_tokens

tasks/seq2seq/evaluate.py:118–125  ·  view source on GitHub ↗
(tokens)

Source from the content-addressed store, hash-verified

116
117
118def count_tokens(tokens):
119 counter = {}
120 for t in tokens:
121 if t in counter.keys():
122 counter[t] += 1
123 else:
124 counter[t] = 1
125 return counter
126
127
128def get_f1(text_a, text_b):

Callers 1

get_f1Function · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected