MCPcopy Index your code
hub / github.com/OpenMOSS/MOSS / truncate

Method truncate

models/tokenization_moss.py:354–380  ·  view source on GitHub ↗
(self, completion, truncate_before_pattern)

Source from the content-addressed store, hash-verified

352 return decoded_text
353
354 def truncate(self, completion, truncate_before_pattern):
355 def find_re(string, pattern, start_pos):
356 m = pattern.search(string, start_pos)
357 return m.start() if m else -1
358
359 terminals = [re.compile(pattern, re.MULTILINE) for pattern in truncate_before_pattern]
360
361 prints = list(re.finditer("^print", completion, re.MULTILINE))
362
363 if len(prints) > 1:
364 completion = completion[: prints[1].start()]
365
366 defs = list(re.finditer("^def", completion, re.MULTILINE))
367
368 if len(defs) > 1:
369 completion = completion[: defs[1].start()]
370
371 start_pos = 0
372
373 terminals_pos = [
374 pos for pos in [find_re(completion, terminal, start_pos) for terminal in terminals] if pos != -1
375 ]
376
377 if len(terminals_pos) > 0:
378 return completion[: min(terminals_pos)]
379 else:
380 return completion

Callers 1

decodeMethod · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected