MCPcopy Create free account
hub / github.com/InternScience/SciReason / normalize

Function normalize

opencompass/datasets/drop_simple_eval.py:18–25  ·  view source on GitHub ↗

Lower text and remove punctuation, articles and extra whitespace.

(s: str)

Source from the content-addressed store, hash-verified

16
17
18def normalize(s: str) -> str:
19 """Lower text and remove punctuation, articles and extra whitespace."""
20 s = s.lower()
21 exclude = set(string.punctuation)
22 s = ''.join(char for char in s if char not in exclude)
23 s = re.sub(r'\b(a|an|the)\b', ' ', s)
24 s = ' '.join(s.split())
25 return s
26
27
28def fuzzy_match(s1: str, s2: str) -> bool:

Callers 2

fuzzy_matchFunction · 0.70
scoreMethod · 0.70

Calls 1

lowerMethod · 0.45

Tested by

no test coverage detected