Function whitespace_clean

wan/modules/tokenizers.py:18–21 · view source on GitHub ↗

(text)

Source from the content-addressed store, hash-verified

16
17
18	def whitespace_clean(text):
19	text = re.sub(r'\s+', ' ', text)
20	text = text.strip()
21	return text
22
23
24	def canonicalize(text, keep_punctuation_exact_string=None):

_cleanMethod · 0.85

no outgoing calls

no test coverage detected