Function basic_clean

wan/modules/tokenizers.py:12–15 · view source on GitHub ↗

(text)

Source from the content-addressed store, hash-verified

10
11
12	def basic_clean(text):
13	text = ftfy.fix_text(text)
14	text = html.unescape(html.unescape(text))
15	return text.strip()
16
17
18	def whitespace_clean(text):

_cleanMethod · 0.85

no outgoing calls

no test coverage detected