Function _normalize

utils/wiki_util.py:7–11 · view source on GitHub ↗

Normalize Unicode strings. Necessary for text which contains non-ASCII characters.

(text)

Source from the content-addressed store, hash-verified

5
6
7	def _normalize(text):
8	"""
9	Normalize Unicode strings. Necessary for text which contains non-ASCII characters.
10	"""
11	return unicodedata.normalize("NFD", text)
12
13
14	def convert_html(text):

load_dpr_wikiFunction · 0.90

normalize_answerFunction · 0.90

has_correct_answerFunction · 0.90

process_wikiFunction · 0.90

get_hyperlinkFunction · 0.85

remove_hyperlinkFunction · 0.85

no outgoing calls

no test coverage detected