MCPcopy Create free account
hub / github.com/aboutcode-org/scancode-toolkit / unicode_text_lines

Function unicode_text_lines

src/textcode/analysis.py:321–333  ·  view source on GitHub ↗

Yield unicode text lines from a file at ``location`` if it contains text. Open the file as binary then try to decode each line as Unicode. Remove verbatim, escaped CR, LF and tabs if ``decrlf`` is True.

(location, decrlf=False)

Source from the content-addressed store, hash-verified

319
320
321def unicode_text_lines(location, decrlf=False):
322 """
323 Yield unicode text lines from a file at ``location`` if it
324 contains text.
325
326 Open the file as binary then try to decode each line as Unicode.
327 Remove verbatim, escaped CR, LF and tabs if ``decrlf`` is True.
328 """
329 lines = _unicode_text_lines(location)
330 if decrlf:
331 return map(remove_verbatim_cr_lf_tab_chars, lines)
332 else:
333 return lines
334
335
336def _unicode_text_lines(location):

Calls 2

_unicode_text_linesFunction · 0.85
mapFunction · 0.85