MCPcopy Create free account
hub / github.com/aboutcode-org/scancode-toolkit / get_demarkuped_text

Function get_demarkuped_text

src/textcode/markup.py:153–176  ·  view source on GitHub ↗

Return text stripped from markup. Use ``splitter`` callable to split text in tokens. Use ``keeper`` is a callable to tell if we should keep a tag.

(text, splitter, keeper)

Source from the content-addressed store, hash-verified

151
152
153def get_demarkuped_text(text, splitter, keeper):
154 """
155 Return text stripped from markup.
156 Use ``splitter`` callable to split text in tokens.
157 Use ``keeper`` is a callable to tell if we should keep a tag.
158 """
159
160 tokens = splitter(text)
161 if TRACE:
162 logger_debug(f"get_demarkuped_text: {text!r}")
163 logger_debug(f"get_demarkuped_text: tokens: {tokens}")
164
165 demarkuped = []
166 demarkuped_append = demarkuped.append
167
168 for token in tokens:
169 if keeper(token):
170 if token == ">":
171 token = "> "
172 demarkuped_append(token)
173 else:
174 demarkuped_append(" ")
175
176 return " ".join("".join(demarkuped).split())
177
178
179def is_debian_tag(t):

Callers 2

demarkup_textFunction · 0.85

Calls 3

logger_debugFunction · 0.70
joinMethod · 0.45
splitMethod · 0.45

Tested by

no test coverage detected