MCPcopy Create free account
hub / github.com/aboutcode-org/scancode-toolkit / get_tokens

Function get_tokens

src/packagedcode/bashparse.py:370–376  ·  view source on GitHub ↗

Return a Tokens list from ``text``.

(text)

Source from the content-addressed store, hash-verified

368
369
370def get_tokens(text):
371 """
372 Return a Tokens list from ``text``.
373 """
374 lexer = BashShellLexer()
375 pygtokens = lexer.get_tokens_unprocessed(text)
376 return list(Token.from_pygments_tokens(pygtokens))
377
378
379def parse_shell(text, grammar=variables_grammar, loop=1, trace=TRACE, validate=VALIDATE):

Callers 1

parse_shellFunction · 0.70

Calls 1

BashShellLexerClass · 0.90

Tested by

no test coverage detected