MCPcopy Create free account
hub / github.com/RT-Thread/env-windows / tokenize.py

File tokenize.py

tools/python-3.11.9-amd64/Lib/tokenize.py:None–None  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

1"""Tokenization help for Python programs.
2
3tokenize(readline) is a generator that breaks a stream of bytes into
4Python tokens. It decodes the bytes according to PEP-0263 for

Callers

nothing calls this directly

Calls 10

_all_string_prefixesFunction · 0.85
sortedFunction · 0.85
anyFunction · 0.70
maybeFunction · 0.70
groupFunction · 0.70
mainFunction · 0.70
mapFunction · 0.50
setFunction · 0.50
compileMethod · 0.45
addMethod · 0.45

Tested by

no test coverage detected