MCPcopy
hub / github.com/fluentpython/example-code-2e / tokenize

Function tokenize

18-with-match/lispy/original/lis.py:64–66  ·  view source on GitHub ↗

Convert a string into a list of tokens.

(s)

Source from the content-addressed store, hash-verified

62 return read_from_tokens(tokenize(program))
63
64def tokenize(s):
65 "Convert a string into a list of tokens."
66 return s.replace('(',' ( ').replace(')',' ) ').split()
67
68def read_from_tokens(tokens):
69 "Read an expression from a sequence of tokens."

Callers 1

parseFunction · 0.70

Calls 1

replaceMethod · 0.45

Tested by

no test coverage detected