Example lexer which will split input stream into words (respecting quotes) To emit SPACE tokens: self.spacetok = ('SPACE',None) To emit NL tokens: self.nltok = ('NL',None) >>> l = WordLexer(r'abc "def\100\x3d\. ghi" jkl') >>> list(l) [('ATOM
Source from the content-addressed store, hash-verified