Method markTokensImpl

An abstract method that splits a line up into tokens. It should parse the line, and call addToken() to add syntax tokens to the token list. Then, it should return the initial token type for the next line. For example if the current line contains the start of a multiline comment that

(byte token, Segment line,
                int lineIndex)