Returns true if token is a delimiter. When the tokenizer is constructed with the RETURN_DELIMS option, this method can be used to check if the returned token is actually a delimiter.
| 141 | // with the RETURN_DELIMS option, this method can be used to check if the |
| 142 | // returned token is actually a delimiter. |
| 143 | bool token_is_delim() const { return token_is_delim_; } |
| 144 | |
| 145 | // If GetNext() returned true, then these methods may be used to read the |
| 146 | // value of the token. |