Method start_token

crates/sith_python_parser/src/lexer/cursor.rs:82–84

Mark the current position of the cursor as the start of the token which is going to be lexed. Use [`Cursor::token_len`] to get the length of the lexed token.

(&mut self)

    ///
    /// Use [`Cursor::token_len`] to get the length of the lexed token.
    pub(super) fn start_token(&mut self) {
        self.source_length = self.text_len();
    }

    /// Returns `true` if the cursor is at the end of file.
    pub(super) fn is_eof(&self) -> bool {