Method ParseFromModelTokens

pkg/sql/parser/parser.go:330–333

ParseFromModelTokens parses tokenizer output ([]models.TokenWithSpan) directly into an AST. This is the preferred entry point for parsing SQL. It accepts the output of the tokenizer directly without any conversion step. Span information is preserved throughout parsing and is available for error rep

(tokens []models.TokenWithSpan)

// Issue #322: token_conversion.go has been removed; preprocessing is now a
// lightweight normalisation step that works entirely with models.TokenWithSpan.
func (p *Parser) ParseFromModelTokens(tokens []models.TokenWithSpan) (*ast.AST, error) {
	preprocessed := preprocessTokens(tokens)
	return p.parseTokens(preprocessed)
}

// ParseFromModelTokensWithPositions is identical to ParseFromModelTokens.
// Position information is embedded in every models.TokenWithSpan.