feat(parser): support MySQL # line comments behind allowHashLineComments

[fudianchn]

AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.

What

MySQL # line comments behind a feature switch, the second step agreed in #2502 (fixes #2499):

CCJSqlParserUtil.parse("SELECT 42 # 24", p -> p.withHashLineComments(true))
// -> SELECT 42

SELECT 1 #comment, 2        -- -> SELECT 1   (no blank needed, like MySQL)
SELECT 42#24                -- -> SELECT 42  (mid-token, like MySQL)
SELECT 1 # c
FROM t                      -- -> SELECT 1 FROM t (statement continues on the next line)

Why / Root cause

One lexeme, two dialects: PostgreSQL reads # as the binary operator from #2507, MySQL as a line comment. The special-token-or-token identity is static per production, and longest match outranks any post-match rewrite (#word lexes as an identifier before an operator or comment rule could act), so a static comment rule would silently drop one of the readings — the "no good solution" from the #2502 discussion.

How

Three pieces, all inert while Feature.allowHashLineComments is off (the default):

1. Under the flag, SimpleCharStream rewrites a token-start # in its buffer to a character no other lexical rule starts with. Rewriting the buffer (instead of synthesizing reads) keeps the matcher's backup / re-read arithmetic intact, and GetImage() restores the #, so the comment token carries the original text.
2. The dedicated HASH_LINE_COMMENT production claims that character, self-contained like LINE_COMMENT, and emits the comment as a special token. Because nothing else starts with the character, it wins the match for every # form (# c, #c, #>, #-, ...).
3. Unquoted identifiers and @@variables end at their first # via their token actions, which back up and re-lex the remainder as the comment — real MySQL reads 42#24 as 42 plus comment too.

Quoted forms ('#', "a#b", `#`) keep their # in both modes. Under the flag the statement semantics are MySQL's: SELECT #temp FROM t comments out the rest of the line and fails, quoted "#temp" still parses.

Follow-up: with this landing there are now three lexer-level switches (square brackets, backslash, # comments). The dialect presets sketched in the #2502 discussion would be a small standalone follow-up if you want them — a mapping from DatabaseType to the existing Feature set, no mechanism changes.

Testing

CCJSqlParserUtilTest: 2 new tests (both states of the same SQL, MySQL statement semantics). SelectASTTest: 1 new test (the comment is a special token carrying the original # ... image). All verified failing under three mutants: stream rewrite removed, identifier truncation removed, # dropped from the identifier start set. Full suite green (4935 tests).

Performance

gradle jmh, JSQLParserBenchmark.parseSQLStatements on performance.sql, version=latest, 10 forks × 10 iterations (100 samples) on a 32-core host, interleaved master/branch (one polluted run excluded, CI ± 0.15):

build	run 1	run 2	run 3
master dff722b	3.770 ± 0.022	3.763 ± 0.023	3.783 ± 0.024
branch (this PR)	3.786 ± 0.027	3.800 ± 0.026	3.801 ± 0.023

Δ ≤ 1% with overlapping CIs in 2 of 3 windows; the residual is the three added per-token branches (token start, image, identifier action), each guarded to a null-check for parses that never opt in.

[manticore-projects]

This is very smart, to put this here! It totally makes sense, but I never thought of it!

[manticore-projects]

Very smart solution! Just add more Warning to the header of SimpleCharStream please so this will never get overwritten accidentally. Then I will merge.
Thank you much already for going the extra round!

[fudianchn]

@manticore-projects Added the warning to the header of SimpleCharStream.

[manticore-projects]

Under the flag, SimpleCharStream rewrites a token-start # in its buffer to a character no other lexical rule starts with. Rewriting the buffer (instead of synthesizing reads) keeps the matcher's backup / re-read arithmetic intact, and GetImage() restores the #, so the comment token carries the original text.

It really bothers me, that I did not think of this myself. So simple, so elegant. Kudos!