Check analyze cases against a real ClickHouse in goldeneye, and make sqlc agree with it - #4603
Open
kyleconroy wants to merge 10 commits into
Open
Check analyze cases against a real ClickHouse in goldeneye, and make sqlc agree with it#4603kyleconroy wants to merge 10 commits into
kyleconroy wants to merge 10 commits into
Conversation
Add a nested, dependency-free Go module at internal/engine/clickhouse/testgen that records what ClickHouse itself reports about a schema, fixture and sqlc query file, in the same JSON shape as `sqlc analyze`, so the two can be diffed. `testgen install` downloads the pinned clickhouse release for the running platform into the user cache directory. `testgen analyze` runs each query in its own `clickhouse local` process: result column types and nullability come from the executed query's result header, provenance from EXPLAIN QUERY TREE (followed through subqueries, CTEs and unions), and parameters from ordinal-carrying sentinel constants substituted for ?, sqlc.arg() and sqlc.narg(), or from DESCRIBE TABLE for INSERT ... VALUES. Golden tests under testdata/ cover the type lowering, expressions, subqueries and exec statements. The analyze_params case reproduces the existing sqlc analyze golden byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
Replace the data_type and is_array fields with one type expression per column: a lowercased name applied to arguments that are numbers, quoted strings, identifiers, other calls, or any of those with a label. Nullable, Array and LowCardinality are ordinary names in that grammar, so nested types such as Array(Nullable(String)), Map(String, Nullable(UInt8)) and Tuple(lat Float64, lon Float64) survive intact. An outer Nullable is lifted into the column's not_null flag; deeper ones stay in the expression. Resolving the names is left to the reader of the output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
A column's type is now a JSON expression: a lowercased name applied to arguments, each carrying an optional label and exactly one of type, int or string. The shape maps one to one onto a protobuf message with a oneof for the argument value. There is no separate nullability flag any more: a nullable column is one whose type is nullable(...), which keeps Nullable at every depth where ClickHouse put it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
Nullable(T) is now T with nullable set, at whatever depth ClickHouse wrote it, instead of a call named nullable. Every engine has nullability and every consumer needs it, so an attribute on the type node spares readers from treating one name as special. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
releaseAsset now looks a build up in a table of version, platform, file name and SHA-512, and Install hashes every byte off the wire, including the tail of a tarball past the binary, and discards a download whose digest does not match. A version missing from the table cannot be installed. The tarball digests are the ones ClickHouse publishes in its .sha512 sidecar files; the macOS binaries have no published digest, so theirs were computed from the downloads. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
… cases testcheck replaces testgen. It generates nothing: each engine package finds the analyze cases under internal/endtoend/testdata, loads a case's schema, fixture and queries into a real database, and compares what the database reports with the output.json the case committed, byte for byte. The ClickHouse package is the testgen code moved over; other engines get their own package alongside it. sqlc analyze now prints each column's type as a call expression instead of data_type, not_null and is_array, so its output and the database's answer share one format. The compiler's flat column description maps onto it as the data type wrapped in one array node per dimension with the column's nullability on the outermost node. Every analyze case's expected output is renamed from stdout.txt to output.json, which the end-to-end harness now reads first, and regenerated. The cases from testgen's testdata become analyze_types, analyze_expressions, analyze_subqueries and analyze_exec under the ClickHouse dialect, with fixture.sql next to the schema, and the existing analyze_basic and analyze_params ClickHouse cases gain fixtures. Two queries from the subqueries case, a CTE and a SELECT * over a join, are left out because sqlc cannot analyze them yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
testcheck showed sqlc disagreeing with ClickHouse on every analyze case that went beyond plain column references. This makes the six ClickHouse cases match the database byte for byte, and moves the testcheck command under cmd/testcheck. Types are carried as expressions. The ClickHouse converter keeps a column's full spelling in the type name's Spelling, the schema stores it as the attribute's declared type, and the analysis core writes each column and parameter a TypeExpr from it, so Array(Nullable(String)), Map(String, Nullable(UInt8)), Tuple(lat Float64, lon Float64) and Decimal(10, 2) survive intact. The analyze command prints that expression when the core produced one. Functions are typed. A ClickHouse function seed of 581 signatures replaces the single count() entry, with "$n" naming the type of the nth argument and never_null marking results that stay non-null. The dialect declares that functions propagate nullability, that comparisons yield UInt8 while true is Bool, that LIMIT counts are UInt64, and that an unconstrained placeholder is Nothing. The analyzer scores overloads, types ORDER BY, LIMIT and OFFSET, follows IN (SELECT ...) into the subquery, makes COALESCE null only when every argument is, and names a placeholder compared with a function call after the function. The converter gives unaliased expression columns ClickHouse's own names, such as sum(amount) and plus(id, 1), captures aliases on every node kind, converts a scalar subquery as a value rather than EXISTS, converts coalesce and ifNull to COALESCE, handles WITH elements as the parser produces them, and counts positions from zero so star expansion and parameter renumbering line up. ClickHouse joins the preprocessed engines, so sqlc.arg() and sqlc.narg() work with ? binding. A dialect that names the second id of a join e.id says so in its seed and the analyzer qualifies such columns. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
goldeneye already generates and checks the dialect seeds and reserved a place for the analysis checks, so the testcheck module folds into it: the case loader becomes goldeneye/endtoend, reusing goldeneye's diff, the ClickHouse analysis joins goldeneye's clickhouse package next to the dialect generator it shares a binary with, and `check` verifies both the committed dialect and the analyze cases. `go test ./...` in internal/goldeneye runs the same checks. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps
kyleconroy
force-pushed
the
claude/clickhouse-testgen-module-rjzyuf
branch
from
September 3, 2026 17:56
78d7394 to
2599966
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends
internal/goldeneyewith the analysis checks its README reserved a place for, and then fixes everything they found wrong in sqlc's ClickHouse analysis. All six ClickHouse analyze cases now match what ClickHouse itself reports, byte for byte.goldeneye checkverifies, for each engine with a database available, both the committed dialect and everyanalyze_*/<engine>case underinternal/endtoend/testdata: it loads the case'sschema.sqland optionalfixture.sqlinto the database, runsquery.sqlthere, prints what the database reports in the JSON shapesqlc analyzeprints, and compares it with the case's committedoutput.json.go test ./...in the module runs the same checks.cd internal/goldeneye go run ./cmd/goldeneye install clickhouse go run ./cmd/goldeneye check clickhousegoldeneye/endtoendfinds the cases and compares an engine's answer with the committed output, reusing goldeneye's diff.goldeneye/clickhousegainsAnalyzeandChecknext to the dialect generator it shares a binary with. Each case runs in an ephemeralclickhouse localprocess: column types come from the executed query's result header, provenance fromEXPLAIN QUERY TREE, and parameters, which ClickHouse never sees, from ordinal-carrying sentinel constants the query tree describes by the operand they are compared with.INSERT ... VALUESparameters map ontoDESCRIBE TABLE.sqlc analyzeoutputsqlc analyzeprints each column's type as a call expression instead ofdata_type,not_nullandis_array: anameapplied toargs, each carrying an optionallabeland exactly one oftype,int,boolorstring, withnullableset at whatever depth it applies.docs/howto/analyze.mdis updated.{"name": "map", "args": [ {"type": {"name": "string"}}, {"type": {"name": "uint8", "nullable": true}}]}Every analyze case's expected output is renamed from
stdout.txttooutput.json, which the harness now reads first, and regenerated. New ClickHouse casesanalyze_types,analyze_expressions,analyze_subqueriesandanalyze_execcome with afixture.sqlnext to the schema, andanalyze_basicandanalyze_paramsgain fixtures.What the check found, and the fixes
Spelling, the schema stores it as the attribute's declared type, and the analysis core writes aTypeExprfor every column and parameter from it.Array(Nullable(String)),Array(Array(UInt8)),LowCardinality(Nullable(String)),Map(String, Nullable(UInt8)),Tuple(lat Float64, lon Float64),Enum8('active' = 1, ...),Decimal(10, 2),DateTime64(3, 'UTC')andFixedString(4)all survive.count()entry, with"$n"naming the type of the nth argument andnever_nullfor results that stay non-null.dialect.jsondeclares that functions propagate nullability, that comparisons yieldUInt8whiletrueisBool, thatLIMITcounts areUInt64, that an unconstrained placeholder isNothing, and that the secondidof a join is namede.id. The analyzer scores overloads instead of taking the first of the right arity, andCOALESCEis now null only when every argument is.sqlc.arg()andsqlc.narg()work with?binding; the analyzer typesORDER BY,LIMITandOFFSET, followsIN (SELECT ...)into the subquery, and names a placeholder compared with a function call after the function. The converter's positions are now zero-based, which is also what made star expansion work.?column?typedbool. The converter treats a subquery in value position as a value rather thanEXISTS, and captures aliases on every expression node kind.Beyond the four: unaliased expression columns get ClickHouse's own names (
sum(amount),plus(id, 1)),WITHelements are converted as the parser produces them so CTEs resolve, andcoalesce/ifNullconvert toCOALESCE.Tests
go test ./...at the root passes apart from the database-backed schema tests that need MySQL. The ClickHouse parse golden changed by one position and one column name, both correct. Ininternal/goldeneye,check clickhousereports the dialect and all six analyze cases matching.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y6XkyWnx7iJFEb8q3AnYps