Skip to content

ast: tag node types and omit absent fields in JSON output #4591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kyleconroy wants to merge 2 commits into from
Closed

ast: tag node types and omit absent fields in JSON output #4591

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
04dba04
ast: tag every node with its type in JSON output
claude Aug 27, 2026
246970c
ast: omit absent fields from JSON output
claude Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/howto/analyze.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,6 @@ reports the result columns and parameters:
]
```

Pass `--ast` to also include each statement's parsed AST under an `ast` key.
Pass `--ast` to also include each statement's parsed AST under an `ast` key. It
has the same shape as the output of [`parse`](parse.md), with every node tagged
by type.
47 changes: 47 additions & 0 deletions docs/howto/parse.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ The output is a JSON array with one object per statement:
"name": "GetAuthor",
"cmd": ":one",
"ast": {
"tag": "RawStmt",
"Stmt": {
"...": "..."
},
Expand All @@ -60,3 +61,49 @@ The output is a JSON array with one object per statement:

Statements without a `-- name:` annotation (for example schema DDL) omit the
`name` and `cmd` fields.

## Node types

Every node in the AST carries a `tag` naming its type. Some nodes have no
fields of their own, so without it a star, a null literal and an untranslated
clause would all print as `{}`.

```json
"Val": {
"tag": "ColumnRef",
"Name": "",
"Fields": {
"tag": "List",
"Items": [
{
"tag": "A_Star"
}
]
},
"Location": 93
}
```

A `tag` of `TODO` marks a clause the dialect's converter does not translate
yet. It means the clause was parsed but is not represented in the AST, not that
the clause was absent from the query.

## Absent fields

A field the statement does not use is left out rather than printed as `null`.
An `A_Const` carrying an integer reports only what it has:

```json
"Val": {
"tag": "A_Const",
"Val": {
"tag": "Integer",
"Ival": 1
},
"Location": 30
}
```

Only absent fields are omitted. A zero keeps its place, because zero is a value
the parser can find: `StmtLocation` is 0 for the first statement in a file, and
`LIMIT 0` parses to an `Ival` of 0.
52 changes: 33 additions & 19 deletions internal/endtoend/testdata/analyze_ast/postgresql/stdout.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,28 @@
}
],
"ast": {
"tag": "RawStmt",
"Stmt": {
"tag": "SelectStmt",
"DistinctClause": {
"Items": null
"tag": "List"
},
"IntoClause": null,
"TargetList": {
"tag": "List",
"Items": [
{
"Name": null,
"tag": "ResTarget",
"Indirection": {
"Items": null
"tag": "List"
},
"Val": {
"tag": "ColumnRef",
"Name": "",
"Fields": {
"tag": "List",
"Items": [
{
"tag": "String",
"Str": "name"
}
]
Expand All @@ -52,68 +57,77 @@
]
},
"FromClause": {
"tag": "List",
"Items": [
{
"Catalogname": null,
"Schemaname": null,
"tag": "RangeVar",
"Relname": "authors",
"Inh": true,
"Relpersistence": 112,
"Alias": null,
"Location": 45
}
]
},
"WhereClause": {
"tag": "A_Expr",
"Kind": 1,
"Name": {
"tag": "List",
"Items": [
{
"tag": "String",
"Str": "="
}
]
},
"Lexpr": {
"tag": "ColumnRef",
"Name": "",
"Fields": {
"tag": "List",
"Items": [
{
"tag": "String",
"Str": "id"
}
]
},
"Location": 59
},
"Rexpr": {
"tag": "ParamRef",
"Number": 1,
"Location": 64,
"Dollar": true
},
"Location": 62
},
"GroupClause": {
"Items": null
"tag": "List"
},
"HavingClause": {
"tag": "TODO"
},
"HavingClause": {},
"WindowClause": {
"Items": null
"tag": "List"
},
"ValuesLists": {
"Items": null
"tag": "List"
},
"SortClause": {
"Items": null
"tag": "List"
},
"LimitOffset": {
"tag": "TODO"
},
"LimitCount": {
"tag": "TODO"
},
"LimitOffset": {},
"LimitCount": {},
"LockingClause": {
"Items": null
"tag": "List"
},
"WithClause": null,
"Op": 0,
"All": false,
"Larg": null,
"Rarg": null
"All": false
},
"StmtLocation": 0,
"StmtLen": 66
Expand Down
25 changes: 7 additions & 18 deletions internal/endtoend/testdata/parse_basic/clickhouse/stdout.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,18 @@
"name": "GetValue",
"cmd": ":one",
"ast": {
"tag": "RawStmt",
"Stmt": {
"DistinctClause": null,
"IntoClause": null,
"tag": "SelectStmt",
"TargetList": {
"tag": "List",
"Items": [
{
"Name": null,
"Indirection": null,
"tag": "ResTarget",
"Val": {
"tag": "A_Const",
"Val": {
"tag": "Integer",
"Ival": 1
},
"Location": 31
Expand All @@ -21,21 +23,8 @@
}
]
},
"FromClause": null,
"WhereClause": null,
"GroupClause": null,
"HavingClause": null,
"WindowClause": null,
"ValuesLists": null,
"SortClause": null,
"LimitOffset": null,
"LimitCount": null,
"LockingClause": null,
"WithClause": null,
"Op": 0,
"All": false,
"Larg": null,
"Rarg": null
"All": false
},
"StmtLocation": 0,
"StmtLen": 32
Expand Down
25 changes: 7 additions & 18 deletions internal/endtoend/testdata/parse_basic/duckdb/stdout.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,18 @@
"name": "GetValue",
"cmd": ":one",
"ast": {
"tag": "RawStmt",
"Stmt": {
"DistinctClause": null,
"IntoClause": null,
"tag": "SelectStmt",
"TargetList": {
"tag": "List",
"Items": [
{
"Name": null,
"Indirection": null,
"tag": "ResTarget",
"Val": {
"tag": "A_Const",
"Val": {
"tag": "Integer",
"Ival": 1
},
"Location": 30
Expand All @@ -21,21 +23,8 @@
}
]
},
"FromClause": null,
"WhereClause": null,
"GroupClause": null,
"HavingClause": null,
"WindowClause": null,
"ValuesLists": null,
"SortClause": null,
"LimitOffset": null,
"LimitCount": null,
"LockingClause": null,
"WithClause": null,
"Op": 0,
"All": false,
"Larg": null,
"Rarg": null
"All": false
},
"StmtLocation": 0,
"StmtLen": 33
Expand Down
25 changes: 7 additions & 18 deletions internal/endtoend/testdata/parse_basic/googlesql/stdout.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,18 @@
"name": "GetValue",
"cmd": ":one",
"ast": {
"tag": "RawStmt",
"Stmt": {
"DistinctClause": null,
"IntoClause": null,
"tag": "SelectStmt",
"TargetList": {
"tag": "List",
"Items": [
{
"Name": null,
"Indirection": null,
"tag": "ResTarget",
"Val": {
"tag": "A_Const",
"Val": {
"tag": "Integer",
"Ival": 1
},
"Location": 30
Expand All @@ -21,21 +23,8 @@
}
]
},
"FromClause": null,
"WhereClause": null,
"GroupClause": null,
"HavingClause": null,
"WindowClause": null,
"ValuesLists": null,
"SortClause": null,
"LimitOffset": null,
"LimitCount": null,
"LockingClause": null,
"WithClause": null,
"Op": 0,
"All": false,
"Larg": null,
"Rarg": null
"All": false
},
"StmtLocation": 0,
"StmtLen": 31
Expand Down
25 changes: 7 additions & 18 deletions internal/endtoend/testdata/parse_basic/mssql/stdout.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,18 @@
"name": "GetValue",
"cmd": ":one",
"ast": {
"tag": "RawStmt",
"Stmt": {
"DistinctClause": null,
"IntoClause": null,
"tag": "SelectStmt",
"TargetList": {
"tag": "List",
"Items": [
{
"Name": null,
"Indirection": null,
"tag": "ResTarget",
"Val": {
"tag": "A_Const",
"Val": {
"tag": "Integer",
"Ival": 1
},
"Location": 30
Expand All @@ -21,21 +23,8 @@
}
]
},
"FromClause": null,
"WhereClause": null,
"GroupClause": null,
"HavingClause": null,
"WindowClause": null,
"ValuesLists": null,
"SortClause": null,
"LimitOffset": null,
"LimitCount": null,
"LockingClause": null,
"WithClause": null,
"Op": 0,
"All": false,
"Larg": null,
"Rarg": null
"All": false
},
"StmtLocation": 0,
"StmtLen": 32
Expand Down
Loading
Loading