firebolt-parquet: fix the EventDate constants in Q36-Q42 - #1619
Conversation
Both Firebolt Parquet entries filtered on `"EventDate" >= 16617 AND
"EventDate" <= 16647` (16630/16631 in Q42) — exactly 730 days past the end
of the dataset. hits.parquet stores EventDate as days since the epoch and
its range is 15888..15917, so all seven queries matched zero rows and their
reported timings measured nothing:
SELECT COUNT(*) FROM hits
WHERE "CounterID" = 62 AND "EventDate" >= 16617 AND "EventDate" <= 16647
-> 0
SELECT COUNT(*) FROM hits
WHERE "CounterID" = 62 AND "EventDate" >= 15887 AND "EventDate" <= 15917
-> 738172
Every other entry that filters on the raw integer uses 15887/15917 and
15900/15901; use the same constants here. Checked against a managed
Firebolt table loaded from the same file: Q36, Q37 and Q42 now return
identical results, and Q38-Q41 differ only in the order of tied rows under
LIMIT 10.
The published results for firebolt-parquet and firebolt-parquet-partitioned
were produced with the broken queries and have to be re-run. Their 3-4 ms
timings for Q36-Q42 are also the global per-query baseline on the
dashboard, so they currently penalise every other system by about 2.5%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub Actions
Error
|
The run of Logs:
|
The PR benchmark failed on all 18 machines with
Load time: 2.069
bench: data-size after load is '239646' (<5 GB)
bench: ClickBench's hits dataset doesn't fit in <5 GB on any
bench: system in the catalog; treating this as a partial load
Both entries copied ./data-size from the ingesting `firebolt` entry, where
it du's the engine's volume. These two ingest nothing - create.sql declares
an external table over the Parquet files, which ./load leaves in ./data -
so fb-volume holds a couple of hundred KB of catalog metadata and the
partial-load guard added in f26f504 rejects every run. The guard landed
on 2026-05-09 and the entries were converted to the shared driver on
2026-05-15, so they have not been able to produce results since; that is
why their newest results are still dated 2026-05-11 while `firebolt` has
2026-07-01 ones.
Report the source Parquet size instead, which is what clickhouse-parquet,
duckdb-parquet, datafusion and chdb-parquet-partitioned all do, and what
the last published results for these entries actually contain
(14779976446 is exactly the size of hits.parquet).
Also drop BENCH_DURABLE=no. That flag is for engines whose data lives in
process memory and is lost on restart, which makes the driver re-run ./load
before every query and fold its wall-clock into the cold try. Here the
Parquet files are on disk, the external table definition lives in
fb-volume, and BENCH_RESTARTABLE=no means the container is never restarted
- so the 43 extra loads do nothing except add ~2 s to every cold
measurement. Every other external-table Parquet entry sets only
BENCH_RESTARTABLE=no.
Verified against a live firebolt-core container using this branch's
scripts: ./load succeeds, ./data-size reports 14779976446 (guard passes),
and Q36-Q42 each return 10 rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The Both entries copied Pushed 3f4ec3b:
Verified against a live Needs the |
…ned (c6a.2xlarge, c6a.4xlarge, c6a.large, c6a.metal, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl, t3a.small)
|
Results for Logs:
|
1 participant
The bug
firebolt-parquet/queries.sqlandfirebolt-parquet-partitioned/queries.sqlfilter Q36–Q41 on"EventDate" >= 16617 AND "EventDate" <= 16647, and Q42 on16630/16631. Those are epoch-day values for July 2015 — exactly 730 days past the end of the dataset.hits.parquetkeepsEventDateas a raw day count, ranging 15888..15917 (2013-07-02 .. 2013-07-31), so all seven queries match zero rows and the timings recorded for them measure nothing:These two entries are the only ones in the repo using 16617/16630; every other entry that filters on the raw integer (
drill,octosql, …) uses 15887/15917 and 15900/15901, which is what this PR switches to.Verification
Loaded
hits.parquetinto Firebolt Core both as a managed table and as a Parquet external table, and diffed all 43 query results. With the corrected constants, 34/43 match exactly — including Q36, Q37 and Q42 of the seven touched here. The remaining 9 (Q17, Q23, Q24, Q31, Q32, Q38–Q41) differ only in the order of tied rows underLIMIT 10, which is nondeterministic for every system in the benchmark.The results have to be re-run
The published results for both entries were produced with the broken queries, so their Q36–Q42 numbers (3–4 ms on
c6a.4xlarge) are meaningless and optimistic. On this branch the same queries take 2–7x longer locally.This is not confined to the two Firebolt entries:
Firebolt (Parquet) (c6a.4xlarge)is currently the global per-query baseline for Q36, Q37, Q39 and Q42 at 3 ms, so on the untuned/hot view every other system is penalised by about 2.5% — Umbra (c8g.metal-48xl) 1.134 → 1.106, ClickHouse (c8g.metal-48xl) 2.337 → 2.279, and so on down the board.The PR benchmark will re-run both systems on
c6a.4xlarge. The existing results cover eight machines each, so amachine:alllabel is needed to replace all of them; alternatively the stale ones can be taggedhistoricaluntil a full re-run lands.🤖 Generated with Claude Code