fix: literal-on-left equality no longer collapses the filter to false - #24763
Open
bharadwaj-pendyala wants to merge 2 commits into
Open
fix: literal-on-left equality no longer collapses the filter to false#24763bharadwaj-pendyala wants to merge 2 commits into
bharadwaj-pendyala wants to merge 2 commits into
Conversation
simplify_predicates groups comparisons by column and accepts both `<col> <op> <literal>` and `<literal> <op> <col>`, but it then compared the grouped equalities as whole Exprs. Two equalities that differ only in operand order looked contradictory, so `s = 'a' AND 'a' = s` became Boolean(false) and the query returned no rows. Normalize the literal to the right with Operator::swap at the point where the grouping already tells the two orientations apart. That also fixes the tie-break in find_most_restrictive_predicate, which counted only Gt as strict and kept `a >= 5` over `5 < a`.
bharadwaj-pendyala
marked this pull request as ready for review
August 29, 2026 02:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No separate issue. I found this while reading
simplify_predicates.Rationale for this change
WHERE s = 'a' AND 'a' = sreturns no rows, where one row is expected:Either half on its own returns
a, and the same query against an INT column returns the row.On
main(4d3e79e),EXPLAIN VERBOSEshows the filter turning into a constant between two rules:PushDownFiltersplits the conjuncts and callssimplify_predicates. It accepts both<col> <op> <literal>and<literal> <op> <col>, butsimplify_column_predicatescompares wholeExprs.t.s = Utf8View("a")andUtf8View("a") = t.saren't structurally equal, so the two equalities read as a contradiction and the conjunction becomesfalse.The INT version survives because the
Canonicalizerreorders it first. It can't do that here: it runs once atexpr_simplifier.rs:203, ahead of the const-evaluation loop, so it seesCAST(Utf8("a") AS Utf8View)rather than aLiteraland its(Literal, Column)arm doesn't match. The cast folds to a literal afterwards. Canonicalization is skipped entirely forJoin(simplify_exprs.rs:130), sosimplify_predicatescan't assume canonical input either way.The same gap costs a strict bound. Given
a >= 5and5 < a,find_most_restrictive_predicatebreaks the tie onop == Gt, doesn't countLtwith the literal on the left as strict, keepsa >= 5, and letsa = 5through.What changes are included in this PR?
simplify_predicatesnow normalizes the literal to the right withop.swap(), at the point where it already distinguishes the two orientations.simplify_column_predicatescan then match on the operator alone. No signature changes.Are these changes tested?
Two unit tests in
simplify_predicates.rsand four cases insimplify_predicates.slt. All six fail before the fix. With onlysimplify_predicates.rsreverted the SLT reportsEmptyRelation: rows=0whereFilter: test_data.str_col = Utf8View("apple")is expected, and theapplerow goes missing.datafusion-optimizeris green (765 lib, 26 integration, 5 doc) and clippy with-D warningsis clean. The fullsqllogictestsrun passes exceptwindow_limits.slt, which fails identically on an unmodifiedmain.SELECT * FROM t WHERE s = 'a' AND 'b' = sstaysEmptyRelation: rows=0before and after, and that's pinned in the SLT.Are there any user-facing changes?
Affected queries return the right rows instead of none.
Predicates reaching
simplify_predicateswith the literal on the left now come back with it on the right, so a plan can showa > 5where it used to show5 < a. Nothing in the test suite depended on that, but the function is public.Equalities whose literals are equal in value but differ in
ScalarValuerepresentation still collapse tofalse. Onmain,[a = 5i32, a = 5i64]in the same orientation already returnsBoolean(false), so that predates this change and isn't orientation related.This PR was written with AI assistance.