refactor: thread PhysicalOptimizerContext through join_selection stats helpers - #24716
Conversation
…s helpers Replace separate config/registry parameters on join_selection stats helpers with a single &dyn PhysicalOptimizerContext, so a future session-scoped CBO input only needs a new context trait method instead of touching every helper signature.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #24716 +/- ##
==========================================
+ Coverage 81.45% 81.58% +0.12%
==========================================
Files 1120 1123 +3
Lines 401289 406603 +5314
Branches 401289 406603 +5314
==========================================
+ Hits 326871 331716 +4845
- Misses 55295 55447 +152
- Partials 19123 19440 +317 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@kosiew a small follow-up of #23651, this is a pure refactor to properly propagate I have a question: since it's a pure refactor and it doesn't really alter coverage, I would consider #24716 (comment), wdyt? Thanks! |
kosiew
left a comment
There was a problem hiding this comment.
@asolimando, thanks for working on this. The refactor looks good to me. I don't see any blocking issues. I left one non-blocking suggestion around adding focused coverage for the new context-threading boundary.
| .transform_up(|plan| { | ||
| statistical_join_selection_subrule(plan, config, registry) | ||
| }) | ||
| .transform_up(|plan| statistical_join_selection_subrule(plan, context)) |
There was a problem hiding this comment.
Could we add a focused test that calls JoinSelection::optimize_with_context with a custom context and statistics registry where the estimates actually change the join decision? The existing registry SLT covers this through the SessionState path, but a test here would directly exercise the context-threading boundary introduced by this refactor. Not blocking, but I think it would be useful coverage.
There was a problem hiding this comment.
Thanks for your review! It's indeed a good chance to improve coverage on JoinSelection::optimize_with_context itself, I have proposed a test along the line you proposed in 809b30b, let me know if that works for you.
|
🚀 |
Which issue does this PR close?
PhysicalOptimizerContextthroughjoin_selectionstats helpers (CBO extension pattern) #23671.Rationale for this change
JoinSelection's internal statistics helpers each took separateconfig: &ConfigOptionsand
registry: Option<&StatisticsRegistry>parameters. Any new session-scoped CBO inputto these rules (e.g. #21120) would require touching every helper's signature.
What changes are included in this PR?
Replace the
config/registryparameter pair on the internal statistics helpers with asingle
context: &dyn PhysicalOptimizerContext, readingcontext.config_options()andcontext.statistics_registry()internally. This is a pure internal signature change: allaffected functions are private or
pub(crate), with no cross-crate callers. Behavior isunchanged.
Are these changes tested?
No new tests; this is a non-functional refactor covered by existing tests
(
cargo test -p datafusion-physical-optimizer, thephysical_optimizer::join_selectionintegration tests, and
statistics_registry.slt).Are there any user-facing changes?
No.
Disclaimer: I used AI to assist in the code generation, I have manually reviewed the output and it matches my intention and understanding.