Skip to content

[chore](mtmv) Document materialized view rewrite flow - #67241

Open
seawinde wants to merge 2 commits into
apache:masterfrom
seawinde:doc-mv-rewrite-flow
Open

[chore](mtmv) Document materialized view rewrite flow#67241
seawinde wants to merge 2 commits into
apache:masterfrom
seawinde:doc-mv-rewrite-flow

Conversation

@seawinde

@seawinde seawinde commented Aug 28, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary:
The transparent materialized view rewrite flow spans pre-RBO plan capture, graph compatibility, predicate compensation, expression rewriting, rule-specific plan reconstruction, and partition UNION compensation. The core code did not provide an end-to-end guide connecting these stages, which made the rewrite invariants difficult to review and maintain.

This PR adds comments only. It documents the pre-rewrite trigger mask and adds staged flow descriptions around the shared materialized view rewrite template and its core helper methods.

File Change Description
PreMaterializedViewRewriter.java Document the pre-rewrite lifecycle, NEED_PRE_REWRITE_RULE_TYPES, and the isolated CBO selection flow.
AbstractMaterializedViewRule.java Add the end-to-end rewrite diagram, numbered core stages, and method contracts for predicate and expression rewriting.
StructInfo.java Explain the semantic hypergraph compatibility result produced by isGraphLogicalEquals.
PartitionCompensator.java Explain eligibility and safety constraints for partition UNION compensation.

No plan behavior, API, or runtime logic is changed.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
      • mvn -pl fe-core -am -DskipTests -Dskip.doc=true validate
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@seawinde
[doc](fe) Document materialized view rewrite flow
a1149a8 
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Document the transparent materialized view pre-rewrite trigger and the core query-to-view matching, predicate compensation, expression rewrite, and partition union compensation flow.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - mvn -pl fe-core -am -DskipTests -Dskip.doc=true validate
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

hello-stephen commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde

seawinde commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

run buildall

@seawinde
[doc](fe) Explain query reconstruction from MV scan
8e17090 
### What problem does this PR solve?

Issue Number: None

Related PR: apache#67241

Problem Summary: Explain the rewriteQueryByView extension point, including its MV-scan input, rule-family responsibilities, expression mappings, output contract, and failure semantics.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - mvn -pl fe-core -am -DskipTests -Dskip.doc=true validate
- Behavior changed: No
- Does this need documentation: No
@seawinde
seawinde marked this pull request as ready for review August 28, 2026 03:14
@morrySnow morrySnow changed the title [doc](fe) Document materialized view rewrite flow [chore](mtmv) Document materialized view rewrite flow Aug 28, 2026
@hello-stephen

hello-stephen commented Aug 28, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16825 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a1149a8008822fe77c9691f086dd82c8bdcb4c06, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17583	3062	3071	3062
q2	2187	260	218	218
q3	10150	869	502	502
q4	4672	257	209	209
q5	7666	572	388	388
q6	134	118	96	96
q7	547	533	392	392
q8	9265	881	906	881
q9	3492	2453	2431	2431
q10	6537	906	724	724
q11	400	196	182	182
q12	616	260	205	205
q13	18115	1552	1181	1181
q14	161	154	135	135
q15	q16	443	401	369	369
q17	1390	895	829	829
q18	3138	2326	2267	2267
q19	1276	940	730	730
q20	394	297	202	202
q21	5691	1589	1851	1589
q22	327	282	233	233
Total cold run time: 94184 ms
Total hot run time: 16825 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3424	3371	3356	3356
q2	520	419	382	382
q3	2291	2310	2243	2243
q4	1217	1185	902	902
q5	2244	2149	2146	2146
q6	175	116	87	87
q7	1063	978	898	898
q8	1641	1447	1458	1447
q9	3206	3160	3155	3155
q10	1895	1888	1687	1687
q11	373	280	259	259
q12	467	434	356	356
q13	1511	1557	1166	1166
q14	194	181	178	178
q15	q16	409	399	375	375
q17	3701	3373	3328	3328
q18	4941	4482	5035	4482
q19	957	873	860	860
q20	1014	983	819	819
q21	3912	3232	3277	3232
q22	413	349	327	327
Total cold run time: 35568 ms
Total hot run time: 31685 ms

@hello-stephen

hello-stephen commented Aug 28, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 83905 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a1149a8008822fe77c9691f086dd82c8bdcb4c06, data reload: false

query5	4261	424	345	345
query6	406	151	140	140
query7	4870	431	239	239
query8	299	132	124	124
query9	8710	3062	3017	3017
query10	412	218	215	215
query11	5390	1056	926	926
query12	136	74	76	74
query13	1197	455	347	347
query14	6037	2313	2184	2184
query14_1	2033	2021	2008	2008
query15	184	124	117	117
query16	941	380	376	376
query17	810	461	373	373
query18	2347	336	250	250
query19	176	142	116	116
query20	75	73	74	73
query21	203	105	89	89
query22	5333	5356	5318	5318
query23	6738	6358	6345	6345
query23_1	6195	6129	6282	6129
query24	7369	1122	804	804
query24_1	785	801	790	790
query25	439	315	279	279
query26	1227	243	138	138
query27	2764	440	263	263
query28	4657	1518	1505	1505
query29	959	462	365	365
query30	266	161	133	133
query31	825	421	345	345
query32	137	90	81	81
query33	470	222	182	182
query34	1003	805	482	482
query35	423	411	355	355
query36	597	575	575	575
query37	122	83	76	76
query38	1033	862	854	854
query39	524	497	473	473
query39_1	476	494	502	494
query40	207	101	84	84
query41	60	61	57	57
query42	78	75	76	75
query43	243	247	222	222
query44	1040	547	583	547
query45	119	108	104	104
query46	813	842	535	535
query47	782	790	723	723
query48	317	336	234	234
query49	585	239	192	192
query50	740	267	196	196
query51	8148	8314	8104	8104
query52	70	71	66	66
query53	265	199	159	159
query54	223	201	152	152
query55	77	64	56	56
query56	206	178	179	178
query57	669	660	610	610
query58	211	180	176	176
query59	1228	1282	1108	1108
query60	237	192	181	181
query61	124	123	129	123
query62	356	217	185	185
query63	171	139	146	139
query64	2879	736	590	590
query65	1603	1655	1594	1594
query66	1793	266	206	206
query67	9842	9862	9827	9827
query68	2912	1192	741	741
query69	340	227	208	208
query70	665	595	640	595
query71	260	180	171	171
query72	2383	1822	1631	1631
query73	685	598	347	347
query74	1991	1264	1156	1156
query75	1201	1124	987	987
query76	2301	756	550	550
query77	255	266	213	213
query78	4078	3766	3229	3229
query79	2779	825	620	620
query80	1593	343	272	272
query81	533	157	138	138
query82	641	125	97	97
query83	289	214	193	193
query84	308	109	92	92
query85	853	362	299	299
query86	478	185	174	174
query87	1053	1005	918	918
query88	2941	2161	2140	2140
query89	294	200	178	178
query90	2006	135	137	135
query91	134	130	103	103
query92	96	72	71	71
query93	1891	1157	771	771
query94	643	268	235	235
query95	574	327	223	223
query96	868	579	284	284
query97	1082	1092	1033	1033
query98	166	134	148	134
query99	434	357	322	322
Total cold run time: 180050 ms
Total hot run time: 83905 ms

@hello-stephen

hello-stephen commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100% (0/0) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Aug 28, 2026

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.89 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a1149a8008822fe77c9691f086dd82c8bdcb4c06, data reload: false

query1	0.00	0.00	0.01
query2	0.08	0.04	0.04
query3	0.25	0.11	0.10
query4	1.61	0.10	0.11
query5	0.18	0.16	0.16
query6	1.22	0.70	0.68
query7	0.03	0.01	0.00
query8	0.05	0.03	0.03
query9	0.29	0.23	0.22
query10	0.37	0.36	0.36
query11	0.16	0.12	0.12
query12	0.14	0.13	0.12
query13	0.33	0.31	0.31
query14	0.48	0.47	0.45
query15	0.38	0.36	0.37
query16	0.23	0.25	0.23
query17	0.73	0.70	0.74
query18	0.18	0.15	0.16
query19	1.30	1.17	1.27
query20	0.01	0.01	0.01
query21	15.43	0.14	0.11
query22	5.08	0.04	0.04
query23	16.19	0.27	0.10
query24	3.01	0.32	0.24
query25	0.12	0.04	0.03
query26	0.73	0.17	0.13
query27	0.04	0.02	0.04
query28	3.62	0.57	0.28
query29	12.42	3.16	2.60
query30	0.26	0.12	0.12
query31	2.75	0.38	0.17
query32	3.51	0.33	0.23
query33	1.53	1.56	1.40
query34	15.40	2.35	1.83
query35	1.86	1.74	1.76
query36	0.47	0.30	0.30
query37	0.06	0.04	0.04
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.12	0.09	0.07
query41	0.07	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.83 s
Total hot run time: 14.89 s

yujun777
yujun777 approved these changes Sep 2, 2026
View reviewed changes

@yujun777 yujun777 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yujun777 yujun777 yujun777 approved these changes
@morrySnow morrySnow Awaiting requested review from morrySnow morrySnow is a code owner
@924060929 924060929 Awaiting requested review from 924060929 924060929 is a code owner
@englefly englefly Awaiting requested review from englefly englefly is a code owner
@starocean999 starocean999 Awaiting requested review from starocean999 starocean999 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@seawinde @hello-stephen @yujun777