Skip to content

[improvement](planner) Reduce planner overhead - #67797

Draft
morrySnow wants to merge 5 commits into
apache:masterfrom
morrySnow:codex/nereids-planner-overhead
Draft

[improvement](planner) Reduce planner overhead#67797
morrySnow wants to merge 5 commits into
apache:masterfrom
morrySnow:codex/nereids-planner-overhead

Conversation

@morrySnow

@morrySnow morrySnow commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Nereids created ProcessState and maintained rewrite-path state even when plan-process tracing was disabled, rendered the final physical plan for every SQL-cache candidate before cache admission, and repeated cost calculations and CostWeight construction in the Cascades hot path.

This PR:

  • Creates ProcessState only while plan-process tracing is active. EXPLAIN PLAN PROCESS remains unchanged.
  • Defers SQL-cache physical-plan rendering until FE or BE cache-admission checks succeed.
  • Removes redundant pre-regulation node-cost calculation and child-cost accumulation, while retaining the final property-aware cost recalculation.
  • Keeps CostWeight as the semantic weight snapshot, creates it lazily once per StatementContext, and reuses it for all Cost construction. The snapshot is taken after SET_VAR preprocessing and refreshed when a prepared statement starts another execution.
  • Adds immutable Cost addition so accumulated costs do not need to be reweighted.

The independently measured ProcessState and cost-cleanup budgets are:

Query CPU saving Planning latency saving Allocation saving
TPCH Q5 0.303 ms / 2.7% 0.552 ms / 4.5% 200.1 KiB / 4.6%
TPCDS Q72 0.335 ms / 2.0% 0.490 ms / 3.0% 526.3 KiB / 5.4%
TPCDS Q64 forced Cascades 1.712 ms / 5.1% 2.027 ms / 5.7% 2618.5 KiB / 9.7%

These are arithmetic budgets from independently measured constituents, not a combined-patch ABBA result. Deferred SQL-plan rendering exposes an additional CPU/allocation cost pool of 0.408 ms/242.8 KiB, 0.588 ms/459.8 KiB, and 1.348 ms/1012.5 KiB respectively. Realized total CPU savings for that part scale with the non-admission ratio.

Release note

None

Check List (For Author)

  • Test: Unit Test
    • RewriteTopDownJobTest, CostModelV1Test, ChildrenPropertiesRegulatorTest, and SqlCacheTest: 14 tests passed before the CostWeight follow-up
    • CostModelV1Test and ChildrenPropertiesRegulatorTest after the CostWeight follow-up: 12 tests passed
    • Final isolated CostModelV1Test rerun: 4 tests passed
    • Full FE Maven build: 81/81 modules passed
    • FE Checkstyle: 0 violations
    • Generated sources completed successfully
  • Behavior changed: No
  • Does this need documentation: No

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Nereids created ProcessState and maintained rewrite-path state even when plan-process tracing was disabled, rendered the final physical plan for every SQL-cache candidate before cache admission, and repeated cost calculations and weighted-cost object construction in the Cascades hot path.

Create ProcessState only while plan-process tracing is active, render the SQL-cache plan body only after FE or BE cache-admission checks succeed, and simplify cost accumulation while retaining the final property-aware node-cost recalculation. The removed CostWeight allocation keeps its non-negative-weight validation in Cost.

The independently measured ProcessState and cost-cleanup budgets are 0.303/0.335/1.712 ms CPU and 200.1/526.3/2618.5 KiB allocation for TPCH Q5, TPCDS Q72, and forced-Cascades TPCDS Q64. The deferred SQL-plan rendering cost pool is 0.408/0.588/1.348 ms CPU and 242.8/459.8/1012.5 KiB allocation; realized total CPU savings scale with the non-admission ratio.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.nereids.jobs.RewriteTopDownJobTest,org.apache.doris.nereids.cost.CostModelV1Test,org.apache.doris.nereids.properties.ChildrenPropertiesRegulatorTest,org.apache.doris.qe.SqlCacheTest (14 tests passed)
    - Final Cost rerun: CostModelV1Test and ChildrenPropertiesRegulatorTest (11 tests passed)
    - mvn clean install -DskipTests -Dskip.doc=true -T 1C -Dmaven.build.cache.enabled=false (81/81 modules passed)
    - mvn checkstyle:check -pl fe-core (0 violations)
    - sh generated-source.sh
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow changed the title [improvement](fe) Reduce Nereids planner overhead [improvement](planner) Reduce Nereids planner overhead Sep 10, 2026
@morrySnow morrySnow changed the title [improvement](planner) Reduce Nereids planner overhead [improvement](planner) Reduce planner overhead Sep 10, 2026
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: CostWeight is a semantic snapshot of the session cost weights and should not be removed to avoid its allocation. Keep one lazily initialized CostWeight in StatementContext and reuse it for every Cost created by the statement. Lazy initialization preserves SET_VAR hint semantics, while rebinding the ConnectContext refreshes the snapshot for reused prepared statements.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - CostModelV1Test and ChildrenPropertiesRegulatorTest: 12 tests passed
    - Final CostModelV1Test rerun: 4 tests passed
    - FE Checkstyle: 0 violations
- Behavior changed: No
- Does this need documentation: No
@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@morrySnow
morrySnow marked this pull request as draft September 12, 2026 02:46
### What problem does this PR solve?

Issue Number: None

Related PR: apache#67797

Problem Summary: PR apache#67797 changed cost accumulation from weighting the summed CPU, memory, and network components to adding already-weighted double values. Floating-point non-associativity introduced one-ULP total-cost differences, which could flip strict memo cost comparisons and unexpectedly change plan shapes. Pass the statement-scoped CostWeight to Cost.add and recompute the weighted value from the accumulated components, preserving the original plan-selection semantics without restoring per-CostWeight allocations.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.nereids.cost.CostModelV1Test: 4 tests passed
    - FE Checkstyle: 0 violations
- Behavior changed: No (restores the pre-PR floating-point evaluation order)
- Does this need documentation: No
@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16870 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9c392eed9436e25f1b4acbdeaf17babc9e7c7391, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17596	3038	3043	3038
q2	2067	248	223	223
q3	10257	889	513	513
q4	4670	249	214	214
q5	7663	566	380	380
q6	136	115	92	92
q7	523	497	404	404
q8	9232	839	894	839
q9	3436	2385	2402	2385
q10	6522	871	687	687
q11	389	196	181	181
q12	616	261	207	207
q13	18127	1539	1172	1172
q14	160	146	142	142
q15	q16	447	396	378	378
q17	1428	900	807	807
q18	3128	2301	2288	2288
q19	1263	919	823	823
q20	369	280	196	196
q21	5609	1677	1884	1677
q22	338	270	224	224
Total cold run time: 93976 ms
Total hot run time: 16870 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3420	3352	3368	3352
q2	525	394	362	362
q3	2276	2379	2218	2218
q4	1216	1177	927	927
q5	2206	2135	2139	2135
q6	171	119	89	89
q7	1031	989	834	834
q8	1596	1389	1398	1389
q9	3176	3158	3134	3134
q10	1873	1816	1628	1628
q11	362	272	260	260
q12	449	432	344	344
q13	1489	1530	1147	1147
q14	170	175	165	165
q15	q16	404	397	354	354
q17	3581	3303	3211	3211
q18	4850	4442	4765	4442
q19	898	890	861	861
q20	993	992	806	806
q21	3762	3089	3252	3089
q22	409	341	328	328
Total cold run time: 34857 ms
Total hot run time: 31075 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 82185 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9c392eed9436e25f1b4acbdeaf17babc9e7c7391, data reload: false

query5	4246	401	332	332
query6	378	140	120	120
query7	4973	413	225	225
query8	285	123	120	120
query9	8684	2918	2911	2911
query10	403	218	190	190
query11	5377	1043	916	916
query12	124	73	71	71
query13	1187	466	357	357
query14	6183	2226	2112	2112
query14_1	2043	1968	1984	1968
query15	172	125	112	112
query16	921	355	349	349
query17	805	431	371	371
query18	2322	319	233	233
query19	164	138	110	110
query20	73	69	71	69
query21	199	102	88	88
query22	5446	5614	5324	5324
query23	6692	6268	6015	6015
query23_1	6201	6060	6078	6060
query24	7294	1094	801	801
query24_1	784	766	780	766
query25	431	290	249	249
query26	1224	237	131	131
query27	2790	402	249	249
query28	4718	1511	1510	1510
query29	930	449	346	346
query30	250	156	130	130
query31	809	419	333	333
query32	121	78	75	75
query33	466	223	191	191
query34	1000	830	483	483
query35	416	413	342	342
query36	574	588	550	550
query37	121	84	72	72
query38	1013	854	816	816
query39	493	477	484	477
query39_1	490	472	471	471
query40	204	93	79	79
query41	57	55	55	55
query42	79	73	73	73
query43	245	244	216	216
query44	1000	529	538	529
query45	110	108	111	108
query46	793	832	527	527
query47	769	742	716	716
query48	307	309	238	238
query49	542	245	192	192
query50	785	259	194	194
query51	8134	8143	8066	8066
query52	67	68	61	61
query53	231	210	155	155
query54	201	163	159	159
query55	70	58	64	58
query56	178	179	153	153
query57	673	686	644	644
query58	184	162	159	159
query59	1226	1255	1107	1107
query60	227	182	169	169
query61	106	112	107	107
query62	357	198	178	178
query63	169	148	147	147
query64	2616	670	609	609
query65	1607	1684	1693	1684
query66	1816	257	199	199
query67	10134	9956	9838	9838
query68	2744	1221	685	685
query69	332	239	193	193
query70	659	639	608	608
query71	248	182	166	166
query72	2208	1648	1431	1431
query73	683	588	347	347
query74	1562	1240	1125	1125
query75	1172	1102	976	976
query76	2294	731	500	500
query77	251	258	217	217
query78	4009	3739	3229	3229
query79	2450	834	569	569
query80	1583	336	259	259
query81	488	153	130	130
query82	619	123	94	94
query83	284	210	191	191
query84	296	107	86	86
query85	766	335	268	268
query86	393	177	167	167
query87	1044	983	890	890
query88	2766	2118	2094	2094
query89	279	195	180	180
query90	2009	131	128	128
query91	132	119	95	95
query92	77	71	67	67
query93	1486	1068	702	702
query94	652	255	207	207
query95	522	330	224	224
query96	799	587	267	267
query97	1058	1041	1026	1026
query98	171	144	129	129
query99	428	347	313	313
Total cold run time: 177615 ms
Total hot run time: 82185 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.85 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9c392eed9436e25f1b4acbdeaf17babc9e7c7391, data reload: false

query1	0.01	0.00	0.01
query2	0.07	0.04	0.03
query3	0.25	0.12	0.11
query4	1.60	0.10	0.09
query5	0.16	0.16	0.17
query6	1.27	0.70	0.72
query7	0.04	0.01	0.00
query8	0.04	0.03	0.03
query9	0.30	0.22	0.22
query10	0.34	0.32	0.35
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.30	0.31	0.32
query14	0.45	0.44	0.45
query15	0.36	0.35	0.36
query16	0.22	0.24	0.23
query17	0.72	0.74	0.67
query18	0.19	0.17	0.17
query19	1.22	1.20	1.15
query20	0.02	0.01	0.01
query21	15.43	0.16	0.13
query22	5.03	0.04	0.04
query23	16.20	0.27	0.10
query24	2.97	0.32	0.27
query25	0.11	0.05	0.04
query26	0.80	0.17	0.13
query27	0.04	0.02	0.03
query28	3.64	0.55	0.27
query29	12.50	3.22	2.57
query30	0.26	0.12	0.12
query31	2.76	0.39	0.17
query32	3.52	0.32	0.25
query33	1.39	1.44	1.53
query34	15.37	2.27	1.84
query35	1.74	1.74	1.72
query36	0.45	0.30	0.28
query37	0.06	0.04	0.04
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.12	0.07	0.08
query41	0.07	0.02	0.03
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.46 s
Total hot run time: 14.85 s

@morrySnow

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Requesting changes for three blocking issues:

  1. The new cost-weight lookup ignores the StatementContext owned by the active planner and can null-dereference in the supported minidump/direct-planner path.
  2. The replacement child-cost path leaves five existing ChildrenPropertiesRegulatorTest cases failing on this exact head.
  3. BE SQL-cache admission now renders a catalog-backed plan after table locks are released and after rows are sent, creating a concurrent-DDL exception/snapshot race.

Critical checkpoints

  • Goal and tests: The allocation/redundant-work reductions are mechanically aligned with the stated performance goal, but correctness is not established because of the three issues above. The new tests cover isolated cost addition/reset and FE-computed cache bodies; they do not cover active direct/minidump planning, BE-cache rendering under DDL, or tracing parity.
  • Scope and clarity: All 24 changed files belong to the four stated planner-overhead reductions. The interface propagation is complete, though combining four independent optimizations broadens the review surface.
  • Concurrency and lifecycle: Cost/rewrite work remains statement-serialized and nullable ProcessState access is consistently guarded. The material concurrency regression is late catalog-backed rendering in the BE cache path. Prepared execution resets the weight snapshot, but direct planners do not guarantee that their active statement is installed on the ambient connection.
  • Configuration and compatibility: No configuration, persisted format, storage metadata, public symbol, RPC/Thrift, or rolling-upgrade contract changes were found. ComputeResultSet is internal and all five implementations plus both callers were updated.
  • Parallel/special paths: FE cache miss/hit, BE miss, cloud preflight, result/blackhole wrappers, empty/one-row leaves, command exclusions, unsupported-cache paths, Cascades regulation/enforcement, memo ranking, and cost recomputation were checked. No distinct issue remains beyond the three inline comments.
  • Conditionals and error handling: Cache eligibility/hit exclusions and every nullable tracing-state use retain coherent guards. The BE admission path must not let optional cache rendering fail an already-produced query.
  • Transactions, persistence, data writes, and FE-BE propagation: These areas are not changed.
  • Observability and performance: No new metric/log is required. The intended allocation and redundant-calculation savings are plausible; they do not justify the context-ownership and post-lock rendering failures.
  • User focus: No additional user-provided review focus was supplied.

Verification status

This was a complete static review of the authoritative diff after two convergence rounds; all round-2 reviewers returned NO_NEW_VALUABLE_FINDINGS. Local builds/tests/source edits were prohibited by the review contract. Exact-head CI was independently triaged: FE UT build 1044397 has the five PR-relevant regulator NPEs; the external-regression failure occurred after all 661 tests passed and was caused by a Maven download EOF, while FE coverage failed on its Node/GLIBC toolchain. Compile, checkstyle, performance, BE UT, and the remaining regression checks passed.

}

public CostWeight getCostWeight() {
return getStatementContext().getCostWeight();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Bind the weight to the planner's active statement

This new lookup assumes the StatementContext owned by NereidsPlanner is also installed on its ConnectContext, but the supported minidump replay path violates that invariant. MinidumpUtils.setConnectContext() creates a connection with no statement, and executeSql() constructs a statement only for NereidsPlanner; its first CostCalculator.calculateCost() therefore reaches this line with getStatementContext() == null and throws instead of producing the replay plan. Please pass the active planner/Cascades statement (or its snapshotted CostWeight) into cost calculation, or establish the ownership invariant for every direct-planner entry point, and cover minidump replay with an active test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9c7aa04. Cost calculation now receives the CostWeight snapshot from the active CascadesContext/StatementContext and threads the same snapshot through memo ranking and enforcer accumulation, so it no longer rediscovers the weight through ConnectContext. I also restored the minidump replay context contract, enabled the previously disabled replay test, and updated its fixture to the current format. The targeted FE run passed all 28 tests across MinidumpUtTest, ChildrenPropertiesRegulatorTest, CostModelV1Test, SqlCacheTest, and OlapQueryCacheTest.

return;
}

sqlCacheContext.setPhysicalPlan(physicalPlan.treeString());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not render the plan after releasing table locks

On the BE-cache path this call runs only after the planner has released its table read locks and after StmtExecutor has sent all result rows, but before it sets EOF. treeString() calls PhysicalOlapScan.toString(), which iterates the live OlapTable.indexNameToId HashMap and copies the nameToPartition TreeMap without a lock. A concurrent index/partition DDL can therefore throw ConcurrentModificationException here, converting an otherwise successful query into a post-row failure; a non-throwing interleaving can also cache a body that does not describe the executed snapshot. Previously the plan string was captured inside planWithLock. Please capture an immutable plan body while the catalog is protected, or otherwise make late rendering snapshot-safe and unable to fail the completed query.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9c7aa04. NereidsPlanner now renders and stores the SQL-cache physical plan inside the lock-protected planning callback. FE/BE cache insertion consumes that already captured string and no longer receives or renders a live PhysicalPlan after result delivery and table-lock release. The targeted FE run passed all 28 tests, including SqlCacheTest and OlapQueryCacheTest.

### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#67797

Problem Summary: ChildrenPropertiesRegulatorTest mocked CascadesContext without a ConnectContext. After cost accumulation began using the statement-scoped CostWeight directly, five property-regulation cases dereferenced that missing context and failed with NullPointerException. Build the test fixture with a real ConnectContext and StatementContext and expose it through the mocked CascadesContext so the unit test matches the production invariant.

### Release note

None

### Check List (For Author)

- Test: Unit Test

  - ./run-fe-ut.sh --run org.apache.doris.nereids.properties.ChildrenPropertiesRegulatorTest (8 tests)

- Behavior changed: No

- Does this need documentation: No
@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@morrySnow

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Requesting changes on current head d4aa2f6ce704ff042368c85de7a8ee1d83fddfcd. The latest commit statically repairs the regulator test-fixture failure, but two previously reported P1 correctness issues remain in the code:

  1. Bind the weight to the planner's active statement: minidump replay installs no StatementContext on its ConnectContext, so the first cost calculation still null-dereferences through PlanContext.getCostWeight(). The same ownership invariant must cover sibling connection-based lookups in enforcer and nth-plan paths.
  2. Do not render the plan after releasing table locks: BE SQL-cache admission still calls physicalPlan.treeString() after planner locks are released and after rows may be sent but before EOF; OLAP scan rendering traverses mutable catalog maps, so concurrent DDL can fail an otherwise completed query or cache a non-snapshot plan body.

No new inline comments are included because both defects are fully covered by those existing threads; reposting them would violate the duplicate fence.

Critical checkpoints

  • Goal and tests: Conditional ProcessState allocation, deferred cache rendering, removal of redundant pre-regulation cost work, and statement-scoped weight reuse are aligned with the stated overhead goal. The arithmetic/reset tests and FE-cache plan-body assertion are statically consistent, and the current fixture now supplies the regulator's required connection/statement. However, there is still no test covering active minidump/direct planning, snapshot-safe BE cache rendering under concurrent DDL, or explicit trace-on parity. The author reports the targeted tests and full FE build passed; this review did not independently run builds or tests because the review contract prohibits them.
  • Scope and clarity: All 24 changed files map to the four stated planner optimizations. The changes are mechanically focused within each mechanism, though combining the mechanisms broadens the review surface.
  • Concurrency and lifecycle: Cascades jobs are statement-serialized; the lazy cost snapshot has no planner-job race, prepared execution resets it, and every nullable ProcessState use is guarded. The remaining lifecycle violations are the direct-planner/connection statement mismatch and the query-versus-DDL race during late catalog-backed rendering.
  • Configuration and compatibility: No configuration, persisted format, RPC/Thrift, storage metadata, or rolling-upgrade contract changes were found. The internal ComputeResultSet signature is propagated to all five implementations and both callers.
  • Parallel and special paths: FE/BE cache admission and replay, cloud preflight, MySQL/Arrow/prepared exclusions, result/blackhole wrappers, one-row/empty/cache leaves, Cascades property alternatives and enforcers, nth-plan ranking, and cost recomputation were checked. No distinct issue remains beyond the two fenced P1s.
  • Conditionals and error handling: Cache eligibility, SQL-cache replay exclusion, and trace-state guards are coherent. Optional cache-plan rendering must not be allowed to throw after BE rows have been produced.
  • Transactions, persistence, data writes, and FE-BE propagation: Not changed by this PR.
  • Observability and performance: No new metric or log appears necessary. The intended allocation and redundant-calculation savings are plausible, but they do not justify the two correctness regressions.
  • User focus: No additional user-provided review focus was supplied.

Verification status

This was a complete static review of the authoritative 24-file diff. One full convergence round used two complete-review agents plus a separate five-risk audit; all returned NO_NEW_VALUABLE_FINDINGS after missed-case rechecks, and every risk was accepted as duplicate-fenced or dismissed with code evidence. No new inline comment was accepted. The live base/head were reverified immediately before submission.

### What problem does this PR solve?

Issue Number: None

Related PR: apache#67797

Problem Summary: Cost calculation rediscovered weights through the connection's current statement instead of using the statement owned by the active planner, which broke direct planner entry points such as minidump replay. SQL cache plan text was also rendered after catalog locks were released. Pass the active statement's cost-weight snapshot through memo costing and enforcer accumulation, restore the minidump replay context contract, and capture the cache plan while table locks are held.

### Release note

None

### Check List (For Author)

- Test: Unit Test
  - MinidumpUtTest
  - ChildrenPropertiesRegulatorTest
  - CostModelV1Test
  - SqlCacheTest
  - OlapQueryCacheTest
- Behavior changed: No
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 85.37% (70/82) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16848 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4aa2f6ce704ff042368c85de7a8ee1d83fddfcd, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17579	3019	3034	3019
q2	2070	249	230	230
q3	10267	860	513	513
q4	4665	243	210	210
q5	7678	542	386	386
q6	133	112	91	91
q7	516	566	397	397
q8	9237	900	928	900
q9	3421	2378	2345	2345
q10	6524	862	713	713
q11	388	200	181	181
q12	616	257	195	195
q13	18123	1532	1142	1142
q14	159	147	136	136
q15	q16	427	391	370	370
q17	1435	822	773	773
q18	3081	2227	2221	2221
q19	1256	893	831	831
q20	370	281	204	204
q21	5626	1763	1853	1763
q22	327	273	228	228
Total cold run time: 93898 ms
Total hot run time: 16848 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3414	3335	3313	3313
q2	506	379	368	368
q3	2208	2242	2158	2158
q4	1206	1165	900	900
q5	2161	2103	2085	2085
q6	172	120	85	85
q7	1080	933	860	860
q8	1579	1386	1390	1386
q9	3128	3081	3081	3081
q10	1840	1799	1629	1629
q11	351	268	246	246
q12	449	428	336	336
q13	1459	1519	1152	1152
q14	174	184	154	154
q15	q16	391	387	383	383
q17	3582	3256	3172	3172
q18	4813	4351	4727	4351
q19	843	806	903	806
q20	1021	991	825	825
q21	3826	3129	3318	3129
q22	408	347	335	335
Total cold run time: 34611 ms
Total hot run time: 30754 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81240 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4aa2f6ce704ff042368c85de7a8ee1d83fddfcd, data reload: false

query5	4263	407	322	322
query6	394	134	126	126
query7	4969	392	202	202
query8	287	119	124	119
query9	8703	2877	2859	2859
query10	387	229	180	180
query11	5355	1031	914	914
query12	121	71	69	69
query13	1190	441	327	327
query14	5986	2210	2079	2079
query14_1	1973	1963	1952	1952
query15	171	119	111	111
query16	896	370	364	364
query17	784	452	365	365
query18	2330	307	229	229
query19	167	143	109	109
query20	72	78	73	73
query21	198	101	88	88
query22	5521	5327	5290	5290
query23	6753	6378	5979	5979
query23_1	6108	5978	5954	5954
query24	7238	1093	769	769
query24_1	758	770	760	760
query25	396	271	245	245
query26	1224	216	125	125
query27	2810	390	259	259
query28	4694	1483	1497	1483
query29	906	408	334	334
query30	241	154	130	130
query31	807	411	325	325
query32	126	75	66	66
query33	445	232	171	171
query34	968	832	482	482
query35	404	410	343	343
query36	559	559	547	547
query37	115	79	68	68
query38	994	835	836	835
query39	496	515	494	494
query39_1	471	476	458	458
query40	202	92	74	74
query41	53	56	52	52
query42	71	71	72	71
query43	241	234	210	210
query44	999	535	514	514
query45	107	109	106	106
query46	769	834	522	522
query47	767	762	720	720
query48	314	331	233	233
query49	528	226	187	187
query50	699	258	195	195
query51	8102	8079	8013	8013
query52	70	63	57	57
query53	186	196	144	144
query54	211	152	161	152
query55	72	59	52	52
query56	186	155	146	146
query57	655	691	652	652
query58	188	178	223	178
query59	1202	1228	1091	1091
query60	245	174	186	174
query61	133	122	102	102
query62	343	205	174	174
query63	175	139	142	139
query64	2677	674	590	590
query65	1606	1639	1579	1579
query66	1803	252	205	205
query67	9970	9643	9745	9643
query68	2735	1155	727	727
query69	331	211	192	192
query70	664	599	586	586
query71	247	174	162	162
query72	2203	1634	1540	1540
query73	670	570	335	335
query74	1569	1227	1127	1127
query75	1184	1077	939	939
query76	2274	702	538	538
query77	235	261	196	196
query78	4092	3600	3148	3148
query79	1184	831	578	578
query80	1170	315	251	251
query81	480	155	129	129
query82	611	136	101	101
query83	298	199	190	190
query84	294	114	91	91
query85	764	334	273	273
query86	388	179	162	162
query87	1025	964	917	917
query88	2761	2079	2101	2079
query89	284	195	174	174
query90	1970	129	128	128
query91	128	120	97	97
query92	78	69	67	67
query93	1264	1013	706	706
query94	663	230	178	178
query95	504	322	216	216
query96	794	616	258	258
query97	1058	1062	979	979
query98	147	137	132	132
query99	421	341	309	309
Total cold run time: 174772 ms
Total hot run time: 81240 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.77 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d4aa2f6ce704ff042368c85de7a8ee1d83fddfcd, data reload: false

query1	0.00	0.00	0.00
query2	0.07	0.03	0.03
query3	0.24	0.11	0.11
query4	1.60	0.10	0.10
query5	0.18	0.15	0.16
query6	1.25	0.71	0.67
query7	0.03	0.01	0.00
query8	0.04	0.03	0.03
query9	0.28	0.22	0.21
query10	0.37	0.34	0.35
query11	0.16	0.12	0.12
query12	0.16	0.13	0.12
query13	0.31	0.31	0.31
query14	0.45	0.45	0.46
query15	0.35	0.34	0.35
query16	0.22	0.23	0.25
query17	0.67	0.68	0.68
query18	0.19	0.17	0.17
query19	1.19	1.12	1.20
query20	0.02	0.01	0.01
query21	15.44	0.15	0.12
query22	5.08	0.04	0.04
query23	16.22	0.25	0.10
query24	3.09	0.32	0.25
query25	0.10	0.05	0.03
query26	0.80	0.16	0.13
query27	0.03	0.03	0.03
query28	3.61	0.54	0.29
query29	12.47	3.15	2.55
query30	0.26	0.10	0.12
query31	2.76	0.37	0.17
query32	3.52	0.32	0.23
query33	1.50	1.54	1.54
query34	15.42	2.28	1.77
query35	1.75	1.72	1.73
query36	0.45	0.30	0.29
query37	0.06	0.04	0.04
query38	0.05	0.03	0.03
query39	0.04	0.02	0.03
query40	0.12	0.07	0.07
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 90.68 s
Total hot run time: 14.77 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants