Skip to content

[fix](function) Align interval folding with runtime - #67840

Merged
morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/align-interval-constant-folding
Sep 14, 2026
Merged

morrySnow merged 1 commit into
apache:masterfrom
morrySnow:fix/align-interval-constant-folding

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

Problem

INTERVAL could return different results for the same repeated thresholds depending on whether the expression was constant-folded in the frontend or evaluated by the backend.

For example, the folded expressions below returned 1, 2, 2, while equivalent expressions using a numbers() column returned 2, 3, 3:

SELECT INTERVAL(0, 0, 0),
       INTERVAL(0, 0, 0, 0),
       INTERVAL(1, 0, 1, 1, 2);

Root cause

Frontend constant folding used Arrays.binarySearch. When a sorted threshold array contains duplicate values, Java may return any matching position. Backend execution uses upper-bound semantics and continues past all thresholds less than or equal to the comparison value.

Fix

Replace frontend binarySearch with the same upper-bound binary-search loop used by backend execution. Existing behavior for NULL comparison values and NULL thresholds remains unchanged.

Tests

  • Added executable-function unit tests for repeated thresholds, empty thresholds, lower/upper boundaries, and NULL values.
  • Added regression coverage comparing frontend-folded expressions with equivalent backend-evaluated expressions.
  • Full frontend build and checkstyle passed.
  • The focused unit test passed with 4 tests and no failures.
  • The new regression suite passed.

Problem: INTERVAL returned different results for repeated thresholds depending on whether the expression was folded in the frontend or executed in the backend.

Cause: Java binarySearch may return any matching duplicate, while backend execution uses upper-bound semantics and counts every threshold less than or equal to the comparison value.

Solution: Replace binarySearch with the same upper-bound loop used by backend execution, including existing NULL threshold behavior.

Tests: Added focused executable-function unit coverage and regression coverage comparing folded and runtime expressions across duplicates, NULLs, and boundaries.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16715 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b8c65ad874f76666f85f2d7e21a125f2b8b9d04f, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17571	3011	2975	2975
q2	2088	252	226	226
q3	10249	888	500	500
q4	4668	252	206	206
q5	7675	558	388	388
q6	137	112	94	94
q7	537	493	394	394
q8	9274	905	905	905
q9	3489	2378	2364	2364
q10	6524	823	691	691
q11	393	196	181	181
q12	619	257	196	196
q13	18129	1546	1156	1156
q14	154	145	136	136
q15	q16	439	390	365	365
q17	1310	879	780	780
q18	3068	2292	2234	2234
q19	1263	908	815	815
q20	364	286	194	194
q21	5660	1693	1820	1693
q22	346	262	222	222
Total cold run time: 93957 ms
Total hot run time: 16715 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3348	3294	3288	3288
q2	491	379	380	379
q3	2171	2336	2167	2167
q4	1181	1162	888	888
q5	2157	2110	2098	2098
q6	164	121	84	84
q7	1014	915	839	839
q8	1568	1379	1384	1379
q9	3110	3070	3079	3070
q10	1872	1802	1617	1617
q11	353	266	254	254
q12	453	424	339	339
q13	1485	1522	1143	1143
q14	172	180	157	157
q15	q16	390	394	353	353
q17	3647	3369	3152	3152
q18	4767	4430	4676	4430
q19	840	792	840	792
q20	1000	977	824	824
q21	3743	3016	3176	3016
q22	404	338	339	338
Total cold run time: 34330 ms
Total hot run time: 30607 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81129 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b8c65ad874f76666f85f2d7e21a125f2b8b9d04f, data reload: false

query5	4254	416	335	335
query6	391	133	118	118
query7	4954	411	229	229
query8	283	127	120	120
query9	8679	2867	2848	2848
query10	381	212	183	183
query11	5370	1031	903	903
query12	129	72	69	69
query13	1194	455	302	302
query14	6041	2160	2051	2051
query14_1	1947	1946	1940	1940
query15	183	123	112	112
query16	952	356	340	340
query17	802	437	364	364
query18	2335	321	228	228
query19	162	139	109	109
query20	70	70	69	69
query21	202	101	85	85
query22	5435	5409	5231	5231
query23	6563	6065	6045	6045
query23_1	5978	6113	6061	6061
query24	7268	1102	762	762
query24_1	749	765	762	762
query25	407	277	223	223
query26	1230	225	131	131
query27	2793	414	252	252
query28	4687	1478	1473	1473
query29	915	408	325	325
query30	258	150	130	130
query31	824	389	319	319
query32	119	71	75	71
query33	449	211	174	174
query34	993	845	483	483
query35	386	398	351	351
query36	560	555	527	527
query37	132	77	65	65
query38	1007	842	800	800
query39	479	467	459	459
query39_1	457	433	454	433
query40	202	88	84	84
query41	52	53	49	49
query42	75	67	68	67
query43	260	235	202	202
query44	971	523	530	523
query45	107	106	99	99
query46	807	833	521	521
query47	760	751	694	694
query48	316	299	217	217
query49	546	235	193	193
query50	730	262	186	186
query51	7976	7964	8076	7964
query52	67	67	66	66
query53	190	192	143	143
query54	205	149	137	137
query55	77	58	53	53
query56	186	169	158	158
query57	652	674	622	622
query58	199	163	205	163
query59	1210	1219	1087	1087
query60	226	182	167	167
query61	129	138	125	125
query62	378	210	184	184
query63	172	142	145	142
query64	2795	722	559	559
query65	1569	1557	1572	1557
query66	1940	275	205	205
query67	9810	9605	9658	9605
query68	2730	1228	758	758
query69	333	224	196	196
query70	668	596	614	596
query71	249	168	163	163
query72	2231	1732	1455	1455
query73	635	584	345	345
query74	1568	1211	1136	1136
query75	1176	1074	944	944
query76	2290	690	512	512
query77	240	263	211	211
query78	4042	3764	3292	3292
query79	1224	845	588	588
query80	642	314	263	263
query81	431	155	134	134
query82	578	122	91	91
query83	300	211	186	186
query84	284	113	89	89
query85	775	341	274	274
query86	337	177	168	168
query87	999	978	884	884
query88	2745	2123	2083	2083
query89	274	196	174	174
query90	1829	117	120	117
query91	126	115	97	97
query92	79	70	61	61
query93	1126	1141	691	691
query94	503	268	224	224
query95	514	242	228	228
query96	773	597	286	286
query97	1020	1025	1032	1025
query98	140	142	128	128
query99	422	339	302	302
Total cold run time: 173303 ms
Total hot run time: 81129 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.61 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b8c65ad874f76666f85f2d7e21a125f2b8b9d04f, data reload: false

query1	0.00	0.01	0.00
query2	0.08	0.04	0.04
query3	0.25	0.11	0.11
query4	1.61	0.09	0.10
query5	0.18	0.16	0.16
query6	1.26	0.67	0.72
query7	0.03	0.00	0.00
query8	0.04	0.02	0.03
query9	0.28	0.22	0.21
query10	0.33	0.37	0.35
query11	0.17	0.11	0.11
query12	0.15	0.12	0.12
query13	0.30	0.31	0.31
query14	0.46	0.46	0.46
query15	0.37	0.37	0.35
query16	0.24	0.22	0.22
query17	0.67	0.68	0.68
query18	0.17	0.16	0.16
query19	1.18	1.18	1.16
query20	0.02	0.01	0.01
query21	15.43	0.16	0.11
query22	5.04	0.04	0.05
query23	16.18	0.27	0.10
query24	2.99	0.32	0.25
query25	0.11	0.04	0.03
query26	0.78	0.16	0.13
query27	0.04	0.03	0.04
query28	3.62	0.55	0.27
query29	12.45	3.13	2.55
query30	0.26	0.11	0.12
query31	2.75	0.38	0.18
query32	3.52	0.31	0.23
query33	1.33	1.36	1.43
query34	15.43	2.16	1.78
query35	1.72	1.71	1.75
query36	0.46	0.29	0.28
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.03	0.02	0.02
query40	0.11	0.08	0.07
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.29 s
Total hot run time: 14.61 s

@morrySnow
morrySnow merged commit a2e9094 into apache:master Sep 14, 2026
34 of 36 checks passed
@morrySnow
morrySnow deleted the fix/align-interval-constant-folding branch September 14, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants