Skip to content

[fix](arrow-flight-sql) Fix arrow-flight-sql ConnectContext to use a unified ID #46284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 3, 2025
Merged

[fix](arrow-flight-sql) Fix arrow-flight-sql ConnectContext to use a unified ID #46284

merged 1 commit into from
Jan 3, 2025

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Jan 2, 2025

What problem does this PR solve?

submit ArrowFlightSqlContext to this ConnectScheduler.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 2, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jan 2, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32741 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9913e0ec88162dcee87325b96f174da0371c561, data reload: false

------ Round 1 ----------------------------------
q1	17632	6240	6048	6048
q2	2045	335	168	168
q3	10478	1330	750	750
q4	10228	887	449	449
q5	7775	2200	2004	2004
q6	214	181	148	148
q7	917	753	626	626
q8	9229	1403	1197	1197
q9	5257	4882	4947	4882
q10	6739	2333	1876	1876
q11	471	285	268	268
q12	349	363	214	214
q13	17776	3641	2933	2933
q14	247	231	209	209
q15	561	500	493	493
q16	624	623	582	582
q17	590	868	351	351
q18	7343	6484	6479	6479
q19	2744	968	577	577
q20	313	337	193	193
q21	2894	2270	1992	1992
q22	376	340	302	302
Total cold run time: 104802 ms
Total hot run time: 32741 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6357	6198	6212	6198
q2	247	327	241	241
q3	2275	2657	2305	2305
q4	1387	1812	1369	1369
q5	4353	4773	5009	4773
q6	187	178	143	143
q7	2174	2022	1811	1811
q8	2613	2861	2717	2717
q9	7425	7196	7262	7196
q10	3061	3324	2818	2818
q11	600	531	497	497
q12	671	764	595	595
q13	3394	3735	3135	3135
q14	290	322	283	283
q15	569	511	493	493
q16	642	677	650	650
q17	1228	1752	1262	1262
q18	7650	7385	7127	7127
q19	816	1121	1195	1121
q20	1963	1933	1804	1804
q21	5501	5209	4849	4849
q22	595	602	570	570
Total cold run time: 53998 ms
Total hot run time: 51957 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190960 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9913e0ec88162dcee87325b96f174da0371c561, data reload: false

query1	983	404	379	379
query2	6521	2419	2384	2384
query3	6724	217	217	217
query4	33738	23598	23356	23356
query5	4336	627	482	482
query6	284	202	185	185
query7	4625	492	297	297
query8	299	243	230	230
query9	9573	2665	2641	2641
query10	457	344	271	271
query11	17986	15876	15574	15574
query12	156	111	106	106
query13	1648	542	399	399
query14	10583	6923	6884	6884
query15	241	208	192	192
query16	8078	599	399	399
query17	1580	748	566	566
query18	2095	397	301	301
query19	208	196	160	160
query20	121	119	114	114
query21	210	127	111	111
query22	4201	4372	4132	4132
query23	35193	33562	33543	33543
query24	7287	2411	2285	2285
query25	495	448	389	389
query26	1215	281	159	159
query27	2631	470	344	344
query28	5762	2427	2405	2405
query29	745	541	412	412
query30	228	184	151	151
query31	971	890	825	825
query32	96	64	60	60
query33	550	358	315	315
query34	749	839	521	521
query35	777	814	754	754
query36	1015	1040	998	998
query37	121	102	75	75
query38	4294	4411	4145	4145
query39	1529	1445	1437	1437
query40	217	118	102	102
query41	48	46	44	44
query42	123	102	106	102
query43	531	547	505	505
query44	1318	806	811	806
query45	185	174	166	166
query46	876	1042	664	664
query47	1921	1931	1854	1854
query48	380	411	319	319
query49	792	476	398	398
query50	643	640	403	403
query51	7223	7140	7140	7140
query52	100	108	96	96
query53	226	258	190	190
query54	475	501	407	407
query55	86	81	83	81
query56	253	269	238	238
query57	1184	1213	1149	1149
query58	239	234	225	225
query59	3327	3249	3150	3150
query60	277	296	250	250
query61	116	107	106	106
query62	877	787	731	731
query63	228	198	197	197
query64	4627	1002	657	657
query65	3284	3216	3273	3216
query66	1049	435	315	315
query67	16058	15871	15569	15569
query68	7860	759	529	529
query69	477	286	254	254
query70	1265	1155	1079	1079
query71	441	292	253	253
query72	6108	3883	3868	3868
query73	654	756	350	350
query74	9942	9167	9006	9006
query75	3814	3143	2685	2685
query76	3639	1172	786	786
query77	766	372	275	275
query78	10134	10158	9508	9508
query79	3276	835	586	586
query80	700	514	439	439
query81	492	277	231	231
query82	452	154	118	118
query83	201	171	144	144
query84	287	87	74	74
query85	836	351	301	301
query86	403	334	314	314
query87	4627	4680	4380	4380
query88	5007	2219	2150	2150
query89	411	329	306	306
query90	1887	191	188	188
query91	134	135	104	104
query92	65	55	53	53
query93	2284	872	546	546
query94	648	408	287	287
query95	340	267	261	261
query96	481	616	277	277
query97	2756	2852	2711	2711
query98	246	203	196	196
query99	1687	1580	1423	1423
Total cold run time: 297288 ms
Total hot run time: 190960 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.18 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9913e0ec88162dcee87325b96f174da0371c561, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.08	0.06
query4	1.62	0.10	0.10
query5	0.43	0.42	0.41
query6	1.18	0.66	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.52	0.50
query10	0.56	0.56	0.55
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.74	2.78	2.85
query15	0.90	0.84	0.84
query16	0.39	0.39	0.40
query17	1.04	1.00	1.07
query18	0.23	0.21	0.21
query19	1.92	1.75	2.03
query20	0.01	0.02	0.01
query21	15.74	0.89	0.62
query22	0.93	0.84	0.84
query23	15.13	1.45	0.56
query24	2.89	1.58	2.00
query25	0.13	0.18	0.12
query26	0.19	0.15	0.14
query27	0.06	0.05	0.05
query28	14.63	1.50	1.05
query29	12.60	3.94	3.26
query30	0.25	0.09	0.06
query31	2.83	0.60	0.38
query32	3.23	0.54	0.46
query33	3.19	3.16	3.19
query34	16.84	5.12	4.50
query35	4.48	4.42	4.50
query36	0.83	0.50	0.52
query37	0.10	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.13	0.14
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.31 s
Total hot run time: 32.18 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 3, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2025

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2025

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Jan 3, 2025
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 4b95e8f into apache:master Jan 3, 2025
27 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 3, 2025
@xinyiZzz
[fix](arrow-flight-sql) Fix arrow-flight-sql ConnectContext to use a …
6594b31 
…unified ID (#46284)
@github-actions github-actions bot mentioned this pull request Jan 3, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Jan 3, 2025
@xinyiZzz
[fix](arrow-flight-sql) Fix arrow-flight-sql ConnectContext to use a …
9055196 
…unified ID (#46284)
@github-actions github-actions bot mentioned this pull request Jan 3, 2025
Merged
yiguolei pushed a commit that referenced this pull request Jan 4, 2025
@github-actions @xinyiZzz
branch-2.1: [fix](arrow-flight-sql) Fix arrow-flight-sql ConnectConte…
714b90a 
…xt to use a unified ID #46284 (#46389)

Cherry-picked from #46284

Co-authored-by: Xinyi Zou <[email protected]>
yiguolei pushed a commit that referenced this pull request Jan 4, 2025
@github-actions @xinyiZzz
branch-3.0: [fix](arrow-flight-sql) Fix arrow-flight-sql ConnectConte…
51a6c63 
…xt to use a unified ID #46284 (#46388)

Cherry-picked from #46284

Co-authored-by: Xinyi Zou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@wangbo wangbo wangbo approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@xinyiZzz @Thearas @doris-robot @yiguolei @wangbo @morrySnow