Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Avoid copy in ByteLiteralNode #5763

Merged
merged 1 commit into from
May 28, 2024
Merged

Conversation

jinchengchenghh
Copy link
Contributor

@jinchengchenghh jinchengchenghh commented May 16, 2024

The serialized bloom filter is binary node, it may be very large, then affect the performance.

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

@jinchengchenghh jinchengchenghh changed the title [CORE] Optimize plan to use ByteBuffer and avoid copy in ByteLiteralNode [CORE] Avoid copy in ByteLiteralNode May 16, 2024
Copy link

Run Gluten Clickhouse CI

@jinchengchenghh
Copy link
Contributor Author

jinchengchenghh commented May 16, 2024

Can not use ByteBuffer to serialize substrait plan because of exception

java.io.NotSerializableException: java.nio.DirectByteBuffer

@jinchengchenghh
Copy link
Contributor Author

/Benchmark Velox

@jinchengchenghh
Copy link
Contributor Author

/Benchmark Velox TPCDS

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_5763_time.csv log/native_master_05_15_2024_2984bd740_time.csv difference percentage
q1 15.01 18.13 3.123 120.80%
q2 18.51 14.70 -3.808 79.42%
q3 4.37 4.30 -0.064 98.54%
q4 62.88 61.79 -1.096 98.26%
q5 8.09 7.96 -0.127 98.43%
q6 3.97 3.75 -0.225 94.32%
q7 4.25 5.67 1.412 133.18%
q8 4.98 3.77 -1.211 75.70%
q9 16.32 18.07 1.747 110.70%
q10 9.08 9.46 0.377 104.15%
q11 36.06 34.82 -1.236 96.57%
q12 3.13 2.40 -0.730 76.66%
q13 5.83 5.58 -0.253 95.67%
q14a 42.08 44.17 2.092 104.97%
q14b 43.64 43.23 -0.414 99.05%
q15 2.44 2.47 0.037 101.53%
q16 38.94 39.90 0.954 102.45%
q17 4.67 7.15 2.483 153.21%
q18 5.78 5.97 0.193 103.34%
q19 3.20 2.18 -1.020 68.17%
q20 3.87 2.68 -1.188 69.30%
q21 1.13 1.00 -0.133 88.22%
q22 7.91 7.87 -0.041 99.49%
q23a 81.84 81.08 -0.767 99.06%
q23b 102.37 99.94 -2.433 97.62%
q24a 80.55 77.77 -2.776 96.55%
q24b 78.65 69.48 -9.171 88.34%
q25 9.98 4.31 -5.665 43.23%
q26 2.82 2.63 -0.184 93.48%
q27 2.70 4.71 2.006 174.19%
q28 22.49 21.78 -0.706 96.86%
q29 8.32 8.32 0.001 100.01%
q30 4.16 5.58 1.419 134.13%
q31 7.09 5.78 -1.312 81.50%
q32 1.00 1.10 0.096 109.57%
q33 5.73 5.73 -0.004 99.93%
q34 4.97 5.46 0.489 109.85%
q35 6.49 6.64 0.149 102.30%
q36 3.13 3.11 -0.014 99.56%
q37 3.91 3.84 -0.069 98.23%
q38 11.85 11.79 -0.051 99.57%
q39a 3.24 3.27 0.030 100.92%
q39b 3.13 3.29 0.164 105.25%
q40 3.56 3.61 0.052 101.46%
q41 0.59 0.55 -0.048 91.85%
q42 0.81 0.83 0.014 101.78%
q43 3.49 3.59 0.093 102.65%
q44 6.38 7.10 0.722 111.32%
q45 3.28 3.53 0.250 107.61%
q46 3.02 3.11 0.086 102.84%
q47 14.21 17.94 3.729 126.25%
q48 4.39 4.84 0.443 110.09%
q49 7.21 7.39 0.180 102.49%
q50 22.36 30.92 8.563 138.30%
q51 8.46 8.59 0.138 101.63%
q52 0.94 1.12 0.181 119.14%
q53 1.62 1.75 0.123 107.56%
q54 3.18 3.11 -0.066 97.93%
q55 0.92 1.04 0.114 112.41%
q56 5.39 5.35 -0.038 99.29%
q57 8.41 8.49 0.080 100.96%
q58 2.48 2.39 -0.089 96.41%
q59 14.17 15.43 1.259 108.88%
q60 8.43 5.87 -2.561 69.62%
q61 6.11 8.72 2.608 142.66%
q62 3.59 3.90 0.306 108.53%
q63 2.99 1.93 -1.061 64.53%
q64 50.12 48.79 -1.338 97.33%
q65 13.67 13.50 -0.167 98.78%
q66 2.81 2.95 0.133 104.74%
q67 358.17 350.79 -7.382 97.94%
q68 6.56 4.02 -2.547 61.19%
q69 10.27 7.71 -2.551 75.15%
q70 7.45 8.44 0.983 113.19%
q71 2.21 4.29 2.084 194.46%
q72 190.82 189.56 -1.262 99.34%
q73 2.19 2.15 -0.041 98.14%
q74 20.97 20.93 -0.042 99.80%
q75 24.74 25.12 0.371 101.50%
q76 7.14 8.64 1.498 120.99%
q77 1.68 1.76 0.080 104.78%
q78 38.38 38.03 -0.352 99.08%
q79 3.32 3.47 0.155 104.67%
q80 11.51 11.54 0.025 100.22%
q81 5.10 5.67 0.563 111.03%
q82 6.23 6.27 0.041 100.65%
q83 1.47 1.56 0.098 106.66%
q84 3.10 2.63 -0.473 84.73%
q85 7.34 7.92 0.584 107.95%
q86 3.21 2.95 -0.260 91.90%
q87 12.12 13.32 1.205 109.94%
q88 16.80 16.55 -0.256 98.48%
q89 4.26 2.74 -1.518 64.33%
q90 6.66 6.45 -0.214 96.79%
q91 2.63 2.60 -0.036 98.65%
q92 1.12 1.26 0.142 112.75%
q93 28.65 28.79 0.135 100.47%
q94 21.29 21.15 -0.132 99.38%
q9 87.06 86.57 -0.494 99.43%
q5 3.01 2.51 -0.507 83.19%
q96 12.26 12.37 0.117 100.95%
q97 1.88 1.86 -0.027 98.58%
q98 8.25 8.93 0.683 108.27%
q99 8.25 8.93 0.683 108.27%
total 1911.03 1897.48 -13.549 99.29%

@jinchengchenghh
Copy link
Contributor Author

/Benchmark Velox

@zhouyuan zhouyuan merged commit 4592e07 into apache:main May 28, 2024
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants