-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[VL] Add config for memory pool init capacity to reduce arbitration times #5815
[VL] Add config for memory pool init capacity to reduce arbitration times #5815
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
Run Gluten Clickhouse CI |
6b92da8
to
12d4024
Compare
Run Gluten Clickhouse CI |
We previously hardcoded it as 8M, this PR make it configurable, right? |
preciously is 0. |
Is there any difference on performance in your test observed within the change? @Yohahaha |
currently, no, but it can reduce the frequency of enter arbitration in theory, and may have benefits when set this config larger, eg 128M. |
shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala
Outdated
Show resolved
Hide resolved
12d4024
to
6cb4ef2
Compare
Run Gluten Clickhouse CI |
/Benchmark Velox |
1 similar comment
/Benchmark Velox |
===== Performance report for TPCH SF2000 with Velox backend, for reference only ====
|
What changes were proposed in this pull request?
Add new config
spark.gluten.sql.columnar.backend.velox.memInitCapacity
to reduce memory arbitration times, default value is 8M.