[native] Add reserved memory capacity configs #22593
Conversation
xiaoxmeng
changed the title
xiaoxmeng
force-pushed
the
config
branch
9 times, most recently
from
9a53624
to
3d60655
Compare
xiaoxmeng
dismissed stale reviews from tanjialiang, NikhilCollooru, and shrinidhijoshi
via
a46a8b6
| @@ -462,12 +463,22 @@ int32_t SystemConfig::queryMemoryGb() const { | |||
| return optionalProperty<int32_t>(kQueryMemoryGb).value(); | |||
| } | |||
|
|
|||
| int32_t SystemConfig::queryReservedMemoryGb() const { | |||
There was a problem hiding this comment.
@xiaoxmeng :
Thanks for your code changes.
Would be great to have some examples for these configs. Do we have Velox docs for them already ? Please can you add Prestissimo documentation for them.
| @@ -462,12 +463,22 @@ int32_t SystemConfig::queryMemoryGb() const { | |||
| return optionalProperty<int32_t>(kQueryMemoryGb).value(); | |||
| } | |||
|
|
|||
| int32_t SystemConfig::queryReservedMemoryGb() const { | |||
There was a problem hiding this comment.
Why is this 32 bit vs 64 bit usage in other places ? Might be good to keep it uniform.
There was a problem hiding this comment.
It is just to be the same type as queryMemoryGb. We could uniform these config settings in followup. Thanks!
| @@ -256,6 +256,17 @@ class SystemConfig : public ConfigBase { | |||
| /// this config only applies if the memory arbitration has been enabled. | |||
| static constexpr std::string_view kQueryMemoryGb{"query-memory-gb"}; | |||
|
|
|||
| /// Specifies the amount of query memory capacity reserved to ensure that a | |||
There was a problem hiding this comment.
How would you typically determine this value ? Whats a good back of the envelope calculation for this ?
There was a problem hiding this comment.
It can be based on the number of concurrent running queries * reserved memory pool capacity.
| @@ -256,6 +256,17 @@ class SystemConfig : public ConfigBase { | |||
| /// this config only applies if the memory arbitration has been enabled. | |||
| static constexpr std::string_view kQueryMemoryGb{"query-memory-gb"}; | |||
|
|
|||
| /// Specifies the amount of query memory capacity reserved to ensure that a | |||
| /// query has minimal memory capacity to run. This capacity should be less | |||
| /// than 'query-memory-gb'. A query's minimal memory capacity is specified by | |||
There was a problem hiding this comment.
The relation between 'query-reserved-memory-gb' and 'memory-pool-reserved-capacity' is a bit difficult to follow. Can you elaborate ?
xiaoxmeng
force-pushed
the
config
branch
2 times, most recently
from
e181dde
to
fd89c5e
Compare
We have velox documentation for these configs. |
Add two system configs for reserved query memory capacity: query-reserved-memory-gb: the total amount of query memory capacity reserved to ensure that a query has minimal memory capacity to run memory-pool-reserved-capacity: the minimal amount of memory capacity in bytes reserved for each query memory pool.
Add two system configs for reserved query memory capacity:
query-reserved-memory-gb: the total amount of query memory capacity reserved to ensure that a
query has minimal memory capacity to run
memory-pool-reserved-capacity: the minimal amount of memory capacity in bytes reserved for
each query memory pool.