[#22161] DocDB: Gate new memory default recommendations on an ordinar…

…y gflag

Summary:
Here we introduce a new gflag:
```
~/code/yugabyte-db/src/yb/tserver/server_main_util.cc:36:
DEFINE_NON_RUNTIME_bool(
    use_memory_defaults_optimized_for_ysql, false,
    "If true, the recommended defaults for the memory usage settings take into account the amount "
    "of RAM and cores available and are optimized for using YSQL.  "
    "If false, the recommended defaults will be the old defaults, which are more suitable "
    "for YCQL but do not take in account the amount of RAM and cores available.");
```
to gate the new memory recommendations.

More precisely, we still have a concept of recommended defaults and individual memory settings can choose whether or not to use the recommended defaults.  For example:
```
~/code/yugabyte-db/src/yb/tserver/tablet_memory_manager.cc:59:
// NOTE: The default here is for tools and tests; the actual defaults
// for the TServer and master processes are set in server_main_util.cc.
DEFINE_NON_RUNTIME_int32(tablet_overhead_size_percentage, 0,
    "Percentage of total available memory to use for tablet-related overheads. A value of 0 means "
    "no limit. Must be between 0 and 100 inclusive. Exception: "
    BOOST_PP_STRINGIZE(USE_RECOMMENDED_MEMORY_VALUE) " specifies to instead use a "
    "recommended value determined in part by the amount of RAM available.");
```
If this gflag is set to USE_RECOMMENDED_MEMORY_VALUE by the customer or left unset (the gflag default is set to that value for the TServer and master processes elsewhere), then we will use the recommended default for it.

If the new gflag is set to true then we use the new recommendations we have derived using our models.  If it is set to false, then we use the same values we would've used prior to version 2024.1 if the customer had not provided a value for this gflag.

Thus, the customer gets essentially the same behavior as before 2024.1 if they do not set the new gflag.
Jira: DB-11090

Test Plan:
Started cluster with and without flag on and look at logs:
```
~/code/yugabyte-db/bin/yb-ctl start

I0425 10:52:25.180203  5935 server_main_util.cc:90] Setting flag db_block_cache_size_percentage to recommended value -3
I0425 10:52:25.180212  5935 server_main_util.cc:92] Flag default_memory_limit_to_ram_ratio has value 0.65 (recommended value is 0.85)
I0425 10:52:25.180217  5935 server_main_util.cc:94] Setting flag tablet_overhead_size_percentage to recommended value 0
```

```
bin/yb-ctl start --tserver_flags "use_memory_defaults_optimized_for_ysql=true" \
    --master_flag "use_memory_defaults_optimized_for_ysql=true"

I0425 10:57:00.198920  6412 server_main_util.cc:101] Total available RAM is 31.183224 GiB
I0425 10:57:00.198928  6412 server_main_util.cc:90] Setting flag db_block_cache_size_percentage to recommended value 32
I0425 10:57:00.198938  6412 server_main_util.cc:92] Flag default_memory_limit_to_ram_ratio has value 0.65 (recommended value is 0.6)
I0425 10:57:00.198943  6412 server_main_util.cc:94] Setting flag tablet_overhead_size_percentage to recommended value 10
```
(yb-ctl overrides default_memory_limit_to_ram_ratio.)

Reviewers: zdrudi

Reviewed By: zdrudi

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D34547

src/yb/tserver/server_main_util.cc

@@ -33,6 +33,13 @@
 #include "yb/util/size_literals.h"
 #include "yb/util/status.h"
 
+DEFINE_NON_RUNTIME_bool(
+    use_memory_defaults_optimized_for_ysql, false,
+    "If true, the recommended defaults for the memory usage settings take into account the amount "
+    "of RAM and cores available and are optimized for using YSQL.  "
+    "If false, the recommended defaults will be the old defaults, which are more suitable "
+    "for YCQL but do not take into account the amount of RAM and cores available.");
+
 DECLARE_double(default_memory_limit_to_ram_ratio);
 DECLARE_int32(db_block_cache_size_percentage);
 DECLARE_int32(tablet_overhead_size_percentage);
@@ -97,16 +104,18 @@ void AdjustMemoryLimitsIfNeeded(bool is_master) {
 
   auto values = GetLegacyMemoryValues(is_master);
 
-  values.tablet_overhead_size_percentage = is_master ? 0 : 10;
-  values.db_block_cache_size_percentage = is_master ? 25 : 32;
-  if (total_ram <= 4_GB) {
-    values.default_memory_limit_to_ram_ratio = is_master ? 0.20 : 0.45;
-  } else if (total_ram <= 8_GB) {
-    values.default_memory_limit_to_ram_ratio = is_master ? 0.15 : 0.48;
-  } else if (total_ram <= 16_GB) {
-    values.default_memory_limit_to_ram_ratio = is_master ? 0.10 : 0.57;
-  } else {
-    values.default_memory_limit_to_ram_ratio = is_master ? 0.10 : 0.60;
+  if (FLAGS_use_memory_defaults_optimized_for_ysql) {
+    values.tablet_overhead_size_percentage = is_master ? 0 : 10;
+    values.db_block_cache_size_percentage = is_master ? 25 : 32;
+    if (total_ram <= 4_GB) {
+      values.default_memory_limit_to_ram_ratio = is_master ? 0.20 : 0.45;
+    } else if (total_ram <= 8_GB) {
+      values.default_memory_limit_to_ram_ratio = is_master ? 0.15 : 0.48;
+    } else if (total_ram <= 16_GB) {
+      values.default_memory_limit_to_ram_ratio = is_master ? 0.10 : 0.57;
+    } else {
+      values.default_memory_limit_to_ram_ratio = is_master ? 0.10 : 0.60;
+    }
   }
 
   AdjustMemoryLimits(values);