From 9661e04b3b02491989346f5ec9c412d19a5f47c2 Mon Sep 17 00:00:00 2001 From: Sean Sheng Date: Thu, 25 Apr 2024 05:19:46 -0700 Subject: [PATCH] docs: update autoscaling (#4691) * docs: update autoscaling * docs: add suggestion for configuring concurency --- .../_static/img/guides/autoscaling/hpa.png | Bin 0 -> 10569 bytes .../source/bentocloud/how-tos/autoscaling.rst | 49 ++++++++++++------ 2 files changed, 33 insertions(+), 16 deletions(-) create mode 100644 docs/source/_static/img/guides/autoscaling/hpa.png diff --git a/docs/source/_static/img/guides/autoscaling/hpa.png b/docs/source/_static/img/guides/autoscaling/hpa.png new file mode 100644 index 0000000000000000000000000000000000000000..5b7686c9162b787996b96b3c56fb2fce5c2adbf1 GIT binary patch literal 10569 zcmYjXbzGC}*QeP4#|Y^L38~SeyB-=8>2M$=FhUxsQ8F4-T0%kzkq~J{N|%(>K)OpB z-ue7K@B4fA$L@X3?sIl_;(M;^zP_>gy6O*z=!meeupVe?s2E~l;V@u6s{jZv@BPiY zBbW=em!Y~6R^>3m78Vu@mZpm06F;lH1^gt@Z({^DlG)I~L2ZyoMCahYuiY=5zahQ0 z@b?)R866?AMj_T)sSdsl2R?uPxS2n-&I$e|&E8UI(6;@$scyQ1kU0qc<1K&#Csu(i z>F=dI2C$osfT?(&jQ_7lfo+sE3Bk^#^q)Ka-iS<4AWs-$9j5=UB1Vvn`(G7^6pkiT zEgI_nuhst;*Mxe4{;SZw3E)7}Fs?K!{?{%Cx=-@|DHO6NAz^*ZB^X;bkTJCwXga1*SwZ)&sz=KP{FF)Kl&@&wH z%Q_VnG7Y?9*2B?gx!Rj`i8h{*pmh;neEqiTg@gW)kh>E%?G5?+%KAMlj0duzLmsALlLPh}PtAQIcUGpCKHa=7W?$9vXYQf$Qpo-7 zfR@#9%JJQKTieYFbsKlD^y1MUn(2`H>!ss{UfIg#tDjW@-&*!u2k-^pVV;99rkbEl zW#-4$m;KNBTQEL)ww~lv1#M|rKUSAnA@jxOlnw^#y*^GI2wXhLJx=g#hVgJffW5kT zLbLanD46izfn*NbJ+FT)wK8~*Dz0&>OZIOD2mbtQ^|aErWheeMv&03)5tCt``WnV? z_|5gwy})!sk63@+{mq73?|j1bblcsl@BL3J4dsqv_0rmI_uMt1Z;xwVEOpIRqj zieSE3p0%D?;YxwOBx}D3_&3pE?aX(F(Zeh!cz{QIg$w&W-*`v-hrQlUtM=4twQ#Jtw&>k-v5Fa7D>A z#;`MPX)M&;W0L#MeDV10S|~1vTmJHgUgHwuIwX1vMSDt#qWK#@X-n^UNTulc#XXOt3CD3jjkWiEy>gWTZY$-b(^ZiAyw9r;9 z=cI}R9Kp_Dy9(Rqn)JjJNTbDJI5nTT{MDGm9p}x;!Iie3GoSSZidxE&Le>b);-DZ@t{bV5+V5Jvw zHSrF{uu2mdI^~N|Eb{=OWwNkURgb8{=Ye zHCpc{cPes)_R4JCj^4P##HXx@APC*QTHF?Lv&uYRkQgu-8^AWR*&FZXT__M}U;Y*W z0tp`zvhdZK(c`H#j{G^^u*Vvg=->tm)epC z{v+nCm>Q=tRX7M-y8hF?ODKP#_nwt!bfR z9+S)MzM-6hpp68X%X^Gx^a~=V$mF)^)<1P?{v&}_=Q8m?QxO5CC@xGNt6pqfV3JMn zwl(Cl`NF48d=J*^&Mw2*f%K5x&n*SJd zlC`u;;!^v!MA!D*LjoOc*d`(+6#v8x;>4pHFNfT%XpUQbW(Tb5vww4N&9X4aMSOrX zh@lm9XneHeww7M$7tYeBl-VNmeW? z8Oy!hb?_aTkdHr)IPY>`{_`c;HE8&KB?h;AS4Y8*tmynMTcY&Y8ul@zW8HeqNWQ)J znmT^EYi2)Sx@kCoVGR>jz4Jk`k1StA+C(=9Q{aO@lh-ns=|vt!gk!J=v#BI12^pDL;C- z!zPsZI4PQ8rFwgW-_0XV;H5zz-uww>pt(H@`M57w@B?@KM2SxZC}Z}wGS9h5e7kwA zs)_Oa|Imd;JOLi4+>)yhjfs-7*>%B=+X>~q5I<7w>lgD=Wh0Sraiinv5D`rHGErbo zo}kK}Ijw~WPMuOHw(Wab8Z{e7BbA+ngV(+erT!9j{)m$Bta-Qe4FqwpVU{YC#!F2^ z>2c+RI&|gAr)$4Vm!K0ym_VY6Y3^pA5GUjMUJPinVSvN1o%ppc1<%X$L*J(yDX1+p*eHHj-^ zTH=J9$I@F1Rqd#967U}ggsgj0G*h?=mCLapnD$>)1->FMn?Y1N#VR9Y5qf@L&bmK= z_m~E2%yHU9M3aL|1(1))S+Z>zNE%#**cf~^QIzn~7NOwadLxITrORPoAKRSPBeIU; zN(3tiHI^cXh!~EnR5H>FCp&LZB?skFtYQM9#42O_%7{j%0Y+s?g37Pg^y9H0L<3fq z?<3!Xc@ZwgO`VMUqsaoclu#nyAI~#Ojv6z(gr89@V8%7wShE#o@RvWtK{mdZT9qvF z2yDYRzT>l!9IuewXY%W#FpaRGu7AG~Rnx8bZ~djhM+|p)>WE&|Ri31e`hif?hv=w? zF=My+g!+Vg>kX;tty&NuT24p>Q?g6ynTUtG`z2!1L7CB`f-J^}_*bw^yBYu^aE#;; zgEt+DSJg5^ zvgWkehBr;ro&9xWA$V9OGn@5X6D5TA@kaaK<%=u;Za$#_HkP|PN@PZmt@25$s;P-q z`_1Cl(py~A0)*8zp$M~W&MX9VxX=e$Tkb7_B-|c4gP0m2M}VL7b|cHm={_}r9Vssa z1ArIw}5iwK?l?MN|6Ul?WR}L`I(NkC2~G|m|I9jhSq0O6eLVf(tnn8TuqD#FQ1-3r|^DRnA%vs z+o%uqs^CZ$40>E6AcNxmRefRy9~+2K&}cY~#K6y}9qghhMA#vTY?kgddGnm?Nzo*m zK|YCeKyaNFD4&fKK$>ta(Sv3Z<$KmRp57Q3C9dqJ7P1iCi|AEfvM(AHWii5g2l-&M z%)eZwFbIY;5+QW%fx-(DCLjBSk23?I-OspjH7UpyH4xvBS|&hF86si%Wx=|qAxauQ z;tG}!nRqq(*08V!7UWmiQzH=&6CjswFfn_sN=sM0 z24z99^__Q$_gBfbr3dQ$hh57JIhx+*?3&bgZhXN1E!p>2&2i zdpc!)`ehTyZ&_U02F}oR?DMcqw3bNc@JE~Y+0yuyhQstlK5eHSI`h@vwPUM zkq(P0wccM(H()%~huD%|i5^!ZD%oW|5Qc5Y&XU>vey5?59iql;jLzIde9xnA?LTe&o5R*ev_FVA@5evlISFoU9PL!zgt5GI! z>Nv5<0FB1EB8Wy*+WgTV?b98Zp;$1sdkmY2v%?{0Et;t(`kiVF>Xt@%k?0jmbEoTx zKZl~CaKmWA=nIun&85A=SQA>)kdlMZDsHfvery+hz5>TG@v~snaAcXAM%oS=*_1vk zpFa#JbE87kX@nP&n{49#lP4_QH=#shu5wg;;f9H7k&ub7o)4wo%)7`sbCK4q=Cta^BI#Py~=yy z-5qvst0oBeV~~0H!QUkIU!sYkYO17uyD^6(xSxTSkI-cMpDCqlzlk@9i;3UX%*nW% zyah%NgB5vVn>0|0ei4xaVF@eH8iLmJHZA-W#ZqAOj&1AA=LVaF0?P6{i&&U|_zIjR zKnDtDRO=mvG0w@P27Fr%gP1@I@Vul2RNjZ@G&Q=#L|+bm)UJ?QKiZrhvA7gNf4QCy zJZRzSIrbem5dziGBM4gWZx0TNZP!eEz&PUVGGqo=Lp}Z?&&LA##4nZH#edce8)rDU zS^?0@W>2B^$q3jiW)XskF5MoauAD!=VQaPuPkaPoXQJcxF^jafAwLxa*o` zHJj59bh0MB#bTK2zB85x`RBq#OHSz**R9j&)^nauq7~{YJ0giJ))eU2@`*u5u&er>iUYoN`R>+F{hSdl&8@>Y=}M;i!0dhY&y8!pCC7b4uBVuSq*u;9wA53 zYmha~DwdZ;X=w0greHO5F|u*Yf{ZXAQN~)D<{)L{I_HG`IkFEAgN;NUMDF#!KR#j@ z=MV0kcfoh)3fkwN#Pz-1h>cB3bU;v=n4gB(Bu1Bjo&}V&oL}urUaNo6?2`Bll&1@^ zbYW{{cTtQ<0eCwCh^)Q>2S2vD3~%e*$o#Vf!6>N4}AFD7$xdJGU<4t!2dptt|6 z2P{-fV$NuoY~Z=5-kJYv)?{ZPNAK}A*mja9!L zk(DVY5*v5f*Rc0y5mnh18rKlk%7pX$ld?owY*lRE&Az`NU|D=@o#e`J0;x% z8_%aZLs{uH!jBBi17l(m{mg&=!>)w{u^U7?Udb@zhMh{-vCti_HTFtM&2EH81k#aW2|sk!`Q-@zaq~`E(HYJRbpo(5`-YJ`-s5@U9{HJYu``Cl#5b z;h=i+JHoEYz80XiE!RL~heYq~xAlc%0XDnIoiQ^M+8E2k>ujA zpSdlfEShBaGpTAJZ}(#}tf;(vCL#S~ z`%Vtk0auX|CJn<8_xWD=i^@EJd`(<{T!@pZkmZlqu5}h8>ca z{^rtgwgRds-32h16g_5EF4Ncbo3t}xfhj^y4RV{lKcxUo+k*WmXk>7-e+{)weHfjhom& zI39WdIexX@8mUV4#D<+e*#M->yrgcN^kOR;Ewn>icDOIJs|dBB;Yyr3D6Z?c?xVng zoiJ*?@s?X;FEG8WlRSJ0_MMr|kXBxQAxACX>lrh>K|tVXM-%$ z(qA$IojP?D|?LdMjgV&1l1ZJgqa-D=9w~q&THdpRvpr4Q?ASaxG=Y9 z6A98`6kQY{dYSPJta!CXBZiUVn@7V+G&tdcNe`bL%nza>dM95+r)<+b)iJ3I+{IeS%;&QsUF{4JDgv`vD zwEK+wcukxY=dC?Ffm9 zaxy%0h{$|)f-rEFsSsOyko$qPH{@D?bgr@J>qtP!$-#hn`&PYj<|dx!$lH;AKiWt* z;`4WL?Xjb`@R{xyz4P=d`^~P-WHJG`ka^QfX)5XFOdTCIhWT5PmkSoXNQ%qMcl= zP-JSAEyWHi?ztfg2+f-Z`EC<^yYoYFnQ%z1n1L9`3TXAvwFc4Y8wq%7%<;9Nlet-L zv99%r^P=KGmUt{&1)$i#;ccJ{AS@94%`*tygNvL9G~2BgpPO`Wb4bd3TrW(D57(Dr zRx;bRQQ36Z3{u`Nce-*HfpiT_SxL)47EAKEJlVotl{z_woOs2yvwnRd! zPJ($p@T0%5*xXL6T>6++|EiuK8(Dix$(yL^NseB&cM_or2V*-M;_JuZ1S29pxt@?G z9G1fG=UCO0-=B`4pcRaPI&B!7iw-4K44`K zXWp0PhY`@C`46oS#hpPEZ%I(LvWox2jQZ2cvfHtGL&^SRirxEt_hU(RbOE`i-ZKf4 z?zpIHbTeeWy%qi}e|fHA^AjB7Yl)-OoS1qAeG{$xGS5Jb(@}i;MKVhfGU*>kh}#}m zuxG%OktA;054o`& zZQ-iyFhZCF>g^OGn1uO;4`Z~o~4`Yag zHvnOMj@^n1uN~BR6JBC}!cQU*n)ZyL2%bW@osY~2A0oXNC+xW&2n>gPc=QE6g$8a1 zpd-YKR<7^@SE{M1;VyCK<1YZyCf;Nqfr~lbu%WTXhy9ZU;?tAzpZx9e-Ke;mfz+G%fsU? zH`1^##J`oIazcCt6rsDI??<#&Y|3bgOgg60%(N}sWP3X>-*tDUXc?`Z6=;MyHom6; zB@7T8@7e4YvDJ^KFsPKZnqTl4Wi4>+hc7GSXR=B|gYK&v%OfCKpRGI!$<>N|*czw;ReYQRd)iZB~Xw z;9?R~E8GFw$>EF3w{l-i?#kEd6%>nAnSHeK$qhR_4Yn{{e`-HXCyi6tm{3U|sU1ze z=xZs4k#Z^*dWtHoEdD7uX}pP={gW3jqr6!Z7q#d=s^+oR?Vj@~7`e|h)y zWKF5DGq^cRByz`~g_CNrkUX4r=cqaje>cVRaHG&2=n>m_OZOXIS&M5Qr#`~Mp=F^y zACS*B6hmD=N*~x|-h^Wi4D(9>m`ciAa+cWV_|r1??Zo2@~wT{yyq2fC8J^eXmcZpqcR? z@1%2S_(CHuV66fY?}>Y&^2$gpd4_7;1Cp{4s&YtfI`dOLEE!3WK?xCMlHcRBmz+^# z3%^P@y>x1(^1C=O5%R7%uI>%8FVS)K8f_*MNB;m!w|JEk3B2f4C0LqclQ*pN0_=MjJtob-Tl9NhvT zU9qg$k$9GrJ!<7AL1WoPz{--X%C=otN@m68b_qXYXknB)_OnsgE92y?^?8 zs)ED36KN5@?wgX6!Od?M@&;uiB1AnRdh&dkKCUQQ*2cKkwCv-d`rWZ1N8ijkv2lqj zEw|Wod(MK8(Q|EjUaXHG^Yh74KcK=&B+8poNvZ_>HL;?7)m&FaJER$t5X49AXw8w) zO7piVBZr0=gzK(Zi_PgC0o2XRl}%3*!3*(1uEl2D2ee;fphV`AL$8uqZTk+NcP+U* zh?c}+_o^A%og-0`kp{4)jQfs%tY+IkvT9Tx)H#`J8tNn7xd_3tC%c^+GzsdxLZwbQ z2Z#r@+*q-FAp*>NVcD1xTD)>{?VDp*%4-_md>mR$e+AJLZKUrJ@1a@Gn1s!TEWr6+EoY-cKE$s?^Ly zW&Sg)w1T}xlTI*#e?sH2cMMMKEBk4(j{TYuIZHs#@9|H4 z2al~pNM^XtRit-^=9&{MG;SA!r}5|gZUt1B}xm>CY3HzoN~9P$eKBJ61R=a<9gHY(~3_T0kfg-#8x{E4?eD+6fuNnoc zN0L4#8a}gK0~(0x1+(jzVn?+-V$eV|ii)8}2?J==n&tMOS1cA0+_fbcpW_RC;_s$d zIqE!@$-A-qrNO*K?3olJ52;k3eqSdO$E>Xq8=b4YI)cPetx6(FhT~d=#%W8moud(V z^)8F0S)I6IJ3*g@P6dIzFZ2fv{-wSg-UCXV$ZhfWPoKKXU;i_zCxvYq;_u{0^OuXE z?49SUAXE!c01%Q!7PyRsL%Z^$4XUSoDUn%|1&DinRL2)RsT&Ylp>!_d6dCUpiPh*P zL*6Xz0irB@5O-RRbtb=>DE8O#x>3y%dT0%gqX-KWOQ{nsRHY0;+-y#{Ft_?4RL`LAn6Z z;htwE5%4ajpP#5F%m>LQhP9x68cCV0QSj26`P|$5m5R~O!?D^ZxL;JCN-Yb6MnaMf zgR^)S3%lT{5P&(zKj{VIS?hvV^mLTCD!NZK+6ajuRQSlFn?o>1E4zMpj_foqdLX)H zR_Th}mGf>{G1XN(VsK2an(DjnfMhET?wP#i)hpX(4zp}9?a9=;(o53nonY#&E)C>Fn^30s=lmt+xZwE_W@l>& zw%b%1)-?G-I2LeZqd7l5jR{?Tn(FrH4&RpDx6xbAu8TF`6fnkM!y}{2L7TyAkl1y` zz+DE^gPe{(Tjh8z#$=}eX{R%IO!{W>S1C+shlRb{=_errygkY#uY=R=k+$oA>wpUD zxE*9CwY_izujb0-MiF2)(9R)axSwbP3{!zBe&P>jRiTW&(yV@1oP0)3sB%2XLhWh> zF(nW&Dzd0#LKudz@vEU22j=Xj(&%y;N6;-_pfkel?AbS| zSpVGOPHiAmVe(@OV<=5YBs*u1J;N3ELmJ7Q3E9%`*b;f>KRG8J>*o7FSN=b@XAf*s zWG@C@6q=eP$IcMb==~&Klyt6EuN(O3wBQ;uF2-Wk&;GKxD#(&rY`^jDetGjsa_S1r zXM4DvIa&(k8eo8D!YRqkQF|M$kvPtVc{^Wb=c#EgA~_gv!}=2+{iWMMD6z$u5qJph zt6^T+fSjB8VP%ELCsO257Be3O!1d0BWc`MDs{jsGwPg~1pcAxS%7`aenLU*6sjmjX!sU$(;EnkK-|0pEssF0%ih#0xJ4 zUVe$!hvxcU^5K923$RyMe>m{}NP;eZd4^(nSdusY{`lXJ(*kaoBkdUb6?doJV7QT3 NnyR`gl}a|@{{tPn4=w-z literal 0 HcmV?d00001 diff --git a/docs/source/bentocloud/how-tos/autoscaling.rst b/docs/source/bentocloud/how-tos/autoscaling.rst index 0f602e7851f..f9d699ce8ab 100644 --- a/docs/source/bentocloud/how-tos/autoscaling.rst +++ b/docs/source/bentocloud/how-tos/autoscaling.rst @@ -6,17 +6,21 @@ BentoCloud autoscales your Bento Deployments to efficiently handle varying loads This document explains the autoscaling feature of BentoCloud. -Configure autoscaling ---------------------- +Replicas +-------- -You can set the :ref:`minimum and maximum values ` to define the boundaries for scaling, allowing the autoscaler to reduce or increase the number of replicas as needed. This feature supports scaling to zero replica during idle time. - -To enable better control over autoscaling, you can set concurrency and external queue. +You can set the :ref:`minimum and maximum replicas ` to define the boundaries for scaling, allowing the autoscaler to reduce or increase the number of replicas as needed. This feature supports scaling to zero replica during idle time. Concurrency -^^^^^^^^^^^ +----------- + +To enable autoscaling, first configure the ``concurrency`` configuration for the service. :doc:`/guides/concurrency` refers to the number of concurrent requests of a BentoML Service is able to to process simultaneously. Setting this parameter means the Service will be automatically scaled on BentoCloud when the concurrent requests per replica exceeds the specified concurrency threshold. + +For instance, consider a scenario where ``concurrency`` is set to 32 and the service is currently operating with 2 replicas. If the service receives 100 concurrent requests, BentoCloud will automatically scale up to 4 replicas to effectively manage the increased traffic. Conversely, if the number of concurrent requests decreases to below 32, BentoCloud will intelligently scale down to 1 replica to optimize resource utilization. + +In general, the autoscaler will scale the number of replicas based on the following formula, permitted by the ``min_replicas`` and ``max_replicas`` settings in the deployment: -:doc:`/guides/concurrency` refers to the ability of a BentoML Service to process multiple requests simultaneously. Setting this parameter means the Service will be automatically scaled on BentoCloud when the average requests per replica exceeds the specified concurrency threshold. +.. image:: ../../_static/img/guides/autoscaling/hpa.png Use the ``@bentoml.service`` decorator to set concurrency: @@ -24,14 +28,28 @@ Use the ``@bentoml.service`` decorator to set concurrency: @bentoml.service( traffic={ - "concurrency": 3, # An integer value + "concurrency": 32, # An integer value } ) class MyService: ... +.. warning:: + + If ``concurrency`` is not set, the Service will only be autoscaled based on CPU utilization, which may not be optimal for your Service. + +To determine the optimal value for ``concurrency``, we recommend conducting a stress test on your service using a load generation tool such as `Locust `_ either locally or on BentoCloud. The purpose of the stress test is to identify the maximum number of concurrent requests your service can manage. After identifying this maximum, set the concurrency parameter to a value slightly below this threshold ensuring that the service has adequate headroom to handle traffic fluctuations. + External queue -^^^^^^^^^^^^^^ +-------------- + +You can also configure to enable an external request queue to buffer incoming requests. This feature is useful when you want to prevent overloading the Service with requests that exceed the concurrency threshold. + +When the external queue is enabled, BentoCloud will hold excess requests in the queue until the Service has the capacity to process them. This ensures that the Service never receives more requests simultaneously than the ``concurrency`` setting allows. BentoCloud will automatically scale the number of replicas based on the number of requests in the queue. + +.. note:: + + Using the external queue increases the latency of the Service because the extra IO operations are needed to handle the requests in the queue. You can enhance concurrency management with an external request queue on BentoCloud using the ``@bentoml.service`` decorator: @@ -39,19 +57,18 @@ You can enhance concurrency management with an external request queue on BentoCl @bentoml.service( traffic={ - "concurrency": 3, # An integer value + "concurrency": 32, # An integer value "external_queue": True, # A BentoCloud-only field. If set to true, BentoCloud will use an external queue to handle excess requests } ) class MyService: ... -The external request queue is used to moderate incoming traffic, ensuring that a Service instance never receives more requests simultaneously than the ``concurrency`` setting allows. Excess requests are held in the queue until the Service has the capacity to process them, preventing overloading and maintaining efficient operation. - .. note:: - - If you enable ``external_queue`` in the ``@bentoml.service`` decorator, you must specify a ``concurrency`` value. - - ``max_concurrency`` does not take effect on BentoCloud. You need to enable ``external_queue`` to handle excess requests. + If you enable ``external_queue`` in the ``@bentoml.service`` decorator, you must specify a ``concurrency`` value. + +It's worth noting that when external queue is enabled, ``max_concurrency`` will no longer take effect because BentoCloud guarantees the Service will never receive more requests simultaneously than the ``concurrency`` setting allows. Therefore, ``max_concurrency`` is never reached. Autoscaling policies -------------------- @@ -61,13 +78,13 @@ You can customize scaling behavior to match your Service's needs with scaling-up Allowed scaling-up policies (``scale_up_behavior``): - ``fast`` (default): There is no stabilization window, so the autoscaler can increase the number of replicas immediately if necessary. It can increase the number of replicas by 100% or by 4 replicas, whichever is higher, every 15 seconds. -- ``stable``: The autoscaler can increase the number of replicas, but it will stabilize the number of replicas for 300 seconds (5 minutes) before deciding to scale up further. It can increase the number of replicas by 100% every 15 seconds. +- ``stable``: The autoscaler can increase the number of replicas, but it will stabilize the number of replicas for 600 seconds (10 minutes) before deciding to scale up further. It can increase the number of replicas by 100% every 15 seconds. - ``disabled``: Scaling-up is turned off. Allowed scaling-down policies (``scale_down_behavior``): - ``fast``: There is no stabilization window, so the autoscaler can reduce the number of replicas immediately if necessary. It can decrease the number of replicas by 100% or by 4 replicas, whichever is higher, every 15 seconds. -- ``stable`` (default): The autoscaler can reduce the number of replicas, but it will stabilize the number of replicas for 300 seconds (5 minutes) before deciding to scale down further. It can decrease the number of replicas by 100% every 15 seconds. +- ``stable`` (default): The autoscaler can reduce the number of replicas, but it will stabilize the number of replicas for 600 seconds (10 minutes) before deciding to scale down further. It can decrease the number of replicas by 100% every 15 seconds. - ``disabled``: Scaling-down is turned off. To set autoscaling policies, you need to configure the above fields in a separate YAML or JSON file. For example: