We have something very similar explained here: https://karpenter.sh/preview/concepts/scheduling/#savings-plans-and-reserved-instances

Ryan,

While I appreciate this reserved instance approach, this won’t work with “targeted” capacity reservations because those need to be “pointed to” by the EC2 Fleet API. The current approach you mentioned also won’t factor in availability of capacity reservations by AZ even if the reservation is of type “open”.

If I reserve 20 instances of c4.large each in 5 AZs, with a total of 100, those 100 instances can be launched by Karpenter in any AZ without regard for placing them evenly across each AZ. Similarly, if I had 10 reservations available across 4 AZs, there’s no mechanism for Karpenter to place them in those AZs accordingly.

Let me know if I'm missing something.

Thanks,
David

if I had 10 reservations available across 4 AZs, there’s no mechanism for Karpenter to place them in those AZs accordingly.

your provisioner can specify which AZs to deploy to. Worst case, you could have a provisioner per AZ

I understand that, but how do I know I have 10 reservations across 4 AZs? There's no way the provisioner would know those exist, you as the infrastructure engineer would have to know how many are in each AZ, and the number could fluctuate due to other workloads or components taking up those reserved slots. Why not let EC2 Fleet figure it out, instead of putting onus on the infrastructure engineers?

And this still does not address targeted Capacity Reservations. If you have multiple workloads or components of a workload in the same account and have different reservations for each via reservation IDs, you are SOL.

I can add some additions:
as we are hosting our servers in Frankfurt, we see lots of issues with capacity.
So we created capacity reservations and noticed, that karpenter ignores it, and creates instance with defined instance type, but in different AZ.
So, it would be great if there will be way to tell karpenter, that on first priority - load reserved spots, and then - cost(or whatever else)-effective.

I also need the ability to reference targeted on demand capacity requests. We use ODCRs for instance types that are capacity constrained where we need a guarantee of being able to launch those instances.

I think it make sense to support capacity reservation but I am not sure it make sense to go down a priority within the same Node Pool. I think Karpenter has a solve for this already with weights. So you would have your ODCR NodePool at a higher weight and if that fails you would fall into the next node pool (maybe spot) and so on. I think its difficult to support the prioritization because ODCR is tied to a launch template, which Karpenter creates in the back. In order to support this fallback behavior, I think Karpenter would have to have a set of Launch Templates which I suspect is too complicated? Note that Spot is supported because CreateFleet supports choosing the more optimized allocation strategy

Fwiw, we maintain many launch templates under the hood per node pool. It's required for things like labels and architectures.

That said, we model "preferential fallback" using the weight feature, so I don't think it's crazy to reuse that here. However, LT combinatorics don't seem like they'll be a design constraint on this question.

Fwiw, we maintain many launch templates under the hood per node pool. It's required for things like labels and architectures.

Ya I realize I don't actually know that code that well and I was suspecting I was wrong after writing this. But I think I agree that it's likely better to push that logic to weights instead. I am trying to write up a brief design on implementing adding capacity reservation to EC2NodeClasses so I wanted to make sure that we want to keep LTs under nodeclass clean/ minimal for the sake for just launching the right node and not for prioriziation