How can I set the gpu_memory_utilization?

[Bearljj]

I encountered this error when running the code:

ValueError: No available memory for the cache blocks. Try increasing gpu_memory_utilization when initializing the engine.

But I have already made these settings

bonito = Bonito("BatsResearch/bonito-v1",gpu_memory_utilization=0.9)

, it seems like they are not working, what should I do?

[nihalnayak]

This is related to the issue in the vllm package (vllm-project/vllm#2248).

You could try the following with Bonito and see if that helps:

bonito = Bonito("BatsResearch/bonito-v1", enforce_eager=True)