Skip to content
This repository has been archived by the owner on May 28, 2024. It is now read-only.

Add a example of Azure GPU #118

Open
chenweisomebody126 opened this issue Jan 17, 2024 · 1 comment
Open

Add a example of Azure GPU #118

chenweisomebody126 opened this issue Jan 17, 2024 · 1 comment

Comments

@chenweisomebody126
Copy link

chenweisomebody126 commented Jan 17, 2024

I knew we had example of google cloud and AWS in folder kuberay.
I need to set up Ray-LLM in Azure cluster. I need to at least know cpu and gpu type. For example, When should I use accelerator_type_a10:0.01 and when to use accelerator_type_v100: 1 as in #44.
At least give more explanation of the meaning of accelerator_type_a10 and accelerator_type_v100 so that we can figure out the azure config on our own.
For example, in the accelerator_type_a100_80g, I can understand what V100 means from https://docs.ray.io/en/latest/ray-core/accelerator-types.html, but what does 80g mean?

@rifkybujana
Copy link

I think it refer to the amount of vram the GPU has, as the A100 GPU have variant which has 40G and 80G vram.

alanwguo pushed a commit that referenced this issue Jan 25, 2024
Allows for private models to be used

---------

Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants