Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(amd): support ROCm detection #950

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Conversation

younseojava
Copy link

This is first attempt to enable ROCm for AMD GPUs. It consists of

  • GPU detection with rocm_smi_lib to check non-default location of ROCm library
  • Add trust_remote_code option for Baichuan model
  • Skip test for gpu_validate

@aarnphm
Copy link
Member

aarnphm commented May 23, 2024

Hi @younseojava, thanks for the PR. For 0.5, we have simplify a lot of the internal logics and decided to use vLLM as the current running backend for now. Which means AMD support will come from vLLM.

I will close the PR for now. Looking forward for future collab :)

@aarnphm aarnphm closed this May 23, 2024
@aarnphm aarnphm reopened this May 26, 2024
@aarnphm
Copy link
Member

aarnphm commented May 26, 2024

Actually, there is a code path that we are still using this; I will reopen this.

Can you fix the merge conflict? ths.

@aarnphm aarnphm changed the title ROCm enablement feat(amd): support ROCm detection May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants