New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
G-retriever (GNN+LLM) example w/ demo & GNN+LLM integration #9167
base: master
Are you sure you want to change the base?
Conversation
@Kh4L reviews welcome |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9167 +/- ##
==========================================
- Coverage 87.33% 87.21% -0.13%
==========================================
Files 460 477 +17
Lines 30385 31051 +666
==========================================
+ Hits 26536 27080 +544
- Misses 3849 3971 +122 ☔ View full report in Codecov by Sentry. |
…am/pytorch_geometric into gnn-llm-model-integration
for more information, see https://pre-commit.ci
…am/pytorch_geometric into gnn-llm-model-integration
for more information, see https://pre-commit.ci
…am/pytorch_geometric into gnn-llm-model-integration
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
…am/pytorch_geometric into gnn-llm-model-integration
for more information, see https://pre-commit.ci
…am/pytorch_geometric into gnn-llm-model-integration
for more information, see https://pre-commit.ci
…am/pytorch_geometric into gnn-llm-model-integration
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9167 +/- ##
==========================================
- Coverage 87.33% 87.21% -0.13%
==========================================
Files 460 477 +17
Lines 30385 31051 +666
==========================================
+ Hits 26536 27080 +544
- Misses 3849 3971 +122 ☔ View full report in Codecov by Sentry. |
repro:
Latest NVIDIA PyG container
+
git config --global credential.helper store; huggingface-cli login; cd /opt/pyg; pip uninstall -y torch-geometric; rm -rf pytorch_geometric; git clone -b gnn-llm-model-integration https://github.com/pyg-team/pytorch_geometric.git; cd /opt/pyg/pytorch_geometric; pip install .; pip install peft datasets transformers pcst_fast sentencepiece; python3 examples/llm_plus_gnn/g_retriever.py
old PR: #9154
note: pure cpu is 220x slower than pure GPU using a single Grace Hopper
info:
tried gemma, performs worse in all train/val/test metrics. most likely needs some tuning, will leave this as future work as part of the community sprint to try many LLM and GNN combos and tune them. Therefore keeping the default llama2