ARGO: an easy-to-use runtime to improve GNN training performance on multi-core processors #8841
Conversation
for more information, see https://pre-commit.ci
jasonlin316
changed the title
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
This is great thanks for adding!
Need your help with a few things to merge this PR
- Add a link to the paper and also give a short explanation of how ARGO speeds things up.
- Could you share some benchmarks on pyg models. Showing how much ARGO speeds up py models on high core cpus?
- Add type hints to all functions and also a short doc for each function.
examples/argo.py
Outdated
| self.acq_func = 'EI' # acqusition function of the auto-tuner | ||
| self.counter = [0] | ||
|
|
||
| def core_binder(self, num_cpu_proc, n_samp, n_train, rank): |
examples/argo.py
Outdated
| acq_func=self.acq_func) | ||
| return result | ||
|
|
||
| def mp_engine(self, x, train, args, ep): # Multi-Process Engine |
There was a problem hiding this comment.
Thanks for your suggestions!
I added type hints to each function and provided a short doc.
The description of ARGO is available on top of the file now.
For benchmarking results, they are available in the paper (Fig. 10 and 11).
Please let me know if you need anything else from me :)
examples/argo.py
Outdated
| Parameters | ||
| ---------- |
There was a problem hiding this comment.
Can we follow the same docstring style as the PyG codebase?
| Parameters | |
| ---------- | |
| Args: |
There was a problem hiding this comment.
Hmm...I can change the term to "Args." But I think the PyG doc uses "PARAMETERS"?
For example, the doc here: https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html
There was a problem hiding this comment.
The built docs show "PARAMETERS", but in the codebase, we follow this docstring style afaik: https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings Please feel free to confirm how the docstrings are written in the codebase.
There was a problem hiding this comment.
Understood. I have updated the document accordingly.
examples/argo.py
Outdated
| random_state: int | ||
| Number of random initializations before searching | ||
|
|
||
| acq_function: str |
There was a problem hiding this comment.
Looks like acq_function isn't used. Mind updating the docsting or the code?
Yes. I do have end-to-end example: https://github.com/jasonlin316/ARGO/tree/main/PyG |
Yes, I agree having a complete example is important for the users. |
examples/argo.py
Outdated
| import time | ||
| from typing import Callable, Tuple | ||
|
|
||
| import dgl.multiprocessing as dmp |
There was a problem hiding this comment.
Is this needed? We don't want to have a dgl dependency in PyG. Can't we use Python/PyTorch for this?
|
@jasonlin316 I moved to |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #8841 +/- ##
==========================================
- Coverage 89.26% 89.01% -0.26%
==========================================
Files 468 469 +1
Lines 29960 30045 +85
==========================================
Hits 26744 26744
- Misses 3216 3301 +85 ☔ View full report in Codecov by Sentry. |
Overview
The GNN training performance on multi-core processors is limited as the current design cannot scale well. We propose a runtime system named ARGO that can improve the scalability of GNN training on multi-core processors. On a CPU platform where the original program can only scale to 16 cores (meaning that no performance improvement is achieved if more than 16 cores are applied), ARGO can further scale the design up to 64 cores, achieving up to 5x speedup compared to the original design without ARGO.