Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Fixed sampler with limit on sampled nodes/edges in batch subgraph #6668

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

ayushnoori
Copy link
Contributor

Description

Subgraph sampler that sets an upper bound on the number of nodes included in each layer of the sampled subgraph. At each layer, the frontier is randomly subsampled. Rare node types can also be upsampled by taking the scaled square root of the sampling probabilities (best strategy TBD). The new FixedSampler performs node-wise neighbor sampling and returns the subgraph induced by all the sampled nodes.

The relevant issue is: #6623, thanks to @frozenbugs and @jermainewang for their input and review.

Checklist

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
  • Code is well-documented
  • Related issue is referred in this PR
  • All changes have test coverage

Please note that unit tests for FixedSampler are not yet written.

@dgl-bot
Copy link
Collaborator

dgl-bot commented Dec 4, 2023

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

  • @dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Dec 4, 2023

Commit ID: 928fe21

Build ID: 1

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Dec 28, 2023

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

  • @dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Dec 28, 2023

Commit ID: fc5ce86

Build ID: 2

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
python/dgl/dataloading/fixed.py Outdated Show resolved Hide resolved
@ayushnoori
Copy link
Contributor Author

Thanks for the edits, @frozenbugs. Happy to address all the formatting errors. Do you see any issues with the function implementation itself?

@dgl-bot
Copy link
Collaborator

dgl-bot commented Jan 28, 2024

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

  • @dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Jan 28, 2024

Commit ID: 4228344

Build ID: 3

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Jan 28, 2024

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

  • @dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Jan 28, 2024

Commit ID: 6f8acdb

Build ID: 4

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

@frozenbugs
Copy link
Collaborator

@dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Jan 29, 2024

Commit ID: 6f8acdb

Build ID: 5

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

@frozenbugs
Copy link
Collaborator

frozenbugs commented Jan 29, 2024

the code LGTM now, can you run lintrunner -a to fix the lint error? and can you add a unit test for your code.

The test case can be put here: tests/python/pytorch/dataloading, and you can use this test case as example: https://github.com/dmlc/dgl/blob/master/tests/python/pytorch/graphbolt/impl/test_in_subgraph_sampler.py

@dgl-bot
Copy link
Collaborator

dgl-bot commented Feb 9, 2024

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

  • @dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Feb 9, 2024

Commit ID: d42b1ec2bdca14a28da11c3e5cbb5cab5f2aa768

Build ID: 6

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

@frozenbugs
Copy link
Collaborator

@dgl-bot

@dgl-bot
Copy link
Collaborator

dgl-bot commented Feb 19, 2024

Commit ID: cdc82adc722c688ee0abd1125069d970bb157f05

Build ID: 7

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

@frozenbugs
Copy link
Collaborator

************* Module dgl.dataloading.capped_neighbor_sampler
python/dgl/dataloading/capped_neighbor_sampler.py:60:4: W0221: Parameters differ from overridden 'sample' method (arguments-differ)
python/dgl/dataloading/capped_neighbor_sampler.py:151:27: C0103: Variable name "node_IDs" doesn't conform to snake_case naming style (invalid-name)
python/dgl/dataloading/capped_neighbor_sampler.py:102:27: W0612: Unused variable 'rel_type' (unused-variable)
python/dgl/dataloading/capped_neighbor_sampler.py:102:37: W0612: Unused variable 'dst_type' (unused-variable)

@frozenbugs
Copy link
Collaborator

@ayushnoori if you don't plan to write a unit test, we can also merge this, can you add a comment in capped_neighbor_sampler.py, says "This code was contributed by a community member (ayushnoori). There aren't currently any unit tests in place to verify its functionality, so please be cautious if you need to make any changes to the code's logic."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants