How can I solve the AttributeError: 'Partition' object has no attribute 'indptr' with the Flickr dataset

[yuhaoxu99]

🐛 Describe the bug

create Flickr data

import torch_geometric.datasets
data_path = os.path.join(path, Flickr)
T = ToSparseTensor() if to_sparse else lambda x: x
dataset_class = getattr(torch_geometric.datasets, dataset_name)
dataset = dataset_class(data_path, transform=T)
processed_dir = dataset.processed_dir
data = dataset[0]
split_masks = {}
split_masks["train"] = data.train_mask
split_masks["valid"] = data.val_mask
split_masks["test"] = data.test_mask
x = data.x
y = data.y

uses ClusterLoader create subgraph

from torch_geometric.data import ClusterData, ClusterLoader
sample_size = max(1, int(args.batch_size / (data.num_nodes / args.num_parts)))
cluster_data = ClusterData(
data,
num_parts=args.num_parts,
recursive=False,
save_dir=self.processed_dir
)
train_loader = ClusterLoader(
cluster_data,
batch_size=sample_size,
shuffle=True
)

when I train the subgraph data, I need use the following to get the batch

for batch in train_loader:

and for this step, it will get the error:
for batch in train_loader:
File "D:\package\Anaconda\envs\Graph118\lib\site-packages\torch\utils\data\dataloader.py", line 630, in next
data = self._next_data()
File "D:\package\Anaconda\envs\Graph118\lib\site-packages\torch\utils\data\dataloader.py", line 674, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "D:\package\Anaconda\envs\Graph118\lib\site-packages\torch\utils\data_utils\fetch.py", line 54, in fetch
return self.collate_fn(data)
File "D:\package\Anaconda\envs\Graph118\lib\site-packages\torch_geometric\loader\cluster.py", line 263, in _collate
global_indptr = self.cluster_data.partition.indptr
AttributeError: 'Partition' object has no attribute 'indptr'

The above code is useful for another dataset, such as Reddit, but it doesn't work for Flickr. Please can you help me solve this problem?

Versions

Collecting environment information...
PyTorch version: 2.1.0+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Could not collect
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.14 | packaged by Anaconda, Inc. | (main, Mar 21 2024, 16:20:14) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: 12.1.66
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: Quadro RTX 5000
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:

Revision=21767

Versions of relevant libraries:
[pip3] numpy==1.26.3
[pip3] torch==2.1.0+cu118
[pip3] torch-cluster==1.6.3+pt21cu118
[pip3] torch_geometric==2.5.3
[pip3] torch-scatter==2.1.2+pt21cu118
[pip3] torch-sparse==0.6.18+pt21cu118
[pip3] torch-spline-conv==1.2.2+pt21cu118
[pip3] torchaudio==2.1.0+cu118
[pip3] torchdata==0.7.1
[pip3] torchvision==0.16.0+cu118
[conda] numpy 1.26.3 pypi_0 pypi
[conda] torch 2.1.0+cu118 pypi_0 pypi
[conda] torch-cluster 1.6.3+pt21cu118 pypi_0 pypi
[conda] torch-geometric 2.5.3 pypi_0 pypi
[conda] torch-scatter 2.1.2+pt21cu118 pypi_0 pypi
[conda] torch-sparse 0.6.18+pt21cu118 pypi_0 pypi
[conda] torch-spline-conv 1.2.2+pt21cu118 pypi_0 pypi
[conda] torchaudio 2.1.0+cu118 pypi_0 pypi
[conda] torchdata 0.7.1 pypi_0 pypi
[conda] torchvision 0.16.0+cu118 pypi_0 pypi

[rusty1s]

The following works for me:

from torch_geometric.data import ClusterData, ClusterLoader
from torch_geometric.datasets import Flickr

dataset = Flickr('/tmp/Flickr')
data = dataset[0]

cluster_data = ClusterData(data, num_parts=10, recursive=False)
train_loader = ClusterLoader(
    cluster_data,
    batch_size=2,
    shuffle=True,
)

for batch in train_loader:
    print(batch)
```
Can you confirm?

[yuhaoxu99]

The following works for me:

from torch_geometric.data import ClusterData, ClusterLoader
from torch_geometric.datasets import Flickr

dataset = Flickr('/tmp/Flickr')
data = dataset[0]

cluster_data = ClusterData(data, num_parts=10, recursive=False)
train_loader = ClusterLoader(
    cluster_data,
    batch_size=2,
    shuffle=True,
)

for batch in train_loader:
    print(batch)

Can you confirm?

Thank you for your reply! Now it works for me! And I really feel strange sometimes it can work, and sometimes it doesn't.