Strange behavior of randint using device=cuda

[smzzl]

🐛 Describe the bug

I want to generate some random numbers on the GPU, but it seems that there is an unreasonably high duplicate rate.
I can get the expected output under CPU

>>> ls = []
>>> for i in range(2): 
...     ls.extend(torch.randint(0, int(6e9), [int(1e6)], device="cpu").cpu().numpy().tolist())
... 
>>> len(set(ls))
1999678

When generating random numbers using torch.randint on GPU, an unexpected behavior is observed.

>>> ls = []
>>> for i in range(2): 
...     ls.extend(torch.randint(0, int(6e9), [int(1e6)], device="cuda:0").cpu().numpy().tolist())
... 
>>> len(set(ls))
1884451

Strange behavior of randint using device=cuda, len(set(ls))=1,884,451 means rand produced 2,000,000-1,884,451=115,549 duplicate numbers. This is impossible for the random range of 6e9

I have tried multiple different versions of pytorch on different machines, and the expected value was obtained under pytorch.__version __='1.1.0'(same machines with pytorch.__version __='2.0.1+cu118' failed), but this value is not reasonable in other versions, ranging from 150w to 190w.

Moreover, if the upper bound of randint is changed to 6e8, this issue will not occur, the output 'len(set (ls))' will be reasonable.

>>> ls = []
>>> for i in range(2) :
...     ls.extend(torch.randint(0, int(6e8), [int(1e6)], device="cuda:0").cpu().numpy().tolist())
... 
>>> len(set(ls))
1996659

Versions

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.3
[pip3] torch==2.0.1+cu118
[pip3] torchaudio==2.0.1+cu118
[pip3] torchvision==0.15.2+cu118
[pip3] triton==2.0.0
[conda] numpy 1.26.3 pypi_0 pypi
[conda] torch 2.0.1+cu118 pypi_0 pypi
[conda] torchaudio 2.0.1+cu118 pypi_0 pypi
[conda] torchvision 0.15.2+cu118 pypi_0 pypi
[conda] triton 2.0.0 pypi_0 pypi

cc @fritzo @neerajprad @alicanb @nikitaved @ptrblck @pbelevich

[tringwald]

Thank you for your bug report. I have tracked down the problem and fixed it in #126066. For me, the effect is even more brutal. I only get 1,556,855 unique numbers when generating 2x 1,000,000 random ints. After the fix I get 1,999,666.

>>> import torch
>>> a=torch.randint(0, int(6e9), [int(1e6)], device="cuda")
>>> b=torch.randint(0, int(6e9), [int(1e6)], device="cuda")
>>> a.unique().shape
torch.Size([999914])
>>> b.unique().shape
torch.Size([999924])
>>> torch.stack([a, b]).unique().shape
torch.Size([1556855])