BUG in DAB-DETR when with_random_refpoints=True

[YangHan-Morningstar]

Hi there, when I set with_random_refpoints=True in DAB-DETR, I observe that there are still grad for xy, I print the value of xy in the iteration 0 and iteration 50 and the xy value is different, but in the paper the xy should be fixed after random init