A sharp decline in performance from OPV2V to V2Xset?

[lubin2022]

I want to train the pointpillar_v2vnet model on both OpV2V and V2XSet dataset, the yaml file are the same except for the 'root_dir' and 'validate_dir'. One is 'opv2v/train', 'opv2v/validate' and the other one is 'v2xset/train', 'v2xset/validate'. For the OPV2V dataset, AP@0.5=0.914, AP@0.7=0.798, but for the V2XSet dataset, AP@0.5=0.672, AP@0.7=0.315
Why does the performance differ so much between two datasets? Is it normal?

[DerrickXuNu]

V2XSet is usually more challenging than opv2v, but from my own experiments, the difference is not that large as yours. I think you didn't train the models well on V2XSet