SHAPES-SyGeT (SHAPES Systematic Generalization Test) is a split of the SHAPES dataset [1] that can be used to evaluate systematic generalization.
- is a
COLORshapeTRANSFORMaCOLORshape - is a
SHAPETRANSFORMaCOLORshape - is a
SHAPETRANSFORMTRANSFORMaCOLORshape - is a
SHAPETRANSFORMTRANSFORMaSHAPE - is a
COLORshape aSHAPE - is a
SHAPECOLOR - is a
SHAPEaSHAPE
- is a
COLORshapeTRANSFORMTRANSFORMaCOLORshape - is a
COLORshapeTRANSFORMaSHAPE - is a
COLORshapeTRANSFORMTRANSFORMaSHAPE - is a
SHAPETRANSFORMaSHAPE - is a
COLORshapeCOLOR
COLOR can take values in 'red', 'green', 'blue'
SHAPE can take values in 'circle', 'triangle', 'square'
TRANSFORM can take values in 'above', 'below', 'left of', 'right of'
Train and Val-IID use train templates. Val-OOD uses evaluation templates.
Train size: 7560
Val-IID size: 1080
Val-OOD size: 6976
[1] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. “Neural module networks.” In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 39–48.