Code for the paper Unsupervised Scene Graph Induction from Natural Language Supervision by Yanpeng Zhao and Ivan Titov.
Currently we consider only artificial datasets such as CLEVR and AbstractScene. We focus on CLEVR but may also discuss the pre-processing of AbstractScene.
We use CELVR v1.0 and rely on this repo to generate bounding boxes for the CLEVR images.
For AbstractScene images, bounding boxes can be automatically inferred (see this script).
Check out the running script bash/run_obj_enc.sh
.
Check out this repo.
Learning with symbolic object representations: bash/run_clevr_sym.sh
.
Learning with visual object representations: bash/run_clevr_vis.sh
.
Learning with additional causal language modeling (CLM): bash/run_clevr_clm.sh
.
Currently we do not have a specific script for inference since evaluation (i.e., inference) is automatically performed after every training epoch. If you want to create an inference script, you may want to check out bash/run_clevr_eval.sh
and the associated inference function.
git clone --branch beta https://github.com/zhaoyanpeng/sgi.git
cd sgi
virtualenv -p python3.8 ./pyenv/oops
source ./pyenv/oops/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install -r requirements.txt
@misc{VC-MLM,
author = {Yanpeng Zhao and Ivan Titov},
title = {Unsupervised Scene Graph Induction from Natural Language Supervision},
year = {2023},
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/zhaoyanpeng/sgi}},
}
MIT