Skip to content

[NeurIPS2023] Official implementation and model release of the paper "What Makes Good Examples for Visual In-Context Learning?"

License

Notifications You must be signed in to change notification settings

ZhangYuanhan-AI/visual_prompt_retrieval

Repository files navigation

What Makes Good Examples for Visual In-Context Learning?

S-Lab, Nanyang Technological University

TL;DR

We study on the effect of in-context examples in computer vision. We propose a Prompt Retrieval framework to automatically select examples, consisting of an unsupervised (UnsupPR) and a supervised method (SupPR).


[arXiv]

Updatas

[01/2023] arXiv paper has been released.

[01/2023] The code for foureground segmentation has been released.

Environment setup

conda create -n XXX python=3.8
conda activate XXX
pip install -r requirements.txt

Data preparation

Our data preparation pipeline is based on visual prompt. Please follow the dataset preparation steps for PASCAL-5i dataset in this repository.

How to run

Click the Unsup/Sup stratedgy below to see the detailed instructions on how to run the code to reproduce the results.

Performance

Here, Random is the baseline method in visual prompt, SupPR and UnsupPR are shorted for supervised prompt retrieval and unsupervised prompt retrieval respectively.

fig1

Model

The SupPR models for each pascal-5i split is uploaded in link.

Citation

If you use this code in your research, please kindly cite this work.

@inproceedings{zhang2023VisualPromptRetrieval,
      title={What Makes Good Examples for Visual In-Context Learning?}, 
      author={Yuanhan Zhang and Kaiyang Zhou and Ziwei Liu},
      year={2023},
      archivePrefix={arXiv},
}

Acknowledgments

Part of the code is borrowed from visual prompt, SupContrast, timm and mmcv.

Hits

About

[NeurIPS2023] Official implementation and model release of the paper "What Makes Good Examples for Visual In-Context Learning?"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published