HF AutoModel | Data Release | Technical Report | Homepage | HF Space (Demo)

Towards Open-ended Visual Quality Comparison

Haoning Wu^1*, Hanwei Zhu^2*, Zicheng Zhang^3*, Erli Zhang¹, Chaofeng Chen¹, Liang Liao¹, Chunyi Li³

Annan Wang¹, Wenxiu Sun⁴, Qiong Yan⁴, Xiaohong Liu³, Guangtao Zhai³, Shiqi Wang², Weisi Lin¹

^*Equal contribution.

A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

Demo

<script type="module" src="https://gradio.s3-us-west-2.amazonaws.com/3.50.0/gradio.js" ></script>

News

[Feb 24, 2024] A technical report for the data and model is coming soon.

Quick Start

Load Model

import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("q-future/co-instruct", 
                                             trust_remote_code=True, 
                                             torch_dtype=torch.float16,
                                             attn_implementation="eager", 
                                             device_map={"":"cuda:0"})

Chat

import requests
from PIL import Image


### Single Image
prompt = "USER: The image: <|image|> Which happens in this image: motion-blur, over-exposure, or under-exposure? ASSISTANT:"
url = "https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/singapore_flyer.jpg"
image = Image.open(requests.get(url,stream=True).raw)
model.chat(prompt, [image], max_new_tokens=200)

## Motion blur

### Double Image Comparison
prompt_cmp = "USER: The first image: <|image|>\nThe second image: <|image|>Which image has better quality, and why? ASSISTANT:"
url = "https://raw.githubusercontent.com/Q-Future/Q-Align/main/fig/boy_colorful.jpg"
image_2 = Image.open(requests.get(url,stream=True).raw)
model.chat(prompt_cmp, [image, image_2], max_new_tokens=200)

## The second image has better quality. The description indicates that the image has accurate exposure, precise focus, clear details, rich colors, and sufficient lighting. Additionally, the texture details are clear, and the composition is centered. In comparison, the first image has good clarity and rich texture details, but the lighting is slightly weak, which can affect the overall quality of the image. Therefore, the second image is of higher quality due to its accurate exposure, precise focus, clear details, rich colors, sufficient lighting, and centered composition.

Data Release

Coming soon...

Training Scripts

Coming soon...

Cite Us

@misc{wu2024openended,
      title={Towards Open-ended Visual Quality Comparison}, 
      author={Haoning Wu and Hanwei Zhu and Zicheng Zhang and Erli Zhang and Chaofeng Chen and Liang Liao and Chunyi Li and Annan Wang and Wenxiu Sun and Qiong Yan and Xiaohong Liu and Guangtao Zhai and Shiqi Wang and Weisi Lin},
      year={2024},
      eprint={2402.16641},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}