Open-source industrial-grade ASR models supporting Mandarin, Chinese dialects and English, achieving a new SOTA on public Mandarin ASR benchmarks, while also offering outstanding singing lyrics recognition capability.

Official implementation of paper "MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens"

Research Code for Multimodal-Cognition Team in Ant Group

[IROS'25 Oral & NeurIPSw'24] Official implementation of "MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control "

[ECCV 2024] Official PyTorch Implementation of "How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs"

[ICCV 2025] Token Activation Map to Visually Explain Multimodal LLMs

The code repository for "Wings: Learning Multimodal LLMs without Text-only Forgetting" [NeurIPS 2024]

Official repository of the paper: Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics

[ACL 2024] Dataset and Code of "ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction"

Streamlit app to chat with images using Multi-modal LLMs.

SpatialFusion-LM is a real-time spatial reasoning framework that combines neural depth, 3D reconstruction, and language-driven scene understanding.

LLaVA base model for use with Autodistill.

[NAACL 2025 Findings] Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding

Kani extension for supporting vision-language models (VLMs). Comes with model-agnostic support for GPT-Vision and LLaVA.

Medical Report Generation And VQA (Adapting XrayGPT to Any Modality)

The future of AI is speaking Chilean, cachai?

NEMO: Can Multimodal LLMs Identify Attribute-Modified Objects?

This is the project webpage for 'SeTok'.

A Spring AI "cookbook" project demonstrating how to build a robust image analysis API. This service uses a Large Language Model (LLM) to analyze images that are provided in multiple formats: direct file uploads (multipart), web URLs, classpath resources, and Base64 strings.

🫁 Generating diagnostic reports from chest X-rays using vision transformers and large language models