Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions
Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro.
69927e3c50fb2c0be478388e
Yunheng Li
69927e3c50fb2c0be478388f
Hengrui Zhang
claimed_verified
2026-02-16T15:36:34.435Z
67836a80a61d3631a39813c3
https://cdn-avatars.huggingface.co/v1/production/uploads/67836a80a61d3631a39813c3/QuXpaW5m4bXgcfujWaKab.jpeg
HenRUIKA
henryrk
user
henryrk
69927e3c50fb2c0be4783890
Meng-Hao Guo
69927e3c50fb2c0be4783891
Wenzhao Gao
69927e3c50fb2c0be4783892
Shaoyong Jia
69927e3c50fb2c0be4783893
Shaohui Jiao
69927e3c50fb2c0be4783894
Qibin Hou
69927e3c50fb2c0be4783895
Ming-Ming Cheng
Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions
Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro.
A large-scale dataset and model for fine-grained audiovisual understanding are introduced, demonstrating improved caption quality and reduced hallucinations through structured annotations and supervised fine-tuning.
audiovisual instruction annotations supervised fine-tuning audiovisual captioning attribute-wise captioning caption-based QA caption-based temporal grounding