WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
Recent advances in unified multimodal models (UMMs) have enabled impressive progress in visual comprehension and generation. However, existing datasets and benchmarks focus primarily on single-turn interactions, failing to capture the multi-turn, context-dependent nature of real-world image creation and editing. To address this gap, we present WEAVE, the first suite for in-context interleaved cross-modality comprehension and generation. Our suite consists of two complementary parts. WEAVE-100k is a large-scale dataset of 100K interleaved samples spanning over 370K dialogue turns and 500K images, covering comprehension, editing, and generation tasks that require reasoning over historical context. WEAVEBench is a human-annotated benchmark with 100 tasks based on 480 images, featuring a hybrid VLM judger evaluation framework based on both the reference image and the combination of the original image with editing instructions that assesses models' abilities in multi-turn generation, visual memory, and world-knowledge reasoning across diverse domains. Experiments demonstrate that training on WEAVE-100k enables vision comprehension, image editing, and comprehension-generation collaboration capabilities. Furthermore, it facilitates UMMs to develop emergent visual-memory capabilities, while extensive evaluations on WEAVEBench expose the persistent limitations and challenges of current approaches in multi-turn, context-aware image generation and editing. We believe WEAVE provides a view and foundation for studying in-context interleaved comprehension and generation for multi-modal community.
691adfa56bfd5965c0fd3604
Wei Chow
691adfa56bfd5965c0fd3605
Jiachun Pan
691adfa56bfd5965c0fd3606
Yongyuan Liang
claimed_verified
2025-11-18T14:09:05.495Z
6646d5819bb34d2b6b7455d3
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/JFH3ZTPvlaVSg4RJJBb6L.jpeg
Yongyuan Liang
cheryyunl
user
cheryyunl
691adfa56bfd5965c0fd3607
Mingze Zhou
691adfa56bfd5965c0fd3608
Xue Song
691adfa56bfd5965c0fd3609
Liyu Jia
691adfa56bfd5965c0fd360a
Saining Zhang
691adfa56bfd5965c0fd360b
Siliang Tang
691adfa56bfd5965c0fd360c
Juncheng Li
691adfa56bfd5965c0fd360d
Fengda Zhang
claimed_verified
2025-11-17T10:25:55.874Z
68637e529734255efb3be60c
/avatars/d9cdefc30f50daabd173d0f3954e2320.svg
Zhang Fengda
ZhangFengda
user
ZhangFengda
691adfa56bfd5965c0fd360e
Weijia Wu
claimed_verified
2025-11-18T14:09:02.904Z
6345a93afe134dfd7a0cfabd
/avatars/65130ce06b1c72ab1066678419731d88.svg
wu weijia
weijiawu
user
weijiawu
691adfa56bfd5965c0fd360f
Hanwang Zhang
691adfa56bfd5965c0fd3610
Tat-Seng Chua
WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
Recent advances in unified multimodal models (UMMs) have enabled impressive progress in visual comprehension and generation. However, existing datasets and benchmarks focus primarily on single-turn interactions, failing to capture the multi-turn, context-dependent nature of real-world image creation and editing. To address this gap, we present WEAVE, the first suite for in-context interleaved cross-modality comprehension and generation. Our suite consists of two complementary parts. WEAVE-100k is a large-scale dataset of 100K interleaved samples spanning over 370K dialogue turns and 500K images, covering comprehension, editing, and generation tasks that require reasoning over historical context. WEAVEBench is a human-annotated benchmark with 100 tasks based on 480 images, featuring a hybrid VLM judger evaluation framework based on both the reference image and the combination of the original image with editing instructions that assesses models' abilities in multi-turn generation, visual memory, and world-knowledge reasoning across diverse domains. Experiments demonstrate that training on WEAVE-100k enables vision comprehension, image editing, and comprehension-generation collaboration capabilities. Furthermore, it facilitates UMMs to develop emergent visual-memory capabilities, while extensive evaluations on WEAVEBench expose the persistent limitations and challenges of current approaches in multi-turn, context-aware image generation and editing. We believe WEAVE provides a view and foundation for studying in-context interleaved comprehension and generation for multi-modal community.
WEAVE introduces a comprehensive suite including a large dataset and a benchmark to assess and improve multi-turn, context-dependent image generation and editing in unified multimodal models.
unified multimodal models WEAVE WEAVE-100k interleaved samples dialogue turns comprehension editing generation WEAVEBench human-annotated benchmark VLM judger visual memory world-knowledge reasoning multi-turn generation vision comprehension image editing comprehension-generation collaboration emergent visual-memory capabilities