Composing Concepts from Images and Videos via Concept-prompt Binding
Visual concept composition, which aims to integrate different elements from images and videos into a single, coherent visual output, still falls short in accurately extracting complex concepts from visual inputs and flexibly combining concepts from both images and videos. We introduce Bind & Compose, a one-shot method that enables flexible visual concept composition by binding visual concepts with corresponding prompt tokens and composing the target prompt with bound tokens from various sources. It adopts a hierarchical binder structure for cross-attention conditioning in Diffusion Transformers to encode visual concepts into corresponding prompt tokens for accurate decomposition of complex visual concepts. To improve concept-token binding accuracy, we design a Diversify-and-Absorb Mechanism that uses an extra absorbent token to eliminate the impact of concept-irrelevant details when training with diversified prompts. To enhance the compatibility between image and video concepts, we present a Temporal Disentanglement Strategy that decouples the training process of video concepts into two stages with a dual-branch binder structure for temporal modeling. Evaluations demonstrate that our method achieves superior concept consistency, prompt fidelity, and motion quality over existing approaches, opening up new possibilities for visual creativity.
693a2f6d74fced5bf9c323c2
Xianghao Kong
claimed_verified
2025-12-11T10:13:32.571Z
6428fd124fe87caede856311
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5OrNPwZkxu3Dm1IInCxML.jpeg
Xianghao Kong
refkxh
user
refkxh
693a2f6d74fced5bf9c323c3
Zeyu Zhang
claimed_verified
2025-12-12T09:25:34.352Z
6539f922eeb6c5f65cc5f486
/avatars/9882c34c1a9c8dc415cf495c12b29fa3.svg
Zeyu Zhang
JohnnyZ666
user
JohnnyZ666
693a2f6d74fced5bf9c323c4
Yuwei Guo
693a2f6d74fced5bf9c323c5
Zhuoran Zhao
claimed_verified
2025-12-11T10:13:29.933Z
67f87bc19d597ac661a75b68
https://cdn-avatars.huggingface.co/v1/production/uploads/67f87bc19d597ac661a75b68/ARLLbu1CJ5mCQu6ptyfmG.jpeg
Zhuoran Zhao
Alicezrzhao
user
Alicezrzhao
693a2f6d74fced5bf9c323c6
Songchun Zhang
693a2f6d74fced5bf9c323c7
Anyi Rao
admin_assigned
2025-12-11T10:45:24.045Z
63f8130749569335b679af62
https://cdn-avatars.huggingface.co/v1/production/uploads/63f8130749569335b679af62/vgTu23-y0UKocwAGqNMwT.jpeg
Anyi Rao
anyirao
user
anyirao
Composing Concepts from Images and Videos via Concept-prompt Binding
Visual concept composition, which aims to integrate different elements from images and videos into a single, coherent visual output, still falls short in accurately extracting complex concepts from visual inputs and flexibly combining concepts from both images and videos. We introduce Bind & Compose, a one-shot method that enables flexible visual concept composition by binding visual concepts with corresponding prompt tokens and composing the target prompt with bound tokens from various sources. It adopts a hierarchical binder structure for cross-attention conditioning in Diffusion Transformers to encode visual concepts into corresponding prompt tokens for accurate decomposition of complex visual concepts. To improve concept-token binding accuracy, we design a Diversify-and-Absorb Mechanism that uses an extra absorbent token to eliminate the impact of concept-irrelevant details when training with diversified prompts. To enhance the compatibility between image and video concepts, we present a Temporal Disentanglement Strategy that decouples the training process of video concepts into two stages with a dual-branch binder structure for temporal modeling. Evaluations demonstrate that our method achieves superior concept consistency, prompt fidelity, and motion quality over existing approaches, opening up new possibilities for visual creativity.
Bind & Compose uses Diffusion Transformers with hierarchical binders and temporal strategies to accurately compose complex visual concepts from images and videos.
visual concept composition prompt tokens cross-attention conditioning Diffusion Transformers hierarchical binder structure Diversify-and-Absorb Mechanism absorbent token Temporal Disentanglement Strategy dual-branch binder structure