LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation
Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.
69c0a94b3fd3fef6237a6842
Jiazheng Xing
69c0a94b3fd3fef6237a6843
Fei Du
69c0a94b3fd3fef6237a6844
Hangjie Yuan
69c0a94b3fd3fef6237a6845
Pengwei Liu
69c0a94b3fd3fef6237a6846
Hongbin Xu
69c0a94b3fd3fef6237a6847
Hai Ci
69c0a94b3fd3fef6237a6848
Ruigang Niu
69c0a94b3fd3fef6237a6849
Weihua Chen
69c0a94b3fd3fef6237a684a
Fan Wang
69c0a94b3fd3fef6237a684b
Yong Liu
LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation
Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.
LumosX framework enhances text-to-video generation through relational attention mechanisms and structured data pipelines for improved face-attribute alignment and subject consistency.
diffusion models text-to-video generation personalized content creation face-attribute alignment multimodal large language models relational self-attention relational cross-attention subject-attribute dependencies intra-group consistency semantic alignment