The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Text
We present WorldCanvas, a framework for promptable world events that enables rich, user-directed simulation by combining text, trajectories, and reference images. Unlike text-only approaches and existing trajectory-controlled image-to-video methods, our multimodal approach combines trajectories -- encoding motion, timing, and visibility -- with natural language for semantic intent and reference images for visual grounding of object identity, enabling the generation of coherent, controllable events that include multi-agent interactions, object entry/exit, reference-guided appearance and counterintuitive events. The resulting videos demonstrate not only temporal coherence but also emergent consistency, preserving object identity and scene despite temporary disappearance. By supporting expressive world events generation, WorldCanvas advances world models from passive predictors to interactive, user-shaped simulators. Our project page is available at: https://worldcanvas.github.io/.
6944cb02fbf17e708e1860bf
Hanlin Wang
6944cb02fbf17e708e1860c0
Hao Ouyang
6944cb02fbf17e708e1860c1
Qiuyu Wang
claimed_verified
2025-12-19T08:57:37.494Z
64981bea09cea550852652af
/avatars/df528e9008972c8e5ae4d278e617476c.svg
Qiuyu Wang
qiuyuu
user
qiuyuu
6944cb02fbf17e708e1860c2
Yue Yu
6944cb02fbf17e708e1860c3
Yihao Meng
claimed_verified
2026-01-29T14:50:49.462Z
656084f44e8918182d4f07c8
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/akAvCUCi7eR31PWOXrVPw.jpeg
Yihao Meng
Yhmeng1106
user
Yhmeng1106
6944cb02fbf17e708e1860c4
Wen Wang
claimed_verified
2025-12-25T20:52:25.336Z
63f089456309c84d5f47f951
/avatars/04b926a7f2ad091ee00fef0c59903492.svg
Wen Wang
wwen1997
user
wwen1997
6944cb02fbf17e708e1860c5
Ka Leong Cheng
claimed_verified
2026-01-29T14:50:47.636Z
64acd2ec39fcfebff8c79c00
https://cdn-avatars.huggingface.co/v1/production/uploads/64acd2ec39fcfebff8c79c00/Avq66l5hO-aggNtk4Y1ss.png
Ka Leong Cheng
felixcheng97
user
felixcheng97
6944cb02fbf17e708e1860c6
Shuailei Ma
6944cb02fbf17e708e1860c7
Qingyan Bai
6944cb02fbf17e708e1860c8
Yixuan Li
6944cb02fbf17e708e1860c9
Cheng Chen
6944cb02fbf17e708e1860ca
Yanhong Zeng
6944cb02fbf17e708e1860cb
Xing Zhu
6944cb02fbf17e708e1860cc
Yujun Shen
6944cb02fbf17e708e1860cd
Qifeng Chen
The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Text
We present WorldCanvas, a framework for promptable world events that enables rich, user-directed simulation by combining text, trajectories, and reference images. Unlike text-only approaches and existing trajectory-controlled image-to-video methods, our multimodal approach combines trajectories -- encoding motion, timing, and visibility -- with natural language for semantic intent and reference images for visual grounding of object identity, enabling the generation of coherent, controllable events that include multi-agent interactions, object entry/exit, reference-guided appearance and counterintuitive events. The resulting videos demonstrate not only temporal coherence but also emergent consistency, preserving object identity and scene despite temporary disappearance. By supporting expressive world events generation, WorldCanvas advances world models from passive predictors to interactive, user-shaped simulators. Our project page is available at: https://worldcanvas.github.io/.
WorldCanvas generates coherent, controllable world events using a multimodal framework that integrates text, trajectories, and reference images.
promptable world events user-directed simulation trajectories natural language semantic intent reference images visual grounding multi-agent interactions object entry/exit reference-guided appearance counterintuitive events temporal coherence emergent consistency world models interactive simulators