DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Data
Despite recent progress, video diffusion models still struggle to synthesize realistic videos involving highly dynamic motions or requiring fine-grained motion controllability. A central limitation lies in the scarcity of such examples in commonly used training datasets. To address this, we introduce DynaVid, a video synthesis framework that leverages synthetic motion data in training, which is represented as optical flow and rendered using computer graphics pipelines. This approach offers two key advantages. First, synthetic motion offers diverse motion patterns and precise control signals that are difficult to obtain from real data. Second, unlike rendered videos with artificial appearances, rendered optical flow encodes only motion and is decoupled from appearance, thereby preventing models from reproducing the unnatural look of synthetic videos. Building on this idea, DynaVid adopts a two-stage generation framework: a motion generator first synthesizes motion, and then a motion-guided video generator produces video frames conditioned on that motion. This decoupled formulation enables the model to learn dynamic motion patterns from synthetic data while preserving visual realism from real-world videos. We validate our framework on two challenging scenarios, vigorous human motion generation and extreme camera motion control, where existing datasets are particularly limited. Extensive experiments demonstrate that DynaVid improves the realism and controllability in dynamic motion generation and camera motion control.
69cf59eae02fdd68b91b7b36
Wonjoon Jin
69cf59eae02fdd68b91b7b37
Jiyun Won
claimed_verified
2026-04-09T18:51:08.178Z
69d79bb236aa554bc01990b8
/avatars/5754841e0fc0d4bdf3cf1fec4f290c87.svg
Jiyun Won
jiyunwon
user
jiyunwon
69cf59eae02fdd68b91b7b38
Janghyeok Han
69cf59eae02fdd68b91b7b39
Qi Dai
69cf59eae02fdd68b91b7b3a
Chong Luo
69cf59eae02fdd68b91b7b3b
Seung-Hwan Baek
69cf59eae02fdd68b91b7b3c
Sunghyun Cho
DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Data
Despite recent progress, video diffusion models still struggle to synthesize realistic videos involving highly dynamic motions or requiring fine-grained motion controllability. A central limitation lies in the scarcity of such examples in commonly used training datasets. To address this, we introduce DynaVid, a video synthesis framework that leverages synthetic motion data in training, which is represented as optical flow and rendered using computer graphics pipelines. This approach offers two key advantages. First, synthetic motion offers diverse motion patterns and precise control signals that are difficult to obtain from real data. Second, unlike rendered videos with artificial appearances, rendered optical flow encodes only motion and is decoupled from appearance, thereby preventing models from reproducing the unnatural look of synthetic videos. Building on this idea, DynaVid adopts a two-stage generation framework: a motion generator first synthesizes motion, and then a motion-guided video generator produces video frames conditioned on that motion. This decoupled formulation enables the model to learn dynamic motion patterns from synthetic data while preserving visual realism from real-world videos. We validate our framework on two challenging scenarios, vigorous human motion generation and extreme camera motion control, where existing datasets are particularly limited. Extensive experiments demonstrate that DynaVid improves the realism and controllability in dynamic motion generation and camera motion control.
DynaVid addresses limitations in video diffusion models by using synthetic motion data represented as optical flow to improve realistic video synthesis with dynamic motions and fine-grained motion control.
video diffusion models optical flow computer graphics pipelines motion generator motion-guided video generator two-stage generation framework dynamic motion patterns visual realism video synthesis framework