DIMO: Diverse 3D Motion Generation for Arbitrary Objects
We present DIMO, a generative approach capable of generating diverse 3D
motions for arbitrary objects from a single image. The core idea of our work is
to leverage the rich priors in well-trained video models to extract the common
motion patterns and then embed them into a shared low-dimensional latent space.
Specifically, we first generate multiple videos of the same object with diverse
motions. We then embed each motion into a latent vector and train a shared
motion decoder to learn the distribution of motions represented by a structured
and compact motion representation, i.e., neural key point trajectories. The
canonical 3D Gaussians are then driven by these key points and fused to model
the geometry and appearance. During inference time with learned latent space,
we can instantly sample diverse 3D motions in a single-forward pass and support
several interesting applications including 3D motion interpolation and
language-guided motion generation. Our project page is available at
https://linzhanm.github.io/dimo.
6912b1b4a644ba07c499c739
Linzhan Mou
6912b1b4a644ba07c499c73a
Jiahui Lei
6912b1b4a644ba07c499c73b
Chen Wang
claimed_verified
2026-01-06T10:05:05.049Z
62e9d5af6687b60b9c01240a
/avatars/3983be5662aa28aefcb18deaf08d7cb1.svg
Chen Wang
chenwang
user
chenwang
6912b1b4a644ba07c499c73c
Lingjie Liu
6912b1b4a644ba07c499c73d
Kostas Daniilidis
DIMO: Diverse 3D Motion Generation for Arbitrary Objects
We present DIMO, a generative approach capable of generating diverse 3D
motions for arbitrary objects from a single image. The core idea of our work is
to leverage the rich priors in well-trained video models to extract the common
motion patterns and then embed them into a shared low-dimensional latent space.
Specifically, we first generate multiple videos of the same object with diverse
motions. We then embed each motion into a latent vector and train a shared
motion decoder to learn the distribution of motions represented by a structured
and compact motion representation, i.e., neural key point trajectories. The
canonical 3D Gaussians are then driven by these key points and fused to model
the geometry and appearance. During inference time with learned latent space,
we can instantly sample diverse 3D motions in a single-forward pass and support
several interesting applications including 3D motion interpolation and
language-guided motion generation. Our project page is available at
https://linzhanm.github.io/dimo.
A generative approach extracts motion patterns from video models, embeds them into a latent space, and uses neural key point trajectories to generate diverse 3D motions from a single image.
generative approach 3D motions video models latent space motion decoder neural key point trajectories canonical 3D Gaussians 3D motion interpolation language-guided motion generation