One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control
We present One4D, a unified framework for 4D generation and reconstruction that produces dynamic 4D content as synchronized RGB frames and pointmaps. By consistently handling varying sparsities of conditioning frames through a Unified Masked Conditioning (UMC) mechanism, One4D can seamlessly transition between 4D generation from a single image, 4D reconstruction from a full video, and mixed generation and reconstruction from sparse frames. Our framework adapts a powerful video generation model for joint RGB and pointmap generation, with carefully designed network architectures. The commonly used diffusion finetuning strategies for depthmap or pointmap reconstruction often fail on joint RGB and pointmap generation, quickly degrading the base video model. To address this challenge, we introduce Decoupled LoRA Control (DLC), which employs two modality-specific LoRA adapters to form decoupled computation branches for RGB frames and pointmaps, connected by lightweight, zero-initialized control links that gradually learn mutual pixel-level consistency. Trained on a mixture of synthetic and real 4D datasets under modest computational budgets, One4D produces high-quality RGB frames and accurate pointmaps across both generation and reconstruction tasks. This work represents a step toward general, high-quality geometry-based 4D world modeling using video diffusion models. Project page: https://mizhenxing.github.io/One4D
6925cee24b7b0d870b182778
Zhenxing Mi
claimed_verified
2025-11-26T09:28:08.858Z
6354bda206d707b33249c4c2
/avatars/bbd9f76274ac52214df92084d50bc7b5.svg
Zhenxing Mi
Mifucius
user
Mifucius
6925cee24b7b0d870b182779
Yuxin Wang
claimed_verified
2025-11-26T09:28:06.912Z
6707f8800812a88fd6d87ebe
/avatars/28cd99359799ce85d5fe503144b776d8.svg
YUXIN WANG
yuxinhk
user
yuxinhk
6925cee24b7b0d870b18277a
Dan Xu
claimed_verified
2025-11-27T10:00:20.156Z
66feab48651e00e22f33222e
/avatars/7344377e2c796c7ec85194bb2fc78521.svg
Dan Xu
danxuhk
user
danxuhk
One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control
We present One4D, a unified framework for 4D generation and reconstruction that produces dynamic 4D content as synchronized RGB frames and pointmaps. By consistently handling varying sparsities of conditioning frames through a Unified Masked Conditioning (UMC) mechanism, One4D can seamlessly transition between 4D generation from a single image, 4D reconstruction from a full video, and mixed generation and reconstruction from sparse frames. Our framework adapts a powerful video generation model for joint RGB and pointmap generation, with carefully designed network architectures. The commonly used diffusion finetuning strategies for depthmap or pointmap reconstruction often fail on joint RGB and pointmap generation, quickly degrading the base video model. To address this challenge, we introduce Decoupled LoRA Control (DLC), which employs two modality-specific LoRA adapters to form decoupled computation branches for RGB frames and pointmaps, connected by lightweight, zero-initialized control links that gradually learn mutual pixel-level consistency. Trained on a mixture of synthetic and real 4D datasets under modest computational budgets, One4D produces high-quality RGB frames and accurate pointmaps across both generation and reconstruction tasks. This work represents a step toward general, high-quality geometry-based 4D world modeling using video diffusion models. Project page: https://mizhenxing.github.io/One4D
One4D is a unified framework for 4D generation and reconstruction that uses a novel decoupled approach to produce high-quality RGB frames and pointmaps from varying sparsities of input frames.
Unified Masked Conditioning (UMC) diffusion finetuning Decoupled LoRA Control (DLC) LoRA adapters video generation model joint RGB and pointmap generation pixel-level consistency 4D datasets