Light-X: Generative 4D Video Rendering with Camera and Illumination Control
Recent advances in illumination control extend image-based methods to video, yet still facing a trade-off between lighting fidelity and temporal consistency. Moving beyond relighting, a key step toward generative modeling of real-world scenes is the joint control of camera trajectory and illumination, since visual dynamics are inherently shaped by both geometry and lighting. To this end, we present Light-X, a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control. 1) We propose a disentangled design that decouples geometry and lighting signals: geometry and motion are captured via dynamic point clouds projected along user-defined camera trajectories, while illumination cues are provided by a relit frame consistently projected into the same geometry. These explicit, fine-grained cues enable effective disentanglement and guide high-quality illumination. 2) To address the lack of paired multi-view and multi-illumination videos, we introduce Light-Syn, a degradation-based pipeline with inverse-mapping that synthesizes training pairs from in-the-wild monocular footage. This strategy yields a dataset covering static, dynamic, and AI-generated scenes, ensuring robust training. Extensive experiments show that Light-X outperforms baseline methods in joint camera-illumination control and surpasses prior video relighting methods under both text- and background-conditioned settings.
693241d66d1060ca587a25f0
Tianqi Liu
claimed_verified
2025-12-05T08:29:55.869Z
66d347eebb76fb26eedb256e
https://cdn-avatars.huggingface.co/v1/production/uploads/66d347eebb76fb26eedb256e/iCPF7GkmZu--XCsWzoucl.jpeg
tianqi liu
tqliu
user
tqliu
693241d66d1060ca587a25f1
Zhaoxi Chen
693241d66d1060ca587a25f2
Zihao Huang
claimed_verified
2025-12-08T08:32:44.815Z
631b24f2f6bc4be4a64c4d43
https://cdn-avatars.huggingface.co/v1/production/uploads/631b24f2f6bc4be4a64c4d43/P9_tVF7SESmVxxGKVCgCk.jpeg
Zihao Huang
Inso
user
Inso
693241d66d1060ca587a25f3
Shaocong Xu
693241d66d1060ca587a25f4
Saining Zhang
693241d66d1060ca587a25f5
Chongjie Ye
693241d66d1060ca587a25f6
Bohan Li
693241d66d1060ca587a25f7
Zhiguo Cao
693241d66d1060ca587a25f8
Wei Li
693241d66d1060ca587a25f9
Hao Zhao
693241d66d1060ca587a25fa
Ziwei Liu
Light-X: Generative 4D Video Rendering with Camera and Illumination Control
Recent advances in illumination control extend image-based methods to video, yet still facing a trade-off between lighting fidelity and temporal consistency. Moving beyond relighting, a key step toward generative modeling of real-world scenes is the joint control of camera trajectory and illumination, since visual dynamics are inherently shaped by both geometry and lighting. To this end, we present Light-X, a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control. 1) We propose a disentangled design that decouples geometry and lighting signals: geometry and motion are captured via dynamic point clouds projected along user-defined camera trajectories, while illumination cues are provided by a relit frame consistently projected into the same geometry. These explicit, fine-grained cues enable effective disentanglement and guide high-quality illumination. 2) To address the lack of paired multi-view and multi-illumination videos, we introduce Light-Syn, a degradation-based pipeline with inverse-mapping that synthesizes training pairs from in-the-wild monocular footage. This strategy yields a dataset covering static, dynamic, and AI-generated scenes, ensuring robust training. Extensive experiments show that Light-X outperforms baseline methods in joint camera-illumination control and surpasses prior video relighting methods under both text- and background-conditioned settings.
Light-X is a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control, outperforming existing methods in joint camera-illumination control and relighting.
disentangled design dynamic point clouds relit frame inverse-mapping degradation-based pipeline Light-X Light-Syn static scenes dynamic scenes AI-generated scenes