ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priors
Synthesizing physically plausible articulated human-object interactions (HOI) without 3D/4D supervision remains a fundamental challenge. While recent zero-shot approaches leverage video diffusion models to synthesize human-object interactions, they are largely confined to rigid-object manipulation and lack explicit 4D geometric reasoning. To bridge this gap, we formulate articulated HOI synthesis as a 4D reconstruction problem from monocular video priors: given only a video generated by a diffusion model, we reconstruct a full 4D articulated scene without any 3D supervision. This reconstruction-based approach treats the generated 2D video as supervision for an inverse rendering problem, recovering geometrically consistent and physically plausible 4D scenes that naturally respect contact, articulation, and temporal coherence. We introduce ArtHOI, the first zero-shot framework for articulated human-object interaction synthesis via 4D reconstruction from video priors. Our key designs are: 1) Flow-based part segmentation: leveraging optical flow as a geometric cue to disentangle dynamic from static regions in monocular video; 2) Decoupled reconstruction pipeline: joint optimization of human motion and object articulation is unstable under monocular ambiguity, so we first recover object articulation, then synthesize human motion conditioned on the reconstructed object states. ArtHOI bridges video-based generation and geometry-aware reconstruction, producing interactions that are both semantically aligned and physically grounded. Across diverse articulated scenes (e.g., opening fridges, cabinets, microwaves), ArtHOI significantly outperforms prior methods in contact accuracy, penetration reduction, and articulation fidelity, extending zero-shot interaction synthesis beyond rigid manipulation through reconstruction-informed synthesis.
69a8fc218b5b27c7ea9fa63e
Zihao Huang
69a8fc218b5b27c7ea9fa63f
Tianqi Liu
admin_assigned
2026-03-05T10:27:14.285Z
66d347eebb76fb26eedb256e
https://cdn-avatars.huggingface.co/v1/production/uploads/66d347eebb76fb26eedb256e/iCPF7GkmZu--XCsWzoucl.jpeg
tianqi liu
tqliu
user
tqliu
69a8fc218b5b27c7ea9fa640
Zhaoxi Chen
admin_assigned
2026-03-05T10:27:26.296Z
62fc8cf7ee999004b5a8b982
/avatars/6c5dda9e58747054a989f077a078f3dc.svg
Zhaoxi Chen
FrozenBurning
user
FrozenBurning
69a8fc218b5b27c7ea9fa641
Shaocong Xu
69a8fc218b5b27c7ea9fa642
Saining Zhang
admin_assigned
2026-03-05T10:27:32.131Z
6688fde24acaceadde4a9f7e
/avatars/6f1b0008ff52c3f794e8223b74bcaeb3.svg
Saining Zhang
simon123905
user
simon123905
69a8fc218b5b27c7ea9fa643
Lixing Xiao
admin_assigned
2026-03-05T10:27:39.053Z
65220fedc709aaca9aa63061
/avatars/36e56199fa8c98ff3430636567a0ed62.svg
Lixing Xiao
lxxiao
user
lxxiao
69a8fc218b5b27c7ea9fa644
Zhiguo Cao
69a8fc218b5b27c7ea9fa645
Wei Li
69a8fc218b5b27c7ea9fa646
Hao Zhao
69a8fc218b5b27c7ea9fa647
Ziwei Liu
admin_assigned
2026-03-05T10:27:50.254Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
liuziwei7
user
liuziwei7
ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priors
Synthesizing physically plausible articulated human-object interactions (HOI) without 3D/4D supervision remains a fundamental challenge. While recent zero-shot approaches leverage video diffusion models to synthesize human-object interactions, they are largely confined to rigid-object manipulation and lack explicit 4D geometric reasoning. To bridge this gap, we formulate articulated HOI synthesis as a 4D reconstruction problem from monocular video priors: given only a video generated by a diffusion model, we reconstruct a full 4D articulated scene without any 3D supervision. This reconstruction-based approach treats the generated 2D video as supervision for an inverse rendering problem, recovering geometrically consistent and physically plausible 4D scenes that naturally respect contact, articulation, and temporal coherence. We introduce ArtHOI, the first zero-shot framework for articulated human-object interaction synthesis via 4D reconstruction from video priors. Our key designs are: 1) Flow-based part segmentation: leveraging optical flow as a geometric cue to disentangle dynamic from static regions in monocular video; 2) Decoupled reconstruction pipeline: joint optimization of human motion and object articulation is unstable under monocular ambiguity, so we first recover object articulation, then synthesize human motion conditioned on the reconstructed object states. ArtHOI bridges video-based generation and geometry-aware reconstruction, producing interactions that are both semantically aligned and physically grounded. Across diverse articulated scenes (e.g., opening fridges, cabinets, microwaves), ArtHOI significantly outperforms prior methods in contact accuracy, penetration reduction, and articulation fidelity, extending zero-shot interaction synthesis beyond rigid manipulation through reconstruction-informed synthesis.
ArtHOI synthesizes articulated human-object interactions by formulating 4D reconstruction from monocular video priors, using optical flow for part segmentation and a decoupled reconstruction pipeline to achieve physically plausible and geometrically consistent interactions.
video diffusion models 4D reconstruction monocular video optical flow part segmentation decoupled reconstruction pipeline inverse rendering contact accuracy articulation fidelity zero-shot synthesis