InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions
Humans rarely plan whole-body interactions with objects at the level of explicit whole-body movements. High-level intentions, such as affordance, define the goal, while coordinated balance, contact, and manipulation can emerge naturally from underlying physical and motor priors. Scaling such priors is key to enabling humanoids to compose and generalize loco-manipulation skills across diverse contexts while maintaining physically coherent whole-body coordination. To this end, we introduce InterPrior, a scalable framework that learns a unified generative controller through large-scale imitation pretraining and post-training by reinforcement learning. InterPrior first distills a full-reference imitation expert into a versatile, goal-conditioned variational policy that reconstructs motion from multimodal observations and high-level intent. While the distilled policy reconstructs training behaviors, it does not generalize reliably due to the vast configuration space of large-scale human-object interactions. To address this, we apply data augmentation with physical perturbations, and then perform reinforcement learning finetuning to improve competence on unseen goals and initializations. Together, these steps consolidate the reconstructed latent skills into a valid manifold, yielding a motion prior that generalizes beyond the training data, e.g., it can incorporate new behaviors such as interactions with unseen objects. We further demonstrate its effectiveness for user-interactive control and its potential for real robot deployment.
69855ee24ad556f294b7eb7c
Sirui Xu
claimed_verified
2026-02-06T18:51:52.134Z
64f17c31b4344f592fb2821e
/avatars/11c8edb0967491822277a8a0d3ff3d31.svg
Sirui Xu
xusirui
user
xusirui
69855ee24ad556f294b7eb7d
Samuel Schulter
69855ee24ad556f294b7eb7e
Morteza Ziyadi
69855ee24ad556f294b7eb7f
Xialin He
69855ee24ad556f294b7eb80
Xiaohan Fei
69855ee24ad556f294b7eb81
Yu-Xiong Wang
69855ee24ad556f294b7eb82
Liangyan Gui
InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions
Humans rarely plan whole-body interactions with objects at the level of explicit whole-body movements. High-level intentions, such as affordance, define the goal, while coordinated balance, contact, and manipulation can emerge naturally from underlying physical and motor priors. Scaling such priors is key to enabling humanoids to compose and generalize loco-manipulation skills across diverse contexts while maintaining physically coherent whole-body coordination. To this end, we introduce InterPrior, a scalable framework that learns a unified generative controller through large-scale imitation pretraining and post-training by reinforcement learning. InterPrior first distills a full-reference imitation expert into a versatile, goal-conditioned variational policy that reconstructs motion from multimodal observations and high-level intent. While the distilled policy reconstructs training behaviors, it does not generalize reliably due to the vast configuration space of large-scale human-object interactions. To address this, we apply data augmentation with physical perturbations, and then perform reinforcement learning finetuning to improve competence on unseen goals and initializations. Together, these steps consolidate the reconstructed latent skills into a valid manifold, yielding a motion prior that generalizes beyond the training data, e.g., it can incorporate new behaviors such as interactions with unseen objects. We further demonstrate its effectiveness for user-interactive control and its potential for real robot deployment.
A scalable framework called InterPrior learns a unified generative controller through imitation learning and reinforcement learning to enable humanoids to generalize loco-manipulation skills across diverse contexts while maintaining physically coherent whole-body coordination.
variational policy imitation learning reinforcement learning motion prior latent skills physical perturbations goal-conditioned policy multimodal observations whole-body coordination loco-manipulation skills