EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation
Video generative models (VGMs) pretrained on large-scale internet data can produce temporally coherent rollout videos that capture rich object dynamics, offering a compelling foundation for zero-shot robotic manipulation. However, VGMs often produce physically implausible rollouts, and converting their pixel-space motion into robot actions through geometric retargeting further introduces cumulative errors from imperfect depth estimation and keypoint tracking. To address these challenges, we present , a data-free framework that aligns VGM outputs with compositional constraints generated by vision-language models (VLMs) at inference time. The key insight is that VLMs offer a capability complementary to VGMs: structured spatial reasoning that can identify the physical constraints critical to the success and safety of manipulation execution. Given a language instruction,  uses a VLM to automatically extract a set of compositional constraints capturing task-specific requirements, which are then applied at two stages: (1) constraint-guided rollout selection, which scores and filters a batch of VGM rollouts to retain the most physically plausible candidate, and (2) constraint-based trajectory optimization, which uses the selected rollout as initialization and refines the robot trajectory under the same constraint set to correct retargeting errors. We evaluate  on six real-robot manipulation tasks requiring precise, constraint-sensitive execution, improving the overall success rate by 43.3\% points over the strongest baseline without any task-specific training data.
69b0b7361f7902d47f6df8f3
Gehao Zhang
claimed_verified
2026-03-11T08:44:39.014Z
6860d4e1e27de01baf0666fe
/avatars/20c05595ef6f6dfaa73ed6812e19cc0b.svg
Harry Zhang
Harry4582
user
Harry4582
69b0b7361f7902d47f6df8f4
Zhenyang Ni
admin_assigned
2026-03-12T10:11:46.503Z
66e2d670cf6d4b7719bb3dfa
/avatars/117bf9c39be68822ebad90a04540ea72.svg
zhenyangni
zhenyangni
user
zhenyangni
69b0b7361f7902d47f6df8f5
Payal Mohapatra
69b0b7361f7902d47f6df8f6
Han Liu
69b0b7361f7902d47f6df8f7
Ruohan Zhang
69b0b7361f7902d47f6df8f8
Qi Zhu
EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation
Video generative models (VGMs) pretrained on large-scale internet data can produce temporally coherent rollout videos that capture rich object dynamics, offering a compelling foundation for zero-shot robotic manipulation. However, VGMs often produce physically implausible rollouts, and converting their pixel-space motion into robot actions through geometric retargeting further introduces cumulative errors from imperfect depth estimation and keypoint tracking. To address these challenges, we present , a data-free framework that aligns VGM outputs with compositional constraints generated by vision-language models (VLMs) at inference time. The key insight is that VLMs offer a capability complementary to VGMs: structured spatial reasoning that can identify the physical constraints critical to the success and safety of manipulation execution. Given a language instruction,  uses a VLM to automatically extract a set of compositional constraints capturing task-specific requirements, which are then applied at two stages: (1) constraint-guided rollout selection, which scores and filters a batch of VGM rollouts to retain the most physically plausible candidate, and (2) constraint-based trajectory optimization, which uses the selected rollout as initialization and refines the robot trajectory under the same constraint set to correct retargeting errors. We evaluate  on six real-robot manipulation tasks requiring precise, constraint-sensitive execution, improving the overall success rate by 43.3\% points over the strongest baseline without any task-specific training data.
A data-free framework aligns video generative model outputs with vision-language model constraints for improved robotic manipulation, achieving higher success rates through constraint-guided selection and trajectory optimization.
video generative models vision-language models compositional constraints constraint-guided rollout selection trajectory optimization robot actions geometric retargeting temporal coherence physically plausible rollouts