Planning with Sketch-Guided Verification for Physics-Aware Video Generation
Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.
6923c372b5612535ed95589b
Yidong Huang
6923c372b5612535ed95589c
Zun Wang
6923c372b5612535ed95589d
Han Lin
6923c372b5612535ed95589e
Dong-Ki Kim
6923c372b5612535ed95589f
Shayegan Omidshafiei
6923c372b5612535ed9558a0
Jaehong Yoon
claimed_verified
2026-01-05T08:28:07.211Z
652066649004117947e46ed6
/avatars/972c97df6f26d2c3d6ce71ec579984bb.svg
Jaehong Yoon
jaehong31
user
jaehong31
6923c372b5612535ed9558a1
Yue Zhang
6923c372b5612535ed9558a2
Mohit Bansal
Planning with Sketch-Guided Verification for Physics-Aware Video Generation
Recent video generation approaches increasingly rely on planning intermediate control signals such as object trajectories to improve temporal coherence and motion fidelity. However, these methods mostly employ single-shot plans that are typically limited to simple motions, or iterative refinement which requires multiple calls to the video generator, incuring high computational cost. To overcome these limitations, we propose SketchVerify, a training-free, sketch-verification-based planning framework that improves motion planning quality with more dynamically coherent trajectories (i.e., physically plausible and instruction-consistent motions) prior to full video generation by introducing a test-time sampling and verification loop. Given a prompt and a reference image, our method predicts multiple candidate motion plans and ranks them using a vision-language verifier that jointly evaluates semantic alignment with the instruction and physical plausibility. To efficiently score candidate motion plans, we render each trajectory as a lightweight video sketch by compositing objects over a static background, which bypasses the need for expensive, repeated diffusion-based synthesis while achieving comparable performance. We iteratively refine the motion plan until a satisfactory one is identified, which is then passed to the trajectory-conditioned generator for final synthesis. Experiments on WorldModelBench and PhyWorldBench demonstrate that our method significantly improves motion quality, physical realism, and long-term consistency compared to competitive baselines while being substantially more efficient. Our ablation study further shows that scaling up the number of trajectory candidates consistently enhances overall performance.
SketchVerify enhances video motion planning and generation by iteratively refining candidate motion plans using a lightweight sketch-based verification process, improving both quality and efficiency.
planning intermediate control signals object trajectories temporal coherence motion fidelity single-shot plans iterative refinement training-free sketch-verification-based planning motion planning quality dynamically coherent trajectories physically plausible instruction-consistent motions test-time sampling and verification loop prompt reference image candidate motion plans vision-language verifier semantic alignment physical plausibility trajectory-conditioned generator WorldModelBench PhyWorldBench motion quality physical realism long-term consistency trajectory candidates