HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Models
Vision-Language-Action (VLA) models have recently enabled robotic manipulation by grounding visual and linguistic cues into actions. However, most VLAs assume the Markov property, relying only on the current observation and thus suffering from temporal myopia that degrades long-horizon coherence. In this work, we view motion as a more compact and informative representation of temporal context and world dynamics, capturing inter-state changes while filtering static pixel-level noise. Building on this idea, we propose HiF-VLA (Hindsight, Insight, and Foresight for VLAs), a unified framework that leverages motion for bidirectional temporal reasoning. HiF-VLA encodes past dynamics through hindsight priors, anticipates future motion via foresight reasoning, and integrates both through a hindsight-modulated joint expert to enable a ''think-while-acting'' paradigm for long-horizon manipulation. As a result, HiF-VLA surpasses strong baselines on LIBERO-Long and CALVIN ABC-D benchmarks, while incurring negligible additional inference latency. Furthermore, HiF-VLA achieves substantial improvements in real-world long-horizon manipulation tasks, demonstrating its broad effectiveness in practical robotic settings.
693a481d74fced5bf9c3247d
Minghui Lin
693a481d74fced5bf9c3247e
Pengxiang Ding
693a481d74fced5bf9c3247f
Shu Wang
693a481d74fced5bf9c32480
Zifeng Zhuang
693a481d74fced5bf9c32481
Yang Liu
693a481d74fced5bf9c32482
Xinyang Tong
693a481d74fced5bf9c32483
Wenxuan Song
693a481d74fced5bf9c32484
Shangke Lyu
693a481d74fced5bf9c32485
Siteng Huang
admin_assigned
2025-12-11T10:50:45.555Z
65fd82762bf2cd20ddaa193f
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/yBYbWp_mT7UusYdkqtAvw.png
Siteng Huang
huangsiteng
user
huangsiteng
693a481d74fced5bf9c32486
Donglin Wang
HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Models
Vision-Language-Action (VLA) models have recently enabled robotic manipulation by grounding visual and linguistic cues into actions. However, most VLAs assume the Markov property, relying only on the current observation and thus suffering from temporal myopia that degrades long-horizon coherence. In this work, we view motion as a more compact and informative representation of temporal context and world dynamics, capturing inter-state changes while filtering static pixel-level noise. Building on this idea, we propose HiF-VLA (Hindsight, Insight, and Foresight for VLAs), a unified framework that leverages motion for bidirectional temporal reasoning. HiF-VLA encodes past dynamics through hindsight priors, anticipates future motion via foresight reasoning, and integrates both through a hindsight-modulated joint expert to enable a ''think-while-acting'' paradigm for long-horizon manipulation. As a result, HiF-VLA surpasses strong baselines on LIBERO-Long and CALVIN ABC-D benchmarks, while incurring negligible additional inference latency. Furthermore, HiF-VLA achieves substantial improvements in real-world long-horizon manipulation tasks, demonstrating its broad effectiveness in practical robotic settings.
HiF-VLA integrates motion for bidirectional temporal reasoning in VLA models, improving long-horizon manipulation performance with minimal additional latency.
Motion bidirectional temporal reasoning hindsight priors foresight reasoning hindsight-modulated joint expert think-while-acting LIBERO-Long CALVIN ABC-D benchmarks real-world long-horizon manipulation tasks