iFlyBot-VLA Technical Report
We introduce iFlyBot-VLA, a large-scale Vision-Language-Action (VLA) model
trained under a novel framework. The main contributions are listed as follows:
(1) a latent action model thoroughly trained on large-scale human and robotic
manipulation videos; (2) a dual-level action representation framework that
jointly supervises both the Vision-Language Model (VLM) and the action expert
during training; (3) a mixed training strategy that combines robot trajectory
data with general QA and spatial QA datasets, effectively enhancing the 3D
perceptual and reasoning capabilities of the VLM backbone. Specifically, the
VLM is trained to predict two complementary forms of actions: latent actions,
derived from our latent action model pretrained on cross-embodiment
manipulation data, which capture implicit high-level intentions; and structured
discrete action tokens, obtained through frequency-domain transformations of
continuous control signals, which encode explicit low-level dynamics. This dual
supervision aligns the representation spaces of language, vision, and action,
enabling the VLM to directly contribute to action generation. Experimental
results on the LIBERO Franka benchmark demonstrate the superiority of our
frame-work, while real-world evaluations further show that iFlyBot-VLA achieves
competitive success rates across diverse and challenging manipulation tasks.
Furthermore, we plan to open-source a portion of our self-constructed dataset
to support future research in the community
690abf69d70e173c84528fde
Yuan Zhang
690abf69d70e173c84528fdf
Chenyu Xue
690abf69d70e173c84528fe0
Wenjie Xu
690abf69d70e173c84528fe1
Chao Ji
690abf69d70e173c84528fe2
Jiajia wu
690abf69d70e173c84528fe3
Jia Pan
iFlyBot-VLA Technical Report
We introduce iFlyBot-VLA, a large-scale Vision-Language-Action (VLA) model
trained under a novel framework. The main contributions are listed as follows:
(1) a latent action model thoroughly trained on large-scale human and robotic
manipulation videos; (2) a dual-level action representation framework that
jointly supervises both the Vision-Language Model (VLM) and the action expert
during training; (3) a mixed training strategy that combines robot trajectory
data with general QA and spatial QA datasets, effectively enhancing the 3D
perceptual and reasoning capabilities of the VLM backbone. Specifically, the
VLM is trained to predict two complementary forms of actions: latent actions,
derived from our latent action model pretrained on cross-embodiment
manipulation data, which capture implicit high-level intentions; and structured
discrete action tokens, obtained through frequency-domain transformations of
continuous control signals, which encode explicit low-level dynamics. This dual
supervision aligns the representation spaces of language, vision, and action,
enabling the VLM to directly contribute to action generation. Experimental
results on the LIBERO Franka benchmark demonstrate the superiority of our
frame-work, while real-world evaluations further show that iFlyBot-VLA achieves
competitive success rates across diverse and challenging manipulation tasks.
Furthermore, we plan to open-source a portion of our self-constructed dataset
to support future research in the community
iFlyBot-VLA, a large-scale VLA model, uses a latent action model and dual-level action representation to enhance 3D perceptual and reasoning capabilities, achieving superior performance in manipulation tasks.
latent action model dual-level action representation Vision-Language Model (VLM) action expert mixed training strategy robot trajectory data general QA spatial QA latent actions structured discrete action tokens frequency-domain transformations continuous control signals LIBERO Franka benchmark