Shallow-π: Knowledge Distillation for Flow-based VLAs
The growing demand for real-time robotic deployment necessitates fast and on-device inference for vision-language-action (VLA) models. Within the VLA literature, efficiency has been extensively studied at the token level, such as visual token pruning. In contrast, systematic transformer layer reduction has received limited attention and, to the best of our knowledge, has not been explored for flow-based VLA models under knowledge distillation. In this work, we propose Shallow-pi, a principled knowledge distillation framework that aggressively reduces the transformer depth of both the VLM backbone and the flow-based action head, compressing the model from 18 to 6 layers. Shallow-pi achieves over two times faster inference with less than one percent absolute drop in success rate on standard manipulation benchmarks, establishing state-of-the-art performance among reduced VLA models. Crucially, we validate our approach through industrial-scale real-world experiments on Jetson Orin and Jetson Thor across multiple robot platforms, including humanoid systems, in complex and dynamic manipulation scenarios.
697ac611df3e800774f13bdb
Boseong Jeon
claimed_verified
2026-01-29T09:16:20.722Z
653b9b677c2fb22dda9020da
/avatars/3cfe55566205f66ec1919479f60f3017.svg
Boseong Jeon
bf-jeon
user
bf-jeon
697ac611df3e800774f13bdc
Yunho Choi
697ac611df3e800774f13bdd
Taehan Kim
Shallow-π: Knowledge Distillation for Flow-based VLAs
The growing demand for real-time robotic deployment necessitates fast and on-device inference for vision-language-action (VLA) models. Within the VLA literature, efficiency has been extensively studied at the token level, such as visual token pruning. In contrast, systematic transformer layer reduction has received limited attention and, to the best of our knowledge, has not been explored for flow-based VLA models under knowledge distillation. In this work, we propose Shallow-pi, a principled knowledge distillation framework that aggressively reduces the transformer depth of both the VLM backbone and the flow-based action head, compressing the model from 18 to 6 layers. Shallow-pi achieves over two times faster inference with less than one percent absolute drop in success rate on standard manipulation benchmarks, establishing state-of-the-art performance among reduced VLA models. Crucially, we validate our approach through industrial-scale real-world experiments on Jetson Orin and Jetson Thor across multiple robot platforms, including humanoid systems, in complex and dynamic manipulation scenarios.
A knowledge distillation framework called Shallow-pi is presented that reduces transformer depth in vision-language-action models, achieving faster inference with minimal performance loss in real-world robotic applications.
knowledge distillation transformer layer reduction vision-language-action models VLA models token level visual token pruning flow-based VLA models model compression real-time robotic deployment inference speed success rate manipulation benchmarks Jetson Orin Jetson Thor robot platforms humanoid systems dynamic manipulation scenarios