Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach
Vision-Language-Action (VLA) models, trained via flow-matching or diffusion objectives, excel at learning complex behaviors from large-scale, multi-modal datasets (e.g., human teleoperation, scripted policies). However, since VLAs incorporate diverse data modes in the pre-training stage, and the finetuning dataset often contains demonstration data collected in a kinematically suboptimal or undesirable way, it exists redundant action modes that are irrelevant to the success action modes of the downstream task. Specifically, we observe a critical inference-time fragility among various sampled noises after supervised finetuning of pre-trained VLAs. In this paper, we attribute this instability to the distribution shift between the VLA policy and the policy induced by stable success modes of the downstream task dataset. Thus, we propose TACO, a test-time-scaling (TTS) framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks. The VLA models integrated with TACO can execute the actions with maximum pseudo-count from all sampled action chunks, thereby preventing distribution shifts while preserving the generalization ability of VLAs since the constraint is applied only during inference. Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits compared to RL update, especially for flow or diffusion-based VLAs which are difficult to perform RL update due to denoising process. Extensive experiments across four simulation benchmarks (RoboTwin2.0, Robotwin, LIBERO, SimplerEnv) and a dual-arm platform demonstrate that our method significantly improves the inference stability and success rates in downstream-task adaptations.
69313ee82d1e5b0a7d84da77
Siyuan Yang
69313ee82d1e5b0a7d84da78
Yang Zhang
claimed_verified
2025-12-04T08:43:32.201Z
667be442c8c087a184094892
https://cdn-avatars.huggingface.co/v1/production/uploads/667be442c8c087a184094892/rVn2sKB9TuABE-sIzxbAx.jpeg
Yang Zhang
breezeyoung
user
breezeyoung
69313ee82d1e5b0a7d84da79
Haoran He
admin_assigned
2025-12-04T10:43:20.162Z
6672937ceac0fb1b9e516595
/avatars/5eea5657016572f60b0ecd0fa9a7dae4.svg
haoran he
haoranhe
user
haoranhe
69313ee82d1e5b0a7d84da7a
Ling Pan
69313ee82d1e5b0a7d84da7b
Xiu Li
69313ee82d1e5b0a7d84da7c
Chenjia Bai
69313ee82d1e5b0a7d84da7d
Xuelong Li
Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach
Vision-Language-Action (VLA) models, trained via flow-matching or diffusion objectives, excel at learning complex behaviors from large-scale, multi-modal datasets (e.g., human teleoperation, scripted policies). However, since VLAs incorporate diverse data modes in the pre-training stage, and the finetuning dataset often contains demonstration data collected in a kinematically suboptimal or undesirable way, it exists redundant action modes that are irrelevant to the success action modes of the downstream task. Specifically, we observe a critical inference-time fragility among various sampled noises after supervised finetuning of pre-trained VLAs. In this paper, we attribute this instability to the distribution shift between the VLA policy and the policy induced by stable success modes of the downstream task dataset. Thus, we propose TACO, a test-time-scaling (TTS) framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks. The VLA models integrated with TACO can execute the actions with maximum pseudo-count from all sampled action chunks, thereby preventing distribution shifts while preserving the generalization ability of VLAs since the constraint is applied only during inference. Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits compared to RL update, especially for flow or diffusion-based VLAs which are difficult to perform RL update due to denoising process. Extensive experiments across four simulation benchmarks (RoboTwin2.0, Robotwin, LIBERO, SimplerEnv) and a dual-arm platform demonstrate that our method significantly improves the inference stability and success rates in downstream-task adaptations.
TACO, a test-time-scaling framework with a pseudo-count estimator, enhances the inference stability and success rates of Vision-Language-Action models in downstream tasks by preventing distribution shifts.
flow-matching diffusion objectives Vision-Language-Action models pre-training finetuning distribution shift test-time-scaling pseudo-count estimator action chunks anti-exploration principle offline reinforcement learning gradient-free denoising process RoboTwin2.0 Robotwin LIBERO SimplerEnv dual-arm platform