OxyGen: Unified KV Cache Management for Vision-Language-Action Models under Multi-Task Parallelism
Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) architecturally support such heterogeneous outputs, yet existing inference systems fail to achieve efficient multi-task parallelism for on-device deployment due to redundant computation and resource contention. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference paradigm that treats KV cache as a first-class shared resource across tasks and over time. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching decouples variable-length language decoding from fixed-rate action generation across control cycles. We implement this paradigm for π_{0.5}, the most popular MoT VLA, and evaluate under representative robotic configurations. OxyGen achieves up to 3.7times speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously without action quality degradation.
69b936e56b3f36c249fbc6ee
Xiangyu Li
claimed_verified
2026-03-17T17:34:13.630Z
64fe8d94e0dc35986bfcdab2
https://cdn-avatars.huggingface.co/v1/production/uploads/64fe8d94e0dc35986bfcdab2/ZEc_4fhWgVVi9vcqQUA7x.jpeg
Xiangyu Li
XXXXyu
user
XXXXyu
69b936e56b3f36c249fbc6ef
Huaizhi Tang
69b936e56b3f36c249fbc6f0
Xin Ding
69b936e56b3f36c249fbc6f1
Weijun Wang
69b936e56b3f36c249fbc6f2
Ting Cao
69b936e56b3f36c249fbc6f3
Yunxin Liu
OxyGen: Unified KV Cache Management for Vision-Language-Action Models under Multi-Task Parallelism
Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) architecturally support such heterogeneous outputs, yet existing inference systems fail to achieve efficient multi-task parallelism for on-device deployment due to redundant computation and resource contention. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference paradigm that treats KV cache as a first-class shared resource across tasks and over time. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching decouples variable-length language decoding from fixed-rate action generation across control cycles. We implement this paradigm for π_{0.5}, the most popular MoT VLA, and evaluate under representative robotic configurations. OxyGen achieves up to 3.7times speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously without action quality degradation.