Found-RL: foundation model-enhanced reinforcement learning for autonomous driving
Reinforcement Learning (RL) has emerged as a dominant paradigm for end-to-end autonomous driving (AD). However, RL suffers from sample inefficiency and a lack of semantic interpretability in complex scenarios. Foundation Models, particularly Vision-Language Models (VLMs), can mitigate this by offering rich, context-aware knowledge, yet their high inference latency hinders deployment in high-frequency RL training loops. To bridge this gap, we present Found-RL, a platform tailored to efficiently enhance RL for AD using foundation models. A core innovation is the asynchronous batch inference framework, which decouples heavy VLM reasoning from the simulation loop, effectively resolving latency bottlenecks to support real-time learning. We introduce diverse supervision mechanisms: Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to effectively distill expert-like VLM action suggestions into the RL policy. Additionally, we adopt high-throughput CLIP for dense reward shaping. We address CLIP's dynamic blindness via Conditional Contrastive Action Alignment, which conditions prompts on discretized speed/command and yields a normalized, margin-based bonus from context-specific action-anchor scoring. Found-RL provides an end-to-end pipeline for fine-tuned VLM integration and shows that a lightweight RL model can achieve near-VLM performance compared with billion-parameter VLMs while sustaining real-time inference (approx. 500 FPS). Code, data, and models will be publicly available at https://github.com/ys-qu/found-rl.
698f40ff0cf1f9a6bbe7fb45
Yansong Qu
claimed_verified
2026-02-17T15:51:40.369Z
66e73f1b55f01fbea93f959f
/avatars/efe55aca1f9e744a1e2d639901fa6baa.svg
Yansong Qu
ys-qu
user
ys-qu
698f40ff0cf1f9a6bbe7fb46
Zihao Sheng
698f40ff0cf1f9a6bbe7fb47
Zilin Huang
698f40ff0cf1f9a6bbe7fb48
Jiancong Chen
698f40ff0cf1f9a6bbe7fb49
Yuhao Luo
698f40ff0cf1f9a6bbe7fb4a
Tianyi Wang
698f40ff0cf1f9a6bbe7fb4b
Yiheng Feng
698f40ff0cf1f9a6bbe7fb4c
Samuel Labi
698f40ff0cf1f9a6bbe7fb4d
Sikai Chen
Found-RL: foundation model-enhanced reinforcement learning for autonomous driving
Reinforcement Learning (RL) has emerged as a dominant paradigm for end-to-end autonomous driving (AD). However, RL suffers from sample inefficiency and a lack of semantic interpretability in complex scenarios. Foundation Models, particularly Vision-Language Models (VLMs), can mitigate this by offering rich, context-aware knowledge, yet their high inference latency hinders deployment in high-frequency RL training loops. To bridge this gap, we present Found-RL, a platform tailored to efficiently enhance RL for AD using foundation models. A core innovation is the asynchronous batch inference framework, which decouples heavy VLM reasoning from the simulation loop, effectively resolving latency bottlenecks to support real-time learning. We introduce diverse supervision mechanisms: Value-Margin Regularization (VMR) and Advantage-Weighted Action Guidance (AWAG) to effectively distill expert-like VLM action suggestions into the RL policy. Additionally, we adopt high-throughput CLIP for dense reward shaping. We address CLIP's dynamic blindness via Conditional Contrastive Action Alignment, which conditions prompts on discretized speed/command and yields a normalized, margin-based bonus from context-specific action-anchor scoring. Found-RL provides an end-to-end pipeline for fine-tuned VLM integration and shows that a lightweight RL model can achieve near-VLM performance compared with billion-parameter VLMs while sustaining real-time inference (approx. 500 FPS). Code, data, and models will be publicly available at https://github.com/ys-qu/found-rl.
Found-RL integrates vision-language models with reinforcement learning for autonomous driving, addressing sample efficiency and latency issues through asynchronous inference and specialized supervision mechanisms.
Reinforcement Learning Vision-Language Models asynchronous batch inference Value-Margin Regularization Advantage-Weighted Action Guidance CLIP Conditional Contrastive Action Alignment real-time learning end-to-end pipeline