LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only through lightweight adapters. Despite sharing base model weights, each agent independently builds and stores its own KV cache for the same long, tool-augmented trajectories, incurring substantial memory and compute overhead. Existing KV cache sharing methods largely overlook this multi-LoRA setting. We observe that, across agents, cache differences are dominated by adapter outputs, while activations from the shared pretrained backbone remain highly similar. Based on this observation, we propose LRAgent, a KV cache sharing framework for multi-LoRA agents that decomposes the cache into a shared base component from the pretrained weights and an adapter-dependent component from LoRA weights. LRAgent reduces memory overhead by sharing the base component and storing the adapter component in its inherent low-rank form, and further reduces compute overhead, enabled by shared-A multi-LoRA architectures, by also sharing the low-rank cache and avoiding redundant computations for contexts already processed by other agents. To efficiently reconstruct adapter contributions at runtime, we introduce Flash-LoRA-Attention, a kernel that reorders attention computation to avoid materializing the low-rank cache to full dimension. LRAgent achieves throughput and time-to-first-token latency close to fully shared caching, while preserving accuracy near the non-shared caching baseline across agentic question-answering benchmarks.
6982ebf29084cb4f0ecb5886
Hyesung Jeon
claimed_verified
2026-03-18T08:46:59.371Z
6400208acafc9d549863af59
/avatars/6c383c810a038ce61e803f1d75132471.svg
Hyesung Jeon
hjeon2k
user
hjeon2k
6982ebf29084cb4f0ecb5887
Hyeongju Ha
claimed_verified
2026-03-18T08:46:57.604Z
6757ab225ce1ff3af158f149
/avatars/cd0c94a389ed9e50c73abc30ef043b2d.svg
Hyeongju Ha
Hyeongju97
user
Hyeongju97
6982ebf29084cb4f0ecb5888
Jae-Joon Kim
LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only through lightweight adapters. Despite sharing base model weights, each agent independently builds and stores its own KV cache for the same long, tool-augmented trajectories, incurring substantial memory and compute overhead. Existing KV cache sharing methods largely overlook this multi-LoRA setting. We observe that, across agents, cache differences are dominated by adapter outputs, while activations from the shared pretrained backbone remain highly similar. Based on this observation, we propose LRAgent, a KV cache sharing framework for multi-LoRA agents that decomposes the cache into a shared base component from the pretrained weights and an adapter-dependent component from LoRA weights. LRAgent reduces memory overhead by sharing the base component and storing the adapter component in its inherent low-rank form, and further reduces compute overhead, enabled by shared-A multi-LoRA architectures, by also sharing the low-rank cache and avoiding redundant computations for contexts already processed by other agents. To efficiently reconstruct adapter contributions at runtime, we introduce Flash-LoRA-Attention, a kernel that reorders attention computation to avoid materializing the low-rank cache to full dimension. LRAgent achieves throughput and time-to-first-token latency close to fully shared caching, while preserving accuracy near the non-shared caching baseline across agentic question-answering benchmarks.
LRAgent is a KV cache sharing framework for multi-LoRA agents that decomposes cache into shared and adapter-dependent components, reducing memory and compute overhead while maintaining accuracy.
multi-LoRA KV cache sharing LoRA weights pretrained backbone Flash-LoRA-Attention low-rank cache shared-$A$ multi-LoRA attention computation