Retrieval-augmented reasoning with lean language models
This technical report details a novel approach to combining reasoning and
retrieval augmented generation (RAG) within a single, lean language model
architecture. While existing RAG systems typically rely on large-scale models
and external APIs, our work addresses the increasing demand for performant and
privacy-preserving solutions deployable in resource-constrained or secure
environments. Building on recent developments in test-time scaling and
small-scale reasoning models, we develop a retrieval augmented conversational
agent capable of interpreting complex, domain-specific queries using a
lightweight backbone model. Our system integrates a dense retriever with
fine-tuned Qwen2.5-Instruct models, using synthetic query generation and
reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a
curated corpus, in this case, the NHS A-to-Z condition pages. We explore the
impact of summarisation-based document compression, synthetic data design, and
reasoning-aware fine-tuning on model performance. Evaluation against both
non-reasoning and general-purpose lean models demonstrates that our
domain-specific fine-tuning approach yields substantial gains in answer
accuracy and consistency, approaching frontier-level performance while
remaining feasible for local deployment. All implementation details and code
are publicly released to support reproducibility and adaptation across domains.
68a622599e4b49496aac68ab
Ryan Sze-Yin Chan
claimed_verified
2025-08-20T19:39:13.251Z
639080fed2cf01fdfe322a8e
https://cdn-avatars.huggingface.co/v1/production/uploads/1671645871849-639080fed2cf01fdfe322a8e.jpeg
Ryan Chan
rchan26
user
rchan26
68a622599e4b49496aac68ac
Federico Nanni
68a622599e4b49496aac68ad
Tomas Lazauskas
68a622599e4b49496aac68ae
Rosie Wood
68a622599e4b49496aac68af
Penelope Yong
68a622599e4b49496aac68b0
Lionel Tarassenko
68a622599e4b49496aac68b1
Mark Girolami
68a622599e4b49496aac68b2
James Geddes
68a622599e4b49496aac68b3
Andrew Duncan
Retrieval-augmented reasoning with lean language models
This technical report details a novel approach to combining reasoning and
retrieval augmented generation (RAG) within a single, lean language model
architecture. While existing RAG systems typically rely on large-scale models
and external APIs, our work addresses the increasing demand for performant and
privacy-preserving solutions deployable in resource-constrained or secure
environments. Building on recent developments in test-time scaling and
small-scale reasoning models, we develop a retrieval augmented conversational
agent capable of interpreting complex, domain-specific queries using a
lightweight backbone model. Our system integrates a dense retriever with
fine-tuned Qwen2.5-Instruct models, using synthetic query generation and
reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a
curated corpus, in this case, the NHS A-to-Z condition pages. We explore the
impact of summarisation-based document compression, synthetic data design, and
reasoning-aware fine-tuning on model performance. Evaluation against both
non-reasoning and general-purpose lean models demonstrates that our
domain-specific fine-tuning approach yields substantial gains in answer
accuracy and consistency, approaching frontier-level performance while
remaining feasible for local deployment. All implementation details and code
are publicly released to support reproducibility and adaptation across domains.
A retrieval-augmented conversational agent using a lightweight model achieves high accuracy and consistency in domain-specific queries through fine-tuning and document compression.
reasoning and retrieval augmented generation (RAG) test-time scaling small-scale reasoning models dense retriever Qwen2.5-Instruct synthetic query generation reasoning traces summarisation-based document compression synthetic data design reasoning-aware fine-tuning domain-specific fine-tuning