Distilling Feedback into Memory-as-a-Tool
We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.
6964aa85138cc47cbd7653f1
Víctor Gallego
claimed_verified
2026-01-12T10:33:24.342Z
5fad8602b8423e1d80b8a965
https://cdn-avatars.huggingface.co/v1/production/uploads/5fad8602b8423e1d80b8a965/tRqTwcZmrGka8c1vFq2wX.jpeg
Victor Gallego
vicgalle
user
vicgalle
Distilling Feedback into Memory-as-a-Tool
We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls. We evaluate this method on the Rubric Feedback Bench, a novel dataset for rubric-based learning. Experiments demonstrate that our augmented LLMs rapidly match the performance of test-time refinement pipelines while drastically reducing inference cost.
A framework converts transient critiques into retrievable guidelines using a file-based memory system and agent-controlled tool calls, enabling LLMs to match test-time refinement performance with reduced inference costs.
LLMs test-time refinement inference-time reasoning file-based memory system agent-controlled tool calls Rubric Feedback Bench rubric-based learning