Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
Fine-tuning safety-aligned large language models (LLMs) can substantially compromise their safety. Previous approaches require many safety samples or calibration sets, which not only incur significant computational overhead during realignment but also lead to noticeable degradation in model utility. Contrary to this belief, we show that safety alignment can be fully recovered with only a single safety example, without sacrificing utility and at minimal cost. Remarkably, this recovery is effective regardless of the number of harmful examples used in fine-tuning or the size of the underlying model, and convergence is achieved within just a few epochs. Furthermore, we uncover the low-rank structure of the safety gradient, which explains why such efficient correction is possible. We validate our findings across five safety-aligned LLMs and multiple datasets, demonstrating the generality of our approach.
696077705b7998385e63950c
Jiawen Zhang
696077705b7998385e63950d
Lipeng He
claimed_verified
2026-01-09T08:35:07.378Z
6433707307bad11484af1d2a
https://cdn-avatars.huggingface.co/v1/production/uploads/6433707307bad11484af1d2a/-dSYst9tDZkR0FyWO14dp.png
Lipeng (Tony) He
ttttonyhe
user
ttttonyhe
696077705b7998385e63950e
Kejia Chen
696077705b7998385e63950f
Jian Lou
696077705b7998385e639510
Jian Liu
696077705b7998385e639511
Xiaohu Yang
696077705b7998385e639512
Ruoxi Jia
Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
Fine-tuning safety-aligned large language models (LLMs) can substantially compromise their safety. Previous approaches require many safety samples or calibration sets, which not only incur significant computational overhead during realignment but also lead to noticeable degradation in model utility. Contrary to this belief, we show that safety alignment can be fully recovered with only a single safety example, without sacrificing utility and at minimal cost. Remarkably, this recovery is effective regardless of the number of harmful examples used in fine-tuning or the size of the underlying model, and convergence is achieved within just a few epochs. Furthermore, we uncover the low-rank structure of the safety gradient, which explains why such efficient correction is possible. We validate our findings across five safety-aligned LLMs and multiple datasets, demonstrating the generality of our approach.
Safety alignment of large language models can be fully recovered with a single safety example, maintaining utility and achieving convergence in few epochs through identified low-rank gradient structures.
large language models fine-tuning safety alignment harmful examples low-rank structure safety gradient convergence