Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs.
698adc4e1b2dc6b37d61b29f
Pengrui Han
claimed_verified
2026-02-11T22:17:01.630Z
664263ea8b2d38e53f04079c
https://cdn-avatars.huggingface.co/v1/production/uploads/664263ea8b2d38e53f04079c/_7og0ggbPpEHcH6g__DF0.jpeg
Pengrui Han
barryhpr
user
barryhpr
698adc4e1b2dc6b37d61b2a0
Xueqiang Xu
claimed_verified
2026-02-11T22:16:59.753Z
66a3f1c4c38ce500371fd8d4
/avatars/381de938091f1a5c179eef72aa247bbf.svg
Xueqiang Xu
XueqiangXu
user
XueqiangXu
698adc4e1b2dc6b37d61b2a1
Keyang Xuan
claimed_verified
2026-02-11T11:19:22.360Z
687ebb87416fcff56959d817
/avatars/8f13090dd6179bb18e8c8d205fd20131.svg
Keyang Xuan
keyangx3
user
keyangx3
698adc4e1b2dc6b37d61b2a2
Peiyang Song
claimed_verified
2026-02-11T11:18:00.362Z
649c5cf5c1ae48cf4d7dda34
https://cdn-avatars.huggingface.co/v1/production/uploads/649c5cf5c1ae48cf4d7dda34/bSJXATqkqBn8ypUy0OezY.jpeg
Peiyang Song
p-song1
user
p-song1
698adc4e1b2dc6b37d61b2a3
Siru Ouyang
698adc4e1b2dc6b37d61b2a4
Runchu Tian
698adc4e1b2dc6b37d61b2a5
Yuqing Jiang
698adc4e1b2dc6b37d61b2a6
Cheng Qian
698adc4e1b2dc6b37d61b2a7
Pengcheng Jiang
698adc4e1b2dc6b37d61b2a8
Jiashuo Sun
698adc4e1b2dc6b37d61b2a9
Junxia Cui
698adc4e1b2dc6b37d61b2aa
Ming Zhong
698adc4e1b2dc6b37d61b2ab
Ge Liu
698adc4e1b2dc6b37d61b2ac
Jiawei Han
698adc4e1b2dc6b37d61b2ad
Jiaxuan You
Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs.
STEER2ADAPT is a lightweight framework that adapts large language models by composing steering vectors from reusable semantic prior subspaces, enabling efficient and flexible task adaptation through linear combinations of basis vectors.
activation steering large language models steering vectors semantic prior subspace linear combination inference-time adaptation data efficiency stability transparency