C-ΔΘ: Circuit-Restricted Weight Arithmetic for Selective Refusal
Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity. Activation steering is widely used, but it requires runtime hooks and scales cost with the number of generations; conditional variants improve selectivity by gating when steering is applied but still retain an inference-time control path. We ask whether selective refusal can be moved entirely offline: can a mechanistic understanding of category-specific refusal be distilled into a circuit-restricted weight update that deploys as a standard checkpoint? We propose C-Δθ: Circuit Restricted Weight Arithmetic, which (i) localizes refusal-causal computation as a sparse circuit using EAP-IG and (ii) computes a constrained weight update ΔθC supported only on that circuit (typically <5% of parameters). Applying ΔθC yields a drop-in edited checkpoint with no inference-time hooks, shifting cost from per-request intervention to a one-time offline update. We evaluate category-targeted selectivity and capability retention on refusal and utility benchmarks.
698c6df0eb12ea7453916829
Aditya Kasliwal
698c6df0eb12ea745391682a
Pratinav Seth
claimed_verified
2026-02-11T12:34:30.458Z
66fce04d927ec45504514afd
https://cdn-avatars.huggingface.co/v1/production/uploads/66fce04d927ec45504514afd/lA_bTErY7JywT6xbzdflo.jpeg
Pratinav Seth
pratinavsetharya
user
pratinavsetharya
698c6df0eb12ea745391682b
Vinay Kumar Sankarapu
C-ΔΘ: Circuit-Restricted Weight Arithmetic for Selective Refusal
Modern deployments require LLMs to enforce safety policies at scale, yet many controls rely on inference-time interventions that add recurring compute cost and serving complexity. Activation steering is widely used, but it requires runtime hooks and scales cost with the number of generations; conditional variants improve selectivity by gating when steering is applied but still retain an inference-time control path. We ask whether selective refusal can be moved entirely offline: can a mechanistic understanding of category-specific refusal be distilled into a circuit-restricted weight update that deploys as a standard checkpoint? We propose C-Δθ: Circuit Restricted Weight Arithmetic, which (i) localizes refusal-causal computation as a sparse circuit using EAP-IG and (ii) computes a constrained weight update ΔθC supported only on that circuit (typically <5% of parameters). Applying ΔθC yields a drop-in edited checkpoint with no inference-time hooks, shifting cost from per-request intervention to a one-time offline update. We evaluate category-targeted selectivity and capability retention on refusal and utility benchmarks.
Offline selective refusal in large language models is achieved through circuit-restricted weight updates that eliminate runtime intervention costs while maintaining performance.
activation steering conditional variants EAP-IG circuit-restricted weight arithmetic refusal-causal computation weight update inference-time hooks mechanistic understanding category-specific refusal selectivity capability retention