Conditioned Activation Transport for T2I Safety Steering
Despite their impressive capabilities, current Text-to-Image (T2I) models remain prone to generating unsafe and toxic content. While activation steering offers a promising inference-time intervention, we observe that linear activation steering frequently degrades image quality when applied to benign prompts. To address this trade-off, we first construct SafeSteerDataset, a contrastive dataset containing 2300 safe and unsafe prompt pairs with high cosine similarity. Leveraging this data, we propose Conditioned Activation Transport (CAT), a framework that employs a geometry-based conditioning mechanism and nonlinear transport maps. By conditioning transport maps to activate only within unsafe activation regions, we minimize interference with benign queries. We validate our approach on two state-of-the-art architectures: Z-Image and Infinity. Experiments demonstrate that CAT generalizes effectively across these backbones, significantly reducing Attack Success Rate while maintaining image fidelity compared to unsteered generations. Warning: This paper contains potentially offensive text and images.
69a800dd2833ff1122ff66a3
Maciej Chrabąszcz
claimed_verified
2026-03-05T09:59:57.003Z
635270e36cfb8f14981312e7
https://cdn-avatars.huggingface.co/v1/production/uploads/635270e36cfb8f14981312e7/FpqjdhABEDuxWR8k2zOwA.jpeg
Maciej Chrabąszcz
mchraba
user
mchraba
69a800dd2833ff1122ff66a4
Aleksander Szymczyk
69a800dd2833ff1122ff66a5
Jan Dubiński
69a800dd2833ff1122ff66a6
Tomasz Trzciński
69a800dd2833ff1122ff66a7
Franziska Boenisch
69a800dd2833ff1122ff66a8
Adam Dziedzic
Conditioned Activation Transport for T2I Safety Steering
Despite their impressive capabilities, current Text-to-Image (T2I) models remain prone to generating unsafe and toxic content. While activation steering offers a promising inference-time intervention, we observe that linear activation steering frequently degrades image quality when applied to benign prompts. To address this trade-off, we first construct SafeSteerDataset, a contrastive dataset containing 2300 safe and unsafe prompt pairs with high cosine similarity. Leveraging this data, we propose Conditioned Activation Transport (CAT), a framework that employs a geometry-based conditioning mechanism and nonlinear transport maps. By conditioning transport maps to activate only within unsafe activation regions, we minimize interference with benign queries. We validate our approach on two state-of-the-art architectures: Z-Image and Infinity. Experiments demonstrate that CAT generalizes effectively across these backbones, significantly reducing Attack Success Rate while maintaining image fidelity compared to unsteered generations. Warning: This paper contains potentially offensive text and images.
Activation steering framework using geometric conditioning and nonlinear transport maps reduces unsafe content generation while preserving image quality in text-to-image models.
Text-to-Image activation steering nonlinear transport maps geometry-based conditioning SafeSteerDataset Z-Image Infinity attack success rate image fidelity