Aligned but Stereotypical? The Hidden Influence of System Prompts on Social Bias in LVLM-Based Text-to-Image Models
Large vision-language model (LVLM) based text-to-image (T2I) systems have become the dominant paradigm in image generation, yet whether they amplify social biases remains insufficiently understood. In this paper, we show that LVLM-based models produce markedly more socially biased images than non-LVLM-based models. We introduce a 1,024 prompt benchmark spanning four levels of linguistic complexity and evaluate demographic bias across multiple attributes in a systematic manner. Our analysis identifies system prompts, the predefined instructions guiding LVLMs, as a primary driver of biased behavior. Through decoded intermediate representations, token-probability diagnostics, and embedding-association analyses, we reveal how system prompts encode demographic priors that propagate into image synthesis. To this end, we propose FairPro, a training-free meta-prompting framework that enables LVLMs to self-audit and construct fairness-aware system prompts at test time. Experiments on two LVLM-based T2I models, SANA and Qwen-Image, show that FairPro substantially reduces demographic bias while preserving text-image alignment. We believe our findings provide deeper insight into the central role of system prompts in bias propagation and offer a practical, deployable approach for building more socially responsible T2I systems.
693290166d1060ca587a2803
NaHyeon Park
693290166d1060ca587a2804
Namin An
693290166d1060ca587a2805
Kunhee Kim
claimed_verified
2025-12-15T08:14:14.298Z
6314968cd8dc27b2f7f1f3e2
/avatars/3830bcff76b0408b1849fd9a44f7aedc.svg
Kunhee Kim
kunheekim
user
kunheekim
693290166d1060ca587a2806
Soyeon Yoon
693290166d1060ca587a2807
Jiahao Huo
claimed_verified
2026-02-17T15:49:12.915Z
64b180b63aecf08f78eae564
/avatars/ae62e9850dae2028ac020517b40a212f.svg
Jiahao
Z1zs
user
Z1zs
693290166d1060ca587a2808
Hyunjung Shim
Aligned but Stereotypical? The Hidden Influence of System Prompts on Social Bias in LVLM-Based Text-to-Image Models
Large vision-language model (LVLM) based text-to-image (T2I) systems have become the dominant paradigm in image generation, yet whether they amplify social biases remains insufficiently understood. In this paper, we show that LVLM-based models produce markedly more socially biased images than non-LVLM-based models. We introduce a 1,024 prompt benchmark spanning four levels of linguistic complexity and evaluate demographic bias across multiple attributes in a systematic manner. Our analysis identifies system prompts, the predefined instructions guiding LVLMs, as a primary driver of biased behavior. Through decoded intermediate representations, token-probability diagnostics, and embedding-association analyses, we reveal how system prompts encode demographic priors that propagate into image synthesis. To this end, we propose FairPro, a training-free meta-prompting framework that enables LVLMs to self-audit and construct fairness-aware system prompts at test time. Experiments on two LVLM-based T2I models, SANA and Qwen-Image, show that FairPro substantially reduces demographic bias while preserving text-image alignment. We believe our findings provide deeper insight into the central role of system prompts in bias propagation and offer a practical, deployable approach for building more socially responsible T2I systems.
LVLM-based T2I systems exhibit higher social bias compared to non-LVLM models, with system prompts identified as a key factor; FairPro reduces demographic bias without sacrificing alignment.
Large vision-language model text-to-image social biases benchmark linguistic complexity demographic bias system prompts decoded intermediate representations token-probability diagnostics embedding-association analyses FairPro text-image alignment