How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities
Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerEval, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research.
69a79b682833ff1122ff63fb
Ziwen Xu
claimed_verified
2026-04-02T08:18:16.448Z
6549caee44e75a7de4fee2fa
/avatars/5aea69671eb1299aaaa948d888b4b64f.svg
Xu Ziwen
xzwnlp
user
xzwnlp
69a79b682833ff1122ff63fc
Kewei Xu
69a79b682833ff1122ff63fd
Haoming Xu
69a79b682833ff1122ff63fe
Haiwen Hong
69a79b682833ff1122ff63ff
Longtao Huang
69a79b682833ff1122ff6400
Hui Xue
69a79b682833ff1122ff6401
Ningyu Zhang
claimed_verified
2026-03-06T13:57:02.019Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
Ningyu
user
Ningyu
69a79b682833ff1122ff6402
Yongliang Shen
claimed_verified
2026-04-07T08:48:42.573Z
5e1058e9fcf41d740b69966d
/avatars/ce74839ba871f2b54313a670a233ba82.svg
Yongliang Shen
tricktreat
user
tricktreat
69a79b682833ff1122ff6403
Guozhou Zheng
69a79b682833ff1122ff6404
Huajun Chen
69a79b682833ff1122ff6405
Shumin Deng
How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities
Large Language Models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors, ranging from misaligned intent to inconsistent personality, pose significant risks. We introduce SteerEval, a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. Each domain is structured into three specification levels: L1 (what to express), L2 (how to express), and L3 (how to instantiate), connecting high-level behavioral intent to concrete textual output. Using SteerEval, we systematically evaluate contemporary steering methods, revealing that control often degrades at finer-grained levels. Our benchmark offers a principled and interpretable framework for safe and controllable LLM behavior, serving as a foundation for future research.
SteerEval is a hierarchical benchmark for evaluating large language model controllability across language features, sentiment, and personality domains with three specification levels.
Large Language Models controllability hierarchical benchmark language features sentiment personality steering methods behavioral intent textual output