Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
The emergence of Large Language Models (LLMs) has illuminated the potential for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a Utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
69d8563479548974da946445
Jiawei Chen
claimed_verified
2026-04-10T07:44:02.280Z
654c7fbe6b51714c2a6ff590
/avatars/db217415c56730872b9a807f3afb4e5b.svg
Jiawei Chen
jiawei-ucas
user
jiawei-ucas
69d8563479548974da946446
Ruoxi Xu
69d8563479548974da946447
Boxi Cao
69d8563479548974da946448
Ruotong Pan
69d8563479548974da946449
Yunfei Zhang
69d8563479548974da94644a
Yifei Hu
69d8563479548974da94644b
Yong Du
69d8563479548974da94644c
Tingting Gao
69d8563479548974da94644d
Yaojie Lu
69d8563479548974da94644e
Yingfei Sun
69d8563479548974da94644f
Xianpei Han
69d8563479548974da946450
Le Sun
69d8563479548974da946451
Xiangyu Wu
69d8563479548974da946452
Hongyu Lin
Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
The emergence of Large Language Models (LLMs) has illuminated the potential for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a Utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
OmniBehavior benchmark reveals that current LLMs fail to accurately simulate complex real-world user behaviors due to structural biases and limited behavioral diversity.
Large Language Models user simulation benchmark real-world data long-horizon cross-scenario heterogeneous behavioral patterns causal chains state-of-the-art LLMs structural bias hyper-activity persona homogenization Utopian bias individual differences long-tail behaviors