Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendation
Most recommendation benchmarks evaluate how well a model imitates user behavior. In financial advisory, however, observed actions can be noisy or short-sighted under market volatility and may conflict with a user's long-term goals. Treating what users chose as the sole ground truth, therefore, conflates behavioral imitation with decision quality. We introduce Conv-FinRe, a conversational and longitudinal benchmark for stock recommendation that evaluates LLMs beyond behavior matching. Given an onboarding interview, step-wise market context, and advisory dialogues, models must generate rankings over a fixed investment horizon. Crucially, Conv-FinRe provides multi-view references that distinguish descriptive behavior from normative utility grounded in investor-specific risk preferences, enabling diagnosis of whether an LLM follows rational analysis, mimics user noise, or is driven by market momentum. We build the benchmark from real market data and human decision trajectories, instantiate controlled advisory conversations, and evaluate a suite of state-of-the-art LLMs. Results reveal a persistent tension between rational decision quality and behavioral alignment: models that perform well on utility-based ranking often fail to match user choices, whereas behaviorally aligned models can overfit short-term noise. The dataset is publicly released on Hugging Face, and the codebase is available on GitHub.
699e71d6dfbcf0b800aecb53
Yan Wang
claimed_verified
2026-02-25T17:29:21.546Z
65d76cc5b9b7b8bf88faa916
/avatars/d95232cd0c307efab6197ade1a66190b.svg
Yan Wang
YanAdjeNole
user
YanAdjeNole
699e71d6dfbcf0b800aecb54
Yi Han
699e71d6dfbcf0b800aecb55
Lingfei Qian
699e71d6dfbcf0b800aecb56
Yueru He
699e71d6dfbcf0b800aecb57
Xueqing Peng
699e71d6dfbcf0b800aecb58
Dongji Feng
699e71d6dfbcf0b800aecb59
Zhuohan Xie
699e71d6dfbcf0b800aecb5a
Vincent Jim Zhang
699e71d6dfbcf0b800aecb5b
Rosie Guo
699e71d6dfbcf0b800aecb5c
Fengran Mo
699e71d6dfbcf0b800aecb5d
Jimin Huang
699e71d6dfbcf0b800aecb5e
Yankai Chen
699e71d6dfbcf0b800aecb5f
Xue Liu
699e71d6dfbcf0b800aecb60
Jian-Yun Nie
Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendation
Most recommendation benchmarks evaluate how well a model imitates user behavior. In financial advisory, however, observed actions can be noisy or short-sighted under market volatility and may conflict with a user's long-term goals. Treating what users chose as the sole ground truth, therefore, conflates behavioral imitation with decision quality. We introduce Conv-FinRe, a conversational and longitudinal benchmark for stock recommendation that evaluates LLMs beyond behavior matching. Given an onboarding interview, step-wise market context, and advisory dialogues, models must generate rankings over a fixed investment horizon. Crucially, Conv-FinRe provides multi-view references that distinguish descriptive behavior from normative utility grounded in investor-specific risk preferences, enabling diagnosis of whether an LLM follows rational analysis, mimics user noise, or is driven by market momentum. We build the benchmark from real market data and human decision trajectories, instantiate controlled advisory conversations, and evaluate a suite of state-of-the-art LLMs. Results reveal a persistent tension between rational decision quality and behavioral alignment: models that perform well on utility-based ranking often fail to match user choices, whereas behaviorally aligned models can overfit short-term noise. The dataset is publicly released on Hugging Face, and the codebase is available on GitHub.
A new conversational financial recommendation benchmark evaluates large language models' ability to balance rational decision-making with user behavior alignment using multi-view references derived from real market data and human decision trajectories.
recommendation benchmarks large language models financial advisory behavioral imitation decision quality multi-view references investor-specific risk preferences rational analysis market volatility user behavior alignment