Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection
Large language models (LLMs) have been widely applied across various domains of finance. Since their training data are largely derived from human-authored corpora, LLMs may inherit a range of human biases. Behavioral biases can lead to instability and uncertainty in decision-making, particularly when processing financial information. However, existing research on LLM bias has mainly focused on direct questioning or simplified, general-purpose settings, with limited consideration of the complex real-world financial environments and high-risk, context-sensitive, multilingual financial misinformation detection tasks (\mfmd). In this work, we propose \mfmdscen, a comprehensive benchmark for evaluating behavioral biases of LLMs in \mfmd across diverse economic scenarios. In collaboration with financial experts, we construct three types of complex financial scenarios: (i) role- and personality-based, (ii) role- and region-based, and (iii) role-based scenarios incorporating ethnicity and religious beliefs. We further develop a multilingual financial misinformation dataset covering English, Chinese, Greek, and Bengali. By integrating these scenarios with misinformation claims, \mfmdscen enables a systematic evaluation of 22 mainstream LLMs. Our findings reveal that pronounced behavioral biases persist across both commercial and open-source models. This project will be available at https://github.com/lzw108/FMD.
6964b5f3138cc47cbd765414
Zhiwei Liu
claimed_verified
2026-01-12T10:33:18.655Z
654d10d918c4ae0b4017fa4a
/avatars/a905ef9419ecffab40b17bdfcf23dce2.svg
Zhiwei Liu
lzw1008
user
lzw1008
6964b5f3138cc47cbd765415
Yupen Cao
6964b5f3138cc47cbd765416
Yuechen Jiang
claimed_verified
2026-04-07T08:49:32.107Z
67c07426a600107b41ed84fb
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/gI7Ok9-LvAjhg8eotX96g.png
Yuechen Jiang
CarolynJiang
user
CarolynJiang
6964b5f3138cc47cbd765417
Mohsinul Kabir
6964b5f3138cc47cbd765418
Polydoros Giannouris
6964b5f3138cc47cbd765419
Chen Xu
6964b5f3138cc47cbd76541a
Ziyang Xu
6964b5f3138cc47cbd76541b
Tianlei Zhu
6964b5f3138cc47cbd76541c
Tariquzzaman Faisal
6964b5f3138cc47cbd76541d
Triantafillos Papadopoulos
6964b5f3138cc47cbd76541e
Yan Wang
6964b5f3138cc47cbd76541f
Lingfei Qian
6964b5f3138cc47cbd765420
Xueqing Peng
6964b5f3138cc47cbd765421
Zhuohan Xie
6964b5f3138cc47cbd765422
Ye Yuan
6964b5f3138cc47cbd765423
Saeed Almheiri
6964b5f3138cc47cbd765424
Abdulrazzaq Alnajjar
6964b5f3138cc47cbd765425
Mingbin Chen
6964b5f3138cc47cbd765426
Harry Stuart
6964b5f3138cc47cbd765427
Paul Thompson
6964b5f3138cc47cbd765428
Prayag Tiwari
6964b5f3138cc47cbd765429
Alejandro Lopez-Lira
6964b5f3138cc47cbd76542a
Xue Liu
6964b5f3138cc47cbd76542b
Jimin Huang
claimed_verified
2026-01-12T11:08:29.266Z
63b58ed5889aa6707f0bb0f4
https://cdn-avatars.huggingface.co/v1/production/uploads/63b58ed5889aa6707f0bb0f4/znl74_aMswlV8VtHrfj3G.jpeg
Jimin Huang
jiminHuang
user
jiminHuang
6964b5f3138cc47cbd76542c
Sophia Ananiadou
Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection
Large language models (LLMs) have been widely applied across various domains of finance. Since their training data are largely derived from human-authored corpora, LLMs may inherit a range of human biases. Behavioral biases can lead to instability and uncertainty in decision-making, particularly when processing financial information. However, existing research on LLM bias has mainly focused on direct questioning or simplified, general-purpose settings, with limited consideration of the complex real-world financial environments and high-risk, context-sensitive, multilingual financial misinformation detection tasks (\mfmd). In this work, we propose \mfmdscen, a comprehensive benchmark for evaluating behavioral biases of LLMs in \mfmd across diverse economic scenarios. In collaboration with financial experts, we construct three types of complex financial scenarios: (i) role- and personality-based, (ii) role- and region-based, and (iii) role-based scenarios incorporating ethnicity and religious beliefs. We further develop a multilingual financial misinformation dataset covering English, Chinese, Greek, and Bengali. By integrating these scenarios with misinformation claims, \mfmdscen enables a systematic evaluation of 22 mainstream LLMs. Our findings reveal that pronounced behavioral biases persist across both commercial and open-source models. This project will be available at https://github.com/lzw108/FMD.
A comprehensive benchmark evaluates behavioral biases in large language models for multilingual financial misinformation detection across diverse economic scenarios.
large language models behavioral biases financial misinformation detection multilingual datasets economic scenarios financial expert collaboration