ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
While Multimodal Large Language Models (MLLMs) show promising performance in automated electrocardiogram interpretation, it remains unclear whether they genuinely perform actual step-by-step reasoning or just rely on superficial visual cues. To investigate this, we introduce ECG-Reasoning-Benchmark, a novel multi-turn evaluation framework comprising over 6,400 samples to systematically assess step-by-step reasoning across 17 core ECG diagnoses. Our comprehensive evaluation of state-of-the-art models reveals a critical failure in executing multi-step logical deduction. Although models possess the medical knowledge to retrieve clinical criteria for a diagnosis, they exhibit near-zero success rates (6% Completion) in maintaining a complete reasoning chain, primarily failing to ground the corresponding ECG findings to the actual visual evidence in the ECG signal. These results demonstrate that current MLLMs bypass actual visual interpretation, exposing a critical flaw in existing training paradigms and underscoring the necessity for robust, reasoning-centric medical AI. The code and data are available at https://github.com/Jwoo5/ecg-reasoning-benchmark.
69b8f8666b3f36c249fbc54c
Jungwoo Oh
claimed_verified
2026-03-19T13:03:41.655Z
67d74a08582e822d0c7c3a79
/avatars/1bd97075b8f2d7f6fb86cb8e957f781c.svg
Jungwoo Oh
Jwoo5
user
Jwoo5
69b8f8666b3f36c249fbc54d
Hyunseung Chung
69b8f8666b3f36c249fbc54e
Junhee Lee
69b8f8666b3f36c249fbc54f
Min-Gyu Kim
69b8f8666b3f36c249fbc550
Hangyul Yoon
claimed_verified
2026-03-19T13:03:44.198Z
62a4d58e81a4b10e93064ad6
/avatars/744d5cbc1745a26b816a458260aba050.svg
hangyulyoon
hangyulmd
user
hangyulmd
69b8f8666b3f36c249fbc551
Ki Seong Lee
69b8f8666b3f36c249fbc552
Youngchae Lee
69b8f8666b3f36c249fbc553
Muhan Yeo
69b8f8666b3f36c249fbc554
Edward Choi
ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
While Multimodal Large Language Models (MLLMs) show promising performance in automated electrocardiogram interpretation, it remains unclear whether they genuinely perform actual step-by-step reasoning or just rely on superficial visual cues. To investigate this, we introduce ECG-Reasoning-Benchmark, a novel multi-turn evaluation framework comprising over 6,400 samples to systematically assess step-by-step reasoning across 17 core ECG diagnoses. Our comprehensive evaluation of state-of-the-art models reveals a critical failure in executing multi-step logical deduction. Although models possess the medical knowledge to retrieve clinical criteria for a diagnosis, they exhibit near-zero success rates (6% Completion) in maintaining a complete reasoning chain, primarily failing to ground the corresponding ECG findings to the actual visual evidence in the ECG signal. These results demonstrate that current MLLMs bypass actual visual interpretation, exposing a critical flaw in existing training paradigms and underscoring the necessity for robust, reasoning-centric medical AI. The code and data are available at https://github.com/Jwoo5/ecg-reasoning-benchmark.