PARSA-Bench: A Comprehensive Persian Audio-Language Model Benchmark
Persian poses unique audio understanding challenges through its classical poetry, traditional music, and pervasive code-switching - none captured by existing benchmarks. We introduce PARSA-Bench (Persian Audio Reasoning and Speech Assessment Benchmark), the first benchmark for evaluating large audio-language models on Persian language and culture, comprising 16 tasks and over 8,000 samples across speech understanding, paralinguistic analysis, and cultural audio understanding. Ten tasks are newly introduced, including poetry meter and style detection, traditional Persian music understanding, and code-switching detection. Text-only baselines consistently outperform audio counterparts, suggesting models may not leverage audio-specific information beyond what transcription alone provides. Culturally-grounded tasks expose a qualitatively distinct failure mode: all models perform near random chance on vazn detection regardless of scale, suggesting prosodic perception remains beyond the reach of current models. The dataset is publicly available at https://huggingface.co/datasets/MohammadJRanjbar/PARSA-Bench
69b9ed755d2b9c28ea09ef6a
Mohammad Javad Ranjbar Kalahroodi
claimed_verified
2026-03-18T08:45:37.707Z
659f2bce24e3a2aa7248c5ed
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/JoDZWgJxVXaC1vF2YH-ex.png
Mohammad Ranjbar
MohammadJRanjbar
user
MohammadJRanjbar
69b9ed755d2b9c28ea09ef6b
Mohammad Amini
69b9ed755d2b9c28ea09ef6c
Parmis Bathayan
69b9ed755d2b9c28ea09ef6d
Heshaam Faili
69b9ed755d2b9c28ea09ef6e
Azadeh Shakery
PARSA-Bench: A Comprehensive Persian Audio-Language Model Benchmark
Persian poses unique audio understanding challenges through its classical poetry, traditional music, and pervasive code-switching - none captured by existing benchmarks. We introduce PARSA-Bench (Persian Audio Reasoning and Speech Assessment Benchmark), the first benchmark for evaluating large audio-language models on Persian language and culture, comprising 16 tasks and over 8,000 samples across speech understanding, paralinguistic analysis, and cultural audio understanding. Ten tasks are newly introduced, including poetry meter and style detection, traditional Persian music understanding, and code-switching detection. Text-only baselines consistently outperform audio counterparts, suggesting models may not leverage audio-specific information beyond what transcription alone provides. Culturally-grounded tasks expose a qualitatively distinct failure mode: all models perform near random chance on vazn detection regardless of scale, suggesting prosodic perception remains beyond the reach of current models. The dataset is publicly available at https://huggingface.co/datasets/MohammadJRanjbar/PARSA-Bench
PARSA-Bench presents the first benchmark for evaluating large audio-language models on Persian language and culture, featuring 16 tasks with over 8,000 samples covering speech understanding, paralinguistic analysis, and cultural audio comprehension.
audio-language models Persian language cultural audio understanding speech understanding paralinguistic analysis code-switching detection poetry meter detection traditional Persian music understanding vazn detection