MAEB: Massive Audio Embedding Benchmark
We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.
6996e55577d9294348451e19
Adnan El Assadi
claimed_verified
2026-02-19T14:42:12.758Z
6671be9ff022d14aa10df864
/avatars/dd085abefa38c1604dc2ceabf472816d.svg
Adnan El Assadi
AdnanElAssadi
user
AdnanElAssadi
6996e55577d9294348451e1a
Isaac Chung
claimed_verified
2026-02-19T11:04:09.879Z
64cc0e80a257a3212c0c4b24
https://cdn-avatars.huggingface.co/v1/production/uploads/64cc0e80a257a3212c0c4b24/wqs6WZN8-3OQthcnQXgN7.png
Isaac Chung
isaacchung
user
isaacchung
6996e55577d9294348451e1b
Chenghao Xiao
6996e55577d9294348451e1c
Roman Solomatin
claimed_verified
2026-02-19T11:04:05.803Z
61af4544d691b3aadd1f62b6
/avatars/7a4067accdd1005f78c3c4adad3ee0a5.svg
Solomatin Roman
Samoed
user
Samoed
6996e55577d9294348451e1d
Animesh Jha
claimed_verified
2026-02-20T08:37:14.884Z
67aaf15be193f6344f7e4390
/avatars/ec7e7a2d56fe5943b30378062a1fc4fd.svg
Animesh Jha
anime-sh
user
anime-sh
6996e55577d9294348451e1e
Rahul Chand
6996e55577d9294348451e1f
Silky Singh
claimed_verified
2026-02-23T09:46:19.918Z
670d8a232d412a30dfa28ac2
/avatars/6c8997140f5e864750b04d21625f86d9.svg
Silky Singh
silky1708
user
silky1708
6996e55577d9294348451e20
Kaitlyn Wang
claimed_verified
2026-02-19T14:41:41.369Z
67d0ea5d94e08c03254dfb56
/avatars/8e511f5bcebfa869e00bc644e9957162.svg
kaitlyn wang
wangusbeef
user
wangusbeef
6996e55577d9294348451e21
Ali Sartaz Khan
claimed_verified
2026-02-20T08:37:19.294Z
66fd9e8b001816b29e0bc2e4
/avatars/190e6711cef3126c51e15276bb5ed8cd.svg
Ali Sartaz Khan
alisartazkhan
user
alisartazkhan
6996e55577d9294348451e22
Marc Moussa Nasser
6996e55577d9294348451e23
Sufen Fong
6996e55577d9294348451e24
Pengfei He
6996e55577d9294348451e25
Alan Xiao
6996e55577d9294348451e26
Ayush Sunil Munot
claimed_verified
2026-02-19T11:04:07.782Z
6754994f0a4a1144aec6ef57
/avatars/9dc00280582bcb0ace57cb34d25e91a0.svg
Ayush Sunil Munot
AyushM6
user
AyushM6
6996e55577d9294348451e27
Aditya Shrivastava
6996e55577d9294348451e28
Artem Gazizov
claimed_verified
2026-02-20T08:37:21.674Z
6744d6ee334ae6264b8b3d99
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/CC8d048suNVsmlA98Opmn.png
Artem G
arteemg
user
arteemg
6996e55577d9294348451e29
Niklas Muennighoff
6996e55577d9294348451e2a
Kenneth Enevoldsen
claimed_verified
2026-02-20T08:37:17.036Z
5ff5943752c26e9bc240bada
https://cdn-avatars.huggingface.co/v1/production/uploads/5ff5943752c26e9bc240bada/Exyzf3C_gJ2KdsL4K5_cq.png
Kenneth C. Enevoldsen
KennethEnevoldsen
user
KennethEnevoldsen
MAEB: Massive Audio Embedding Benchmark
We introduce the Massive Audio Embedding Benchmark (MAEB), a large-scale benchmark covering 30 tasks across speech, music, environmental sounds, and cross-modal audio-text reasoning in 100+ languages. We evaluate 50+ models and find that no single model dominates across all tasks: contrastive audio-text models excel at environmental sound classification (e.g., ESC50) but score near random on multilingual speech tasks (e.g., SIB-FLEURS), while speech-pretrained models show the opposite pattern. Clustering remains challenging for all models, with even the best-performing model achieving only modest results. We observe that models excelling on acoustic understanding often perform poorly on linguistic tasks, and vice versa. We also show that the performance of audio encoders on MAEB correlates highly with their performance when used in audio large language models. MAEB is derived from MAEB+, a collection of 98 tasks. MAEB is designed to maintain task diversity while reducing evaluation cost, and it integrates into the MTEB ecosystem for unified evaluation across text, image, and audio modalities. We release MAEB and all 98 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.
MAEB is a large-scale audio benchmark evaluating 50+ models across 30 tasks in speech, music, and environmental sounds, revealing diverse model strengths and establishing correlations with audio LLM performance.
audio embedding benchmark audio-text reasoning multilingual speech tasks acoustic understanding linguistic tasks audio encoders audio large language models MTEB ecosystem