An Empirical Recipe for Universal Phone Recognition
Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks, yet robust performance remains elusive. Highly performant English-focused models do not generalize across languages, while multilingual models underutilize pretrained representations. It also remains unclear how data scale, architecture, and training objective contribute to multilingual PR. We present PhoneticXEUS -- trained on large-scale multilingual data and achieving state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER). Through controlled ablations with evaluations across 100+ languages under a unified scheme, we empirically establish our training recipe and quantify the impact of SSL representations, data scale, and loss objectives. In addition, we analyze error patterns across language families, accented speech, and articulatory features. All data and code are released openly.
69d026c62c523d36431fa854
Shikhar Bharadwaj
claimed_verified
2026-04-07T08:39:56.608Z
66ccd2fcdba9f642125e3a55
https://cdn-avatars.huggingface.co/v1/production/uploads/66ccd2fcdba9f642125e3a55/hTKwQQahkbpNuGlaRhKp7.jpeg
Shikhar Bharadwaj
shikhar7ssu
user
shikhar7ssu
69d026c62c523d36431fa855
Chin-Jou Li
claimed_verified
2026-04-08T08:06:36.035Z
643e6952f6bb3c31a26d3650
/avatars/70bfa6a28f71e78446ccac5742bc7be8.svg
Chin-Jou Li
cjli
user
cjli
69d026c62c523d36431fa856
Kwanghee Choi
69d026c62c523d36431fa857
Eunjung Yeo
claimed_verified
2026-04-08T08:06:38.138Z
618b80845dc6e1c37cc46643
/avatars/60cdf870213f41d63d077cc5cfaf75cb.svg
Eunjung Yeo
speech31
user
speech31
69d026c62c523d36431fa858
William Chen
69d026c62c523d36431fa859
Shinji Watanabe
69d026c62c523d36431fa85a
David R. Mortensen
An Empirical Recipe for Universal Phone Recognition
Phone recognition (PR) is a key enabler of multilingual and low-resource speech processing tasks, yet robust performance remains elusive. Highly performant English-focused models do not generalize across languages, while multilingual models underutilize pretrained representations. It also remains unclear how data scale, architecture, and training objective contribute to multilingual PR. We present PhoneticXEUS -- trained on large-scale multilingual data and achieving state-of-the-art performance on both multilingual (17.7% PFER) and accented English speech (10.6% PFER). Through controlled ablations with evaluations across 100+ languages under a unified scheme, we empirically establish our training recipe and quantify the impact of SSL representations, data scale, and loss objectives. In addition, we analyze error patterns across language families, accented speech, and articulatory features. All data and code are released openly.
PhoneticXEUS achieves state-of-the-art performance on multilingual and accented speech recognition through large-scale training and systematic analysis of key factors affecting model performance.
phonetic recognition multilingual speech processing pretrained representations self-supervised learning loss objectives error analysis articulatory features