Accent Vector: Controllable Accent Manipulation for Multilingual TTS Without Accented Data
Accent is an integral part of society, reflecting multiculturalism and shaping how individuals express identity. The majority of English speakers are non-native (L2) speakers, yet current Text-To-Speech (TTS) systems primarily model American-accented English due limited accented data. We propose Accent Vector, a controllable representation that enables accent manipulation in multilingual TTS without requiring accented training data. Accent Vector is derived by fine-tuning a TTS system on native speech of a different language (i.e. non-English) and computing task vectors capturing accent characteristics (i.e. in English). By scaling and interpolating the vector, we achieve fine-grained control over accent strength and generate mixed-accent speech. In addition, it generalizes beyond English, enabling accent control across multiple languages. Objective and human evaluations confirm the effectiveness of Accent Vector for fine-grained and compositional accent control.
69b38e022ad6c5d4cf486b50
Thanathai Lertpetchpun
69b38e022ad6c5d4cf486b51
Thanapat Trachu
69b38e022ad6c5d4cf486b52
Jihwan Lee
69b38e022ad6c5d4cf486b53
Tiantian Feng
69b38e022ad6c5d4cf486b54
Dani Byrd
69b38e022ad6c5d4cf486b55
Shrikanth Narayanan
Accent Vector: Controllable Accent Manipulation for Multilingual TTS Without Accented Data
Accent is an integral part of society, reflecting multiculturalism and shaping how individuals express identity. The majority of English speakers are non-native (L2) speakers, yet current Text-To-Speech (TTS) systems primarily model American-accented English due limited accented data. We propose Accent Vector, a controllable representation that enables accent manipulation in multilingual TTS without requiring accented training data. Accent Vector is derived by fine-tuning a TTS system on native speech of a different language (i.e. non-English) and computing task vectors capturing accent characteristics (i.e. in English). By scaling and interpolating the vector, we achieve fine-grained control over accent strength and generate mixed-accent speech. In addition, it generalizes beyond English, enabling accent control across multiple languages. Objective and human evaluations confirm the effectiveness of Accent Vector for fine-grained and compositional accent control.
Accent Vector enables controllable accent manipulation in multilingual TTS systems through fine-tuning on native speech from different languages and computing task vectors that capture accent characteristics.
Text-To-Speech accent manipulation multilingual TTS accent vector task vectors fine-tuning accent control speech synthesis