Thrilled to share our latest work: Voila - a family of fully opensourced voice models for real-time autonomous convos and role-play, some of our major contributions include 🧵:
text
Thrilled to share our latest work: Voila - a family of fully opensourced voice models for real-time autonomous convos and role-play, some of our major contributions include 🧵:
new_line
1) An End-to-End Full-Duplex Arch: that directly processes & handles simultaneous audio token streams from user to model and vice versa.
text
1) An End-to-End Full-Duplex Arch: that directly processes & handles simultaneous audio token streams from user to model and vice versa.
new_line
2) Voila-Tokenizer: A 100K-hour trained tokenizer with interleaved alignment (audio & text) that distills semantic/acoustic tokens via RVQ.
text
2) Voila-Tokenizer: A 100K-hour trained tokenizer with interleaved alignment (audio & text) that distills semantic/acoustic tokens via RVQ.
new_line
3) Text-Audio Interleaved Alignment: We leveraged a fine-grained alignment of text and audio tokens that allows synchronization and expressiveness for tasks like ASR (WER 2.7%) and TTS (WER 2.8%).
text
3) Text-Audio Interleaved Alignment: We leveraged a fine-grained alignment of text and audio tokens that allows synchronization and expressiveness for tasks like ASR (WER 2.7%) and TTS (WER 2.8%).
new_line
4) Voice Customization: Supports 1M+ pre-built voices and 1 shot voice clone from 10s audio clips using Wespeaker embeddings.
text
4) Voice Customization: Supports 1M+ pre-built voices and 1 shot voice clone from 10s audio clips using Wespeaker embeddings.
new_line
new_line
Models: 
text
Models: 
https://huggingface.co/collections/maitrix-org/voila-67e0d96962c19f221fc73fa5
maitrix-org/voila-67e0d96962c19f221fc73fa5
collection
resource
https://huggingface.co/collections/maitrix-org/voila-67e0d96962c19f221fc73fa5
new_line
Code: 
text
Code: 
https://github.com/maitrix-org/Voila
https://github.com/maitrix-org/Voila
link
new_line
Demo: 
text
Demo: 
https://hf.co/spaces/maitrix-org/Voila-demo
maitrix-org/Voila-demo
space
resource
https://hf.co/spaces/maitrix-org/Voila-demo
new_line
Project page: 
text
Project page: 
https://hf.co/spaces/maitrix-org/Voila-demo
maitrix-org/Voila-demo
space
resource
https://hf.co/spaces/maitrix-org/Voila-demo
text
new_line
Thrilled to share our latest work: Voila - a family of fully opensourced voice models for real-time autonomous convos and role-play, some of our major contributions include 🧵:
1) An End-to-End Full-Duplex Arch: that directly processes & handles simultaneous audio token streams from user to model and vice versa.
2) Voila-Tokenizer: A 100K-hour trained tokenizer with interleaved alignment (audio & text) that distills semantic/acoustic tokens via RVQ.
3) Text-Audio Interleaved Alignment: We leveraged a fine-grained alignment of text and audio tokens that allows synchronization and expressiveness for tasks like ASR (WER 2.7%) and TTS (WER 2.8%).
4) Voice Customization: Supports 1M+ pre-built voices and 1 shot voice clone from 10s audio clips using Wespeaker embeddings.

Models: https://huggingface.co/collections/maitrix-org/voila-67e0d96962c19f221fc73fa5
Code: https://github.com/maitrix-org/Voila
Demo: https://hf.co/spaces/maitrix-org/Voila-demo
Project page: https://hf.co/spaces/maitrix-org/Voila-demo 

🔥
John6666
6438a9027de34e8ea7e4b257
https://cdn-avatars.huggingface.co/v1/production/uploads/6438a9027de34e8ea7e4b257/vib8QSd1AWMr_bR9ig_xJ.jpeg
Jaward Sesay
Jaward
user
65bb837dbfb878f46c77de4c
https://cdn-avatars.huggingface.co/v1/production/uploads/65bb837dbfb878f46c77de4c/23gZ_lBEwyoqjexFy9QLD.jpeg
Prithiv Sakthi
prithivMLmods
user