SimulU: Training-free Policy for Long-form Simultaneous Speech-to-Speech Translation
Simultaneous speech-to-speech translation (SimulS2S) is essential for real-time multilingual communication, with increasing integration into meeting and streaming platforms. Despite this, SimulS2S remains underexplored in research, where current solutions often rely on resource-intensive training procedures and operate on short-form, pre-segmented utterances, failing to generalize to continuous speech. To bridge this gap, we propose SimulU, the first training-free policy for long-form SimulS2S. SimulU adopts history management and speech output selection strategies that exploit cross-attention in pre-trained end-to-end models to regulate both input history and output generation. Evaluations on MuST-C across 8 languages show that SimulU achieves a better or comparable quality-latency trade-off against strong cascaded models. By eliminating the need for ad-hoc training, SimulU offers a promising path to end-to-end SimulS2S in realistic, long-form scenarios.
69bbbd66081b9ad49af56e80
Amirbek Djanibekov
claimed_verified
2026-03-23T10:56:12.940Z
63791fc77df2fefdcaf17c65
https://cdn-avatars.huggingface.co/v1/production/uploads/63791fc77df2fefdcaf17c65/kdq8k_xc28VhFsSMPHbV0.jpeg
Amir Djanibekov
amupd
user
amupd
69bbbd66081b9ad49af56e81
Luisa Bentivogli
69bbbd66081b9ad49af56e82
Matteo Negri
69bbbd66081b9ad49af56e83
Sara Papi
claimed_verified
2026-03-19T12:56:26.797Z
66309b3833ccd9e68c5d5171
https://cdn-avatars.huggingface.co/v1/production/uploads/66309b3833ccd9e68c5d5171/UGt7pZazJHhtwjg0iiyBu.jpeg
Sara Papi
spapi
user
spapi
SimulU: Training-free Policy for Long-form Simultaneous Speech-to-Speech Translation
Simultaneous speech-to-speech translation (SimulS2S) is essential for real-time multilingual communication, with increasing integration into meeting and streaming platforms. Despite this, SimulS2S remains underexplored in research, where current solutions often rely on resource-intensive training procedures and operate on short-form, pre-segmented utterances, failing to generalize to continuous speech. To bridge this gap, we propose SimulU, the first training-free policy for long-form SimulS2S. SimulU adopts history management and speech output selection strategies that exploit cross-attention in pre-trained end-to-end models to regulate both input history and output generation. Evaluations on MuST-C across 8 languages show that SimulU achieves a better or comparable quality-latency trade-off against strong cascaded models. By eliminating the need for ad-hoc training, SimulU offers a promising path to end-to-end SimulS2S in realistic, long-form scenarios.
SimulU presents a training-free approach for long-form simultaneous speech-to-speech translation that uses history management and cross-attention mechanisms to regulate input and output generation effectively.
simultaneous speech-to-speech translation cross-attention end-to-end models history management speech output selection MuST-C cascaded models