Woosh: A Sound Effects Foundation Model
The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.
69cf4129e02fdd68b91b7ad3
Gaëtan Hadjeres
69cf4129e02fdd68b91b7ad4
Marc Ferras
69cf4129e02fdd68b91b7ad5
Khaled Koutini
69cf4129e02fdd68b91b7ad6
Benno Weck
69cf4129e02fdd68b91b7ad7
Alexandre Bittar
69cf4129e02fdd68b91b7ad8
Thomas Hummel
69cf4129e02fdd68b91b7ad9
Zineb Lahrici
69cf4129e02fdd68b91b7ada
Hakim Missoum
69cf4129e02fdd68b91b7adb
Joan Serrà
69cf4129e02fdd68b91b7adc
Yuki Mitsufuji
Woosh: A Sound Effects Foundation Model
The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.
Woosh is a sound effect foundation model featuring audio encoding/decoding, text-audio alignment, and text-to-audio/video-to-audio generation capabilities with distilled versions for efficient deployment.
audio encoder/decoder text-audio alignment text-to-audio video-to-audio distilled models generative models foundation model