SAO-Instruct: Free-form Audio Editing using Natural Language
  Instructions
Generative models have made significant progress in synthesizing
high-fidelity audio from short textual descriptions. However, editing existing
audio using natural language has remained largely underexplored. Current
approaches either require the complete description of the edited audio or are
constrained to predefined edit instructions that lack flexibility. In this
work, we introduce SAO-Instruct, a model based on Stable Audio Open capable of
editing audio clips using any free-form natural language instruction. To train
our model, we create a dataset of audio editing triplets (input audio, edit
instruction, output audio) using Prompt-to-Prompt, DDPM inversion, and a manual
editing pipeline. Although partially trained on synthetic data, our model
generalizes well to real in-the-wild audio clips and unseen edit instructions.
We demonstrate that SAO-Instruct achieves competitive performance on objective
metrics and outperforms other audio editing approaches in a subjective
listening study. To encourage future research, we release our code and model
weights.
6900bec1646208eac0d1effa
Michael Ungersböck
claimed_verified
2025-10-29T12:49:30.694Z
67002a13c4deb90b3fb8a040
/avatars/ba977617128ef88c75b47d5dd12c5fb0.svg
Michael
ungersboeck
user
ungersboeck
6900bec1646208eac0d1effb
Florian Grötschla
6900bec1646208eac0d1effc
Luca A. Lanzendörfer
claimed_verified
2025-12-16T22:33:22.349Z
62ee461c964e1bff01ba08ad
/avatars/31a9c5b5cb5e6e7a2bc97e355f6ac8af.svg
Luca Lanzendörfer
lucala
user
lucala
6900bec1646208eac0d1effd
June Young Yi
6900bec1646208eac0d1effe
Changho Choi
claimed_verified
2026-03-25T10:52:19.878Z
6295be891fccd01e1612b8ca
https://cdn-avatars.huggingface.co/v1/production/uploads/6295be891fccd01e1612b8ca/7s_TH5EYrUIFvrVCk1Rsy.jpeg
Changho Choi
usingcolor
user
usingcolor
6900bec1646208eac0d1efff
Roger Wattenhofer
SAO-Instruct: Free-form Audio Editing using Natural Language
  Instructions
Generative models have made significant progress in synthesizing
high-fidelity audio from short textual descriptions. However, editing existing
audio using natural language has remained largely underexplored. Current
approaches either require the complete description of the edited audio or are
constrained to predefined edit instructions that lack flexibility. In this
work, we introduce SAO-Instruct, a model based on Stable Audio Open capable of
editing audio clips using any free-form natural language instruction. To train
our model, we create a dataset of audio editing triplets (input audio, edit
instruction, output audio) using Prompt-to-Prompt, DDPM inversion, and a manual
editing pipeline. Although partially trained on synthetic data, our model
generalizes well to real in-the-wild audio clips and unseen edit instructions.
We demonstrate that SAO-Instruct achieves competitive performance on objective
metrics and outperforms other audio editing approaches in a subjective
listening study. To encourage future research, we release our code and model
weights.
SAO-Instruct, a model based on Stable Audio Open, enables flexible audio editing using natural language instructions, outperforming existing methods in both objective and subjective evaluations.
generative models high-fidelity audio natural language audio editing free-form natural language instruction Prompt-to-Prompt DDPM inversion synthetic data real in-the-wild audio clips objective metrics subjective listening study