Generalized Discrete Diffusion from Snapshots
We introduce Generalized Discrete Diffusion from Snapshots (GDDS), a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. Our formulation encompasses all existing discrete diffusion approaches, while allowing significantly greater flexibility in the choice of corruption dynamics. The forward noising process relies on uniformization and enables fast arbitrary corruption. For the reverse process, we derive a simple evidence lower bound (ELBO) based on snapshot latents, instead of the entire noising path, that allows efficient training of standard generative modeling architectures with clear probabilistic interpretation. Our experiments on large-vocabulary discrete generation tasks suggest that the proposed framework outperforms existing discrete diffusion methods in terms of training efficiency and generation quality, and beats autoregressive models for the first time at this scale. We provide the code along with a blog post on the project page : https://oussamazekri.fr/gdds{https://oussamazekri.fr/gdds}.
69c23a1fc105f45cb0482080
Oussama Zekri
claimed_verified
2026-03-24T09:20:38.490Z
651e90f4b395a05dff63fe9e
https://cdn-avatars.huggingface.co/v1/production/uploads/651e90f4b395a05dff63fe9e/51g5erCgvJ43szg26EQ_M.png
Oussama Zekri
Xssama
user
Xssama
69c23a1fc105f45cb0482081
Théo Uscidda
69c23a1fc105f45cb0482082
Nicolas Boullé
69c23a1fc105f45cb0482083
Anna Korba
Generalized Discrete Diffusion from Snapshots
We introduce Generalized Discrete Diffusion from Snapshots (GDDS), a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. Our formulation encompasses all existing discrete diffusion approaches, while allowing significantly greater flexibility in the choice of corruption dynamics. The forward noising process relies on uniformization and enables fast arbitrary corruption. For the reverse process, we derive a simple evidence lower bound (ELBO) based on snapshot latents, instead of the entire noising path, that allows efficient training of standard generative modeling architectures with clear probabilistic interpretation. Our experiments on large-vocabulary discrete generation tasks suggest that the proposed framework outperforms existing discrete diffusion methods in terms of training efficiency and generation quality, and beats autoregressive models for the first time at this scale. We provide the code along with a blog post on the project page : https://oussamazekri.fr/gdds{https://oussamazekri.fr/gdds}.
GDDS presents a unified framework for discrete diffusion modeling that supports arbitrary noising processes and demonstrates superior performance in large-vocabulary discrete generation tasks.
discrete diffusion modeling noising processes uniformization evidence lower bound ELBO snapshot latents generative modeling discrete generation tasks