HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models
Diffusion models achieve state-of-the-art performance but often fail to generate outputs that align with human preferences and intentions, resulting in images with poor aesthetic quality and semantic inconsistencies. Existing alignment methods present a difficult trade-off: fine-tuning approaches suffer from loss of diversity with reward over-optimization, while test-time scaling methods introduce significant computational overhead and tend to under-optimize. To address these limitations, we propose HyperAlign, a novel framework that trains a hypernetwork for efficient and effective test-time alignment. Instead of modifying latent states, HyperAlign dynamically generates low-rank adaptation weights to modulate the diffusion model's generation operators. This allows the denoising trajectory to be adaptively adjusted based on input latents, timesteps and prompts for reward-conditioned alignment. We introduce multiple variants of HyperAlign that differ in how frequently the hypernetwork is applied, balancing between performance and efficiency. Furthermore, we optimize the hypernetwork using a reward score objective regularized with preference data to reduce reward hacking. We evaluate HyperAlign on multiple extended generative paradigms, including Stable Diffusion and FLUX. It significantly outperforms existing fine-tuning and test-time scaling baselines in enhancing semantic consistency and visual appeal.
697a0cdfdf3e800774f139ec
Xin Xie
claimed_verified
2026-03-31T09:32:15.576Z
630cb757cb1d6a23842a4312
/avatars/bf51ef36a00afdb8b311253fd71c9006.svg
Xin Xie
Shelsin
user
Shelsin
697a0cdfdf3e800774f139ed
Jiaxian Guo
697a0cdfdf3e800774f139ee
Dong Gong
claimed_verified
2026-01-28T14:41:37.251Z
6978aa1e2904a5468ef339f6
https://cdn-avatars.huggingface.co/v1/production/uploads/6978aa1e2904a5468ef339f6/cZ7Ae94lOrge_WBSzRSMG.jpeg
Dong Gong
dginf
user
dginf
HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Models
Diffusion models achieve state-of-the-art performance but often fail to generate outputs that align with human preferences and intentions, resulting in images with poor aesthetic quality and semantic inconsistencies. Existing alignment methods present a difficult trade-off: fine-tuning approaches suffer from loss of diversity with reward over-optimization, while test-time scaling methods introduce significant computational overhead and tend to under-optimize. To address these limitations, we propose HyperAlign, a novel framework that trains a hypernetwork for efficient and effective test-time alignment. Instead of modifying latent states, HyperAlign dynamically generates low-rank adaptation weights to modulate the diffusion model's generation operators. This allows the denoising trajectory to be adaptively adjusted based on input latents, timesteps and prompts for reward-conditioned alignment. We introduce multiple variants of HyperAlign that differ in how frequently the hypernetwork is applied, balancing between performance and efficiency. Furthermore, we optimize the hypernetwork using a reward score objective regularized with preference data to reduce reward hacking. We evaluate HyperAlign on multiple extended generative paradigms, including Stable Diffusion and FLUX. It significantly outperforms existing fine-tuning and test-time scaling baselines in enhancing semantic consistency and visual appeal.
HyperAlign enhances diffusion model output quality by using a hypernetwork to dynamically adjust denoising trajectories based on input conditions and rewards, improving semantic consistency and visual appeal.
diffusion models hypernetwork low-rank adaptation denoising trajectory reward-conditioned alignment reward score objective preference data Stable Diffusion FLUX