CARE-Edit: Condition-Aware Routing of Experts for Contextual Image Editing
Unified diffusion editors often rely on a fixed, shared backbone for diverse tasks, suffering from task interference and poor adaptation to heterogeneous demands (e.g., local vs global, semantic vs photometric). In particular, prevalent ControlNet and OmniControl variants combine multiple conditioning signals (e.g., text, mask, reference) via static concatenation or additive adapters which cannot dynamically prioritize or suppress conflicting modalities, thus resulting in artifacts like color bleeding across mask boundaries, identity or style drift, and unpredictable behavior under multi-condition inputs. To address this, we propose Condition-Aware Routing of Experts (CARE-Edit) that aligns model computation with specific editing competencies. At its core, a lightweight latent-attention router assigns encoded diffusion tokens to four specialized experts--Text, Mask, Reference, and Base--based on multi-modal conditions and diffusion timesteps: (i) a Mask Repaint module first refines coarse user-defined masks for precise spatial guidance; (ii) the router applies sparse top-K selection to dynamically allocate computation to the most relevant experts; (iii) a Latent Mixture module subsequently fuses expert outputs, coherently integrating semantic, spatial, and stylistic information to the base images. Experiments validate CARE-Edit's strong performance on contextual editing tasks, including erasure, replacement, text-driven edits, and style transfer. Empirical analysis further reveals task-specific behavior of specialized experts, showcasing the importance of dynamic, condition-aware processing to mitigate multi-condition conflicts.
69af95c3b3ab31dee1281d91
Yucheng Wang
claimed_verified
2026-03-10T09:44:48.704Z
674e8f1ab311aeec3f2c084e
https://cdn-avatars.huggingface.co/v1/production/uploads/674e8f1ab311aeec3f2c084e/n2QQM3e61LoaFA-dUgFfp.jpeg
WANG Yucheng
WANGAndyYucheng
user
WANGAndyYucheng
69af95c3b3ab31dee1281d92
Zedong Wang
claimed_verified
2026-03-10T09:44:46.494Z
6594d390674349122ce6f368
https://cdn-avatars.huggingface.co/v1/production/uploads/6594d390674349122ce6f368/KdWz6lZyGYQpjAgBDeiC1.jpeg
Zedong Wang
JackyWangAI
user
JackyWangAI
69af95c3b3ab31dee1281d93
Yuetong Wu
69af95c3b3ab31dee1281d94
Yue Ma
69af95c3b3ab31dee1281d95
Dan Xu
CARE-Edit: Condition-Aware Routing of Experts for Contextual Image Editing
Unified diffusion editors often rely on a fixed, shared backbone for diverse tasks, suffering from task interference and poor adaptation to heterogeneous demands (e.g., local vs global, semantic vs photometric). In particular, prevalent ControlNet and OmniControl variants combine multiple conditioning signals (e.g., text, mask, reference) via static concatenation or additive adapters which cannot dynamically prioritize or suppress conflicting modalities, thus resulting in artifacts like color bleeding across mask boundaries, identity or style drift, and unpredictable behavior under multi-condition inputs. To address this, we propose Condition-Aware Routing of Experts (CARE-Edit) that aligns model computation with specific editing competencies. At its core, a lightweight latent-attention router assigns encoded diffusion tokens to four specialized experts--Text, Mask, Reference, and Base--based on multi-modal conditions and diffusion timesteps: (i) a Mask Repaint module first refines coarse user-defined masks for precise spatial guidance; (ii) the router applies sparse top-K selection to dynamically allocate computation to the most relevant experts; (iii) a Latent Mixture module subsequently fuses expert outputs, coherently integrating semantic, spatial, and stylistic information to the base images. Experiments validate CARE-Edit's strong performance on contextual editing tasks, including erasure, replacement, text-driven edits, and style transfer. Empirical analysis further reveals task-specific behavior of specialized experts, showcasing the importance of dynamic, condition-aware processing to mitigate multi-condition conflicts.
CARE-Edit introduces a condition-aware routing mechanism that dynamically allocates diffusion model computation to specialized experts for improved contextual image editing tasks.
diffusion models ControlNet OmniControl latent-attention router sparse top-K selection latent mixture module contextual editing text-driven edits style transfer multi-modal conditions diffusion timesteps