ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA
Existing video personalization methods preserve visual likeness but treat video and audio separately. Without access to the visual scene, audio models cannot synchronize sounds with on-screen actions; and because classical voice-cloning models condition only on a reference recording, a text prompt cannot redirect speaking style or acoustic environment. We propose ID-LoRA (Identity-Driven In-Context LoRA), which jointly generates a subject's appearance and voice in a single model, letting a text prompt, a reference image, and a short audio clip govern both modalities together. ID-LoRA adapts the LTX-2 joint audio-video diffusion backbone via parameter-efficient In-Context LoRA and, to our knowledge, is the first method to personalize visual appearance and voice in a single generative pass. Two challenges arise. Reference and generation tokens share the same positional-encoding space, making them hard to distinguish; we address this with negative temporal positions, placing reference tokens in a disjoint RoPE region while preserving their internal temporal structure. Speaker characteristics also tend to be diluted during denoising; we introduce identity guidance, a classifier-free guidance variant that amplifies speaker-specific features by contrasting predictions with and without the reference signal. In human preference studies, ID-LoRA is preferred over Kling 2.6 Pro by 73% of annotators for voice similarity and 65% for speaking style. On cross-environment settings, speaker similarity improves by 24% over Kling, with the gap widening as conditions diverge. A preliminary user study further suggests that joint generation provides a useful inductive bias for physically grounded sound synthesis. ID-LoRA achieves these results with only ~3K training pairs on a single GPU. Code, models, and data will be released.
69b2f3c92ad6c5d4cf4868c0
Aviad Dahan
claimed_verified
2026-03-13T08:13:34.006Z
62aef7d9aa0a36b80938665f
https://cdn-avatars.huggingface.co/v1/production/uploads/62aef7d9aa0a36b80938665f/xZ0JrG6u6YKRa4g62MViY.jpeg
Aviad Dahan
AviadDahan
user
AviadDahan
69b2f3c92ad6c5d4cf4868c1
Moran Yanuka
claimed_verified
2026-03-13T08:13:37.052Z
64ca8d8e0d249f4ad27db0e8
https://cdn-avatars.huggingface.co/v1/production/uploads/64ca8d8e0d249f4ad27db0e8/fRgfF-HDVmBRUzai4gLeB.png
Moran Yanuka
moranyanuka
user
moranyanuka
69b2f3c92ad6c5d4cf4868c2
Noa Kraicer
claimed_verified
2026-03-18T10:17:01.302Z
6501b416ca3877c454619476
https://cdn-avatars.huggingface.co/v1/production/uploads/6501b416ca3877c454619476/ecd9XVb6R2wgTfCcN6zEG.png
noa kraicer
noakraicer
user
noakraicer
69b2f3c92ad6c5d4cf4868c3
Lior Wolf
69b2f3c92ad6c5d4cf4868c4
Raja Giryes
ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA
Existing video personalization methods preserve visual likeness but treat video and audio separately. Without access to the visual scene, audio models cannot synchronize sounds with on-screen actions; and because classical voice-cloning models condition only on a reference recording, a text prompt cannot redirect speaking style or acoustic environment. We propose ID-LoRA (Identity-Driven In-Context LoRA), which jointly generates a subject's appearance and voice in a single model, letting a text prompt, a reference image, and a short audio clip govern both modalities together. ID-LoRA adapts the LTX-2 joint audio-video diffusion backbone via parameter-efficient In-Context LoRA and, to our knowledge, is the first method to personalize visual appearance and voice in a single generative pass. Two challenges arise. Reference and generation tokens share the same positional-encoding space, making them hard to distinguish; we address this with negative temporal positions, placing reference tokens in a disjoint RoPE region while preserving their internal temporal structure. Speaker characteristics also tend to be diluted during denoising; we introduce identity guidance, a classifier-free guidance variant that amplifies speaker-specific features by contrasting predictions with and without the reference signal. In human preference studies, ID-LoRA is preferred over Kling 2.6 Pro by 73% of annotators for voice similarity and 65% for speaking style. On cross-environment settings, speaker similarity improves by 24% over Kling, with the gap widening as conditions diverge. A preliminary user study further suggests that joint generation provides a useful inductive bias for physically grounded sound synthesis. ID-LoRA achieves these results with only ~3K training pairs on a single GPU. Code, models, and data will be released.
ID-LoRA enables joint generation of visual appearance and voice using a single model, improving personalization through identity-driven in-context LoRA adaptation and identity guidance techniques.
ID-LoRA In-Context LoRA audio-video diffusion classifier-free guidance RoPE denoising parameter-efficient fine-tuning joint generation identity guidance visual appearance voice cloning