13 New types of LoRA
text
13 New types of LoRA
new_line
new_line
LoRA (Low-Rank Adaptation) is a popular lightweight method for fine-tuning AI models. It doesn't update the full model, it adds small trainable components, low-rank matrices, while keeping the original weights frozen. Only these adapters are trained.
text
LoRA (Low-Rank Adaptation) is a popular lightweight method for fine-tuning AI models. It doesn't update the full model, it adds small trainable components, low-rank matrices, while keeping the original weights frozen. Only these adapters are trained.
new_line
new_line
Recently, many interesting new LoRA variations came out, so it’s a great time to take a look at these 13 clever approaches:
text
Recently, many interesting new LoRA variations came out, so it’s a great time to take a look at these 13 clever approaches:
new_line
new_line
1. T-LoRA → 
text
1. T-LoRA → 
T-LoRA: Single Image Diffusion Model Customization Without Overfitting (2507.05964)
https://huggingface.co/papers/2507.05964
2507.05964
paper
resource
https://huggingface.co/papers/2507.05964
new_line
A timestep-dependent LoRA method for adapting diffusion models with a single image. It dynamically adjusts updates and uses orthogonal initialization to reduce overlap, achieving better fidelity–alignment balance than standard LoRA
text
A timestep-dependent LoRA method for adapting diffusion models with a single image. It dynamically adjusts updates and uses orthogonal initialization to reduce overlap, achieving better fidelity–alignment balance than standard LoRA
new_line
new_line
2. SingLoRA → 
text
2. SingLoRA → 
SingLoRA: Low Rank Adaptation Using a Single Matrix (2507.05566)
https://huggingface.co/papers/2507.05566
2507.05566
paper
resource
https://huggingface.co/papers/2507.05566
new_line
Simplifies LoRA by using only one small matrix instead of usual two, and multiplying it by its own transpose (like A × Aᵀ). It uses half the parameters of LoRA and avoids scale mismatch between different matrices
text
Simplifies LoRA by using only one small matrix instead of usual two, and multiplying it by its own transpose (like A × Aᵀ). It uses half the parameters of LoRA and avoids scale mismatch between different matrices
new_line
new_line
3. LiON-LoRA → 
text
3. LiON-LoRA → 
LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and
  Temporal Generation for Video Diffusion (2507.05678)
https://huggingface.co/papers/2507.05678
2507.05678
paper
resource
https://huggingface.co/papers/2507.05678
new_line
Improves control and precision in video diffusion models when training data is limited. It builds on LoRA, adding 3 key principles: linear scalability, orthogonality, and norm consistency. A controllable token and modified self-attention enables smooth adjustment of motion
text
Improves control and precision in video diffusion models when training data is limited. It builds on LoRA, adding 3 key principles: linear scalability, orthogonality, and norm consistency. A controllable token and modified self-attention enables smooth adjustment of motion
new_line
new_line
4. LoRA-Mixer → 
text
4. LoRA-Mixer → 
LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention
  Routing (2507.00029)
https://huggingface.co/papers/2507.00029
2507.00029
paper
resource
https://huggingface.co/papers/2507.00029
new_line
Combines LoRA and mixture-of-experts (MoE) to adapt LLMs for multiple tasks. It dynamically routes task-specific LoRA experts into linear projections of attention modules, supporting both joint training and frozen expert reuse
text
Combines LoRA and mixture-of-experts (MoE) to adapt LLMs for multiple tasks. It dynamically routes task-specific LoRA experts into linear projections of attention modules, supporting both joint training and frozen expert reuse
new_line
new_line
5. QR-LoRA → 
text
5. QR-LoRA → 
QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for
  Customized Generation (2507.04599)
https://huggingface.co/papers/2507.04599
2507.04599
paper
resource
https://huggingface.co/papers/2507.04599
new_line
Separates content and style when combining multiple LoRA adapters. It implements QR decomposition to structure parameter updates, where the orthogonal Q matrix reduces interference between features, and the R matrix captures specific transformations
text
Separates content and style when combining multiple LoRA adapters. It implements QR decomposition to structure parameter updates, where the orthogonal Q matrix reduces interference between features, and the R matrix captures specific transformations
new_line
new_line
Read further in the comments 👇
text
Read further in the comments 👇
new_line
new_line
If you like it, also subscribe to the Turing Post: 
text
If you like it, also subscribe to the Turing Post: 
https://www.turingpost.com/subscribe
https://www.turingpost.com/subscribe
link
13 New types of LoRA

LoRA (Low-Rank Adaptation) is a popular lightweight method for fine-tuning AI models. It doesn't update the full model, it adds small trainable components, low-rank matrices, while keeping the original weights frozen. Only these adapters are trained.

Recently, many interesting new LoRA variations came out, so it’s a great time to take a look at these 13 clever approaches:

1. T-LoRA → https://huggingface.co/papers/2507.05964
A timestep-dependent LoRA method for adapting diffusion models with a single image. It dynamically adjusts updates and uses orthogonal initialization to reduce overlap, achieving better fidelity–alignment balance than standard LoRA

2. SingLoRA → https://huggingface.co/papers/2507.05566
Simplifies LoRA by using only one small matrix instead of usual two, and multiplying it by its own transpose (like A × Aᵀ). It uses half the parameters of LoRA and avoids scale mismatch between different matrices

3. LiON-LoRA → https://huggingface.co/papers/2507.05678
Improves control and precision in video diffusion models when training data is limited. It builds on LoRA, adding 3 key principles: linear scalability, orthogonality, and norm consistency. A controllable token and modified self-attention enables smooth adjustment of motion

4. LoRA-Mixer → https://huggingface.co/papers/2507.00029
Combines LoRA and mixture-of-experts (MoE) to adapt LLMs for multiple tasks. It dynamically routes task-specific LoRA experts into linear projections of attention modules, supporting both joint training and frozen expert reuse

5. QR-LoRA → https://huggingface.co/papers/2507.04599
Separates content and style when combining multiple LoRA adapters. It implements QR decomposition to structure parameter updates, where the orthogonal Q matrix reduces interference between features, and the R matrix captures specific transformations

Read further in the comments 👇

If you like it, also subscribe to the Turing Post: https://www.turingpost.com/subscribe
👍
kaushikvr06 khyaatisrivastava Aanuoluwapo65 minnmamin Kutches Rexschwert Yukkkop
❤️
Rexschwert ibralamnaouar juancopi81 Aanuoluwapo65
🚀
John6666 Rexschwert HV-Khurdula
🔥
Rexschwert
64838b28c235ef76b63e4999
https://cdn-avatars.huggingface.co/v1/production/uploads/64838b28c235ef76b63e4999/ZhQCYoU3vps71Ag7Jezj6.jpeg
Ksenia Se
Kseniase
user