Supercool Weekend Read🤖
text
Supercool Weekend Read🤖
new_line
Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.
text
Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.
new_line
new_line
Details on Techniques (Simplified):
text
Details on Techniques (Simplified):
new_line
They started off with a large pre-trained language model (15B params), then:
text
They started off with a large pre-trained language model (15B params), then:
new_line
new_line
1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.
text
1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.
new_line
new_line
2. Pruned (remove) less important parts of the model to reduce its size.
text
2. Pruned (remove) less important parts of the model to reduce its size.
new_line
new_line
3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.
text
3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.
new_line
new_line
4. Used a lightweight neural architecture search to find the best configuration for the pruned model.
text
4. Used a lightweight neural architecture search to find the best configuration for the pruned model.
new_line
new_line
5. Repeated this process iteratively to create even smaller models.
text
5. Repeated this process iteratively to create even smaller models.
new_line
new_line
Cool, giving it a try this weekend 😎
text
Cool, giving it a try this weekend 😎
new_line
Code: 
text
Code: 
https://github.com/NVlabs/Minitron
https://github.com/NVlabs/Minitron
link
new_line
Paper: 
text
Paper: 
https://arxiv.org/abs/2407.14679
https://arxiv.org/abs/2407.14679
link
new_line
Demo: 
text
Demo: 
https://huggingface.co/spaces/nvidia/minitron
nvidia/minitron
space
resource
https://huggingface.co/spaces/nvidia/minitron
Supercool Weekend Read🤖
Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.

Details on Techniques (Simplified):
They started off with a large pre-trained language model (15B params), then:

1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.

2. Pruned (remove) less important parts of the model to reduce its size.

3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.

4. Used a lightweight neural architecture search to find the best configuration for the pruned model.

5. Repeated this process iteratively to create even smaller models.

Cool, giving it a try this weekend 😎
Code: https://github.com/NVlabs/Minitron
Paper: https://arxiv.org/abs/2407.14679
Demo: https://huggingface.co/spaces/nvidia/minitron
👍
Edouard360 ajibawa-2023 adamelliotfields davidle7 KingNish
👀
John6666 Dihelson tuanlda78202
❤️
Dihelson