I wrote a article on Parameter-Efficient Fine-Tuning (PEFT), exploring techniques for  efficient fine-tuning in LLMs, their implementations, and variations. 
text
I wrote a article on Parameter-Efficient Fine-Tuning (PEFT), exploring techniques for  efficient fine-tuning in LLMs, their implementations, and variations. 
new_line
new_line
The study is based on the article "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning" and the PEFT library integrated with Hugging Face's Transformers. 
text
The study is based on the article "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning" and the PEFT library integrated with Hugging Face's Transformers. 
new_line
new_line
Article: 
text
Article: 
https://huggingface.co/blog/samuellimabraz/peft-methods
https://huggingface.co/blog/samuellimabraz/peft-methods
link
new_line
Notebook: 
text
Notebook: 
https://colab.research.google.com/drive/1B9RsKLMa8SwTxLsxRT8g9OedK10zfBEP?usp=sharing
https://colab.research.google.com/drive/1B9RsKLMa8SwTxLsxRT8g9OedK10zfBEP?usp=sharing
link
new_line
Collection: 
text
Collection: 
https://huggingface.co/collections/samuellimabraz/service-summary-6793ccfe774073328ea9f8df
samuellimabraz/service-summary-6793ccfe774073328ea9f8df
collection
resource
https://huggingface.co/collections/samuellimabraz/service-summary-6793ccfe774073328ea9f8df
new_line
new_line
Analyzed methods:
text
Analyzed methods:
new_line
- Adapters:  Soft Prompts (Prompt Tuning, Prefix Tuning, P-tuning), IA³.
text
- Adapters:  Soft Prompts (Prompt Tuning, Prefix Tuning, P-tuning), IA³.
new_line
- Reparameterization: LoRA, QLoRA, LoHa, LoKr, X-LoRA, Intrinsic SAID, and variations of initializations (PiSSA, OLoRA, rsLoRA, DoRA).
text
- Reparameterization: LoRA, QLoRA, LoHa, LoKr, X-LoRA, Intrinsic SAID, and variations of initializations (PiSSA, OLoRA, rsLoRA, DoRA).
new_line
- Selective Tuning: BitFit, DiffPruning, FAR, FishMask.
text
- Selective Tuning: BitFit, DiffPruning, FAR, FishMask.
new_line
new_line
I'm starting out in generative AI, I have more experience with computer vision and robotics. Just sharing here 🤗 
text
I'm starting out in generative AI, I have more experience with computer vision and robotics. Just sharing here 🤗 
I wrote a article on Parameter-Efficient Fine-Tuning (PEFT), exploring techniques for  efficient fine-tuning in LLMs, their implementations, and variations. 

The study is based on the article "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning" and the PEFT library integrated with Hugging Face's Transformers. 

Article: https://huggingface.co/blog/samuellimabraz/peft-methods
Notebook: https://colab.research.google.com/drive/1B9RsKLMa8SwTxLsxRT8g9OedK10zfBEP?usp=sharing
Collection: https://huggingface.co/collections/samuellimabraz/service-summary-6793ccfe774073328ea9f8df

Analyzed methods:
- Adapters:  Soft Prompts (Prompt Tuning, Prefix Tuning, P-tuning), IA³.
- Reparameterization: LoRA, QLoRA, LoHa, LoKr, X-LoRA, Intrinsic SAID, and variations of initializations (PiSSA, OLoRA, rsLoRA, DoRA).
- Selective Tuning: BitFit, DiffPruning, FAR, FishMask.

I'm starting out in generative AI, I have more experience with computer vision and robotics. Just sharing here 🤗 
👍
John6666 nicholasKluge LucasBer