Introducing UNA-ThePitbull Series
text
Introducing UNA-ThePitbull Series
new_line
new_line
We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
text
We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
new_line
* Replete-AI/code_bagel_hermes-2.5
text
* Replete-AI/code_bagel_hermes-2.5
new_line
* mlabonne/orpo-dpo-mix-40k
text
* mlabonne/orpo-dpo-mix-40k
new_line
* jondurbin/py-dpo-v0.1
text
* jondurbin/py-dpo-v0.1
new_line
Available in the hub 
text
Available in the hub 
https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2
fblgit/UNA-ThePitbull-21.4B-v2
model
resource
https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2
 and you can grab Quant versions sponsored by 
text
 and you can grab Quant versions sponsored by 
@bartowski
mention
bartowski
 at 
text
 at 
https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF
bartowski/UNA-ThePitbull-21.4B-v2-GGUF
model
resource
https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF
 fully compatible with Ollama, llama.cpp, etc.
text
 fully compatible with Ollama, llama.cpp, etc.
new_line
new_line
UNA
text
UNA
new_line
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.
text
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.
new_line
new_line
We trained him under these terms:
text
We trained him under these terms:
new_line
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
text
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
new_line
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
text
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
new_line
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.
text
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.
new_line
new_line
Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!
text
Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!
new_line
Introducing UNA-ThePitbull Series

We are happy to announce the release of our latest model UNA-ThePitbull, the most powerful model below 70B in the industry. In this new generation, inspired on our previous Beagle series we curated a model that balance nicely EQ and IQ. It was trained with some of the latest datasets including:
* Replete-AI/code_bagel_hermes-2.5
* mlabonne/orpo-dpo-mix-40k
* jondurbin/py-dpo-v0.1
Available in the hub https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2 and you can grab Quant versions sponsored by @bartowski at https://huggingface.co/bartowski/UNA-ThePitbull-21.4B-v2-GGUF fully compatible with Ollama, llama.cpp, etc.

UNA
In this case we tried something new by alternating uniformity across layers of both MLP & Attention reducing computational requirements while keep a high performant result.

We trained him under these terms:
* ThePitbull-v1 as base: SFT maxLR 1e-4 minLR 5e-5 for 1 Epoch
* DPO maxLR 1e-4 minLR 5e-5 for 1 Epoch
You can continue the training by merely using 5e-5 maxLR and 0 warmup steps, it should minimize catastrophic forgetting of the model.

Remember if you do so, please include a Pitbull picture on your model and cite :) Have fun!

🔥
bartowski Tonic Yhyu13
🚀
thesven