Old mixtral model quants may be broken!
text
Old mixtral model quants may be broken!
new_line
new_line
Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models
text
Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models
new_line
new_line
You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue
text
You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue
new_line
new_line
If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)
text
If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)
Old mixtral model quants may be broken!

Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models

You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue

If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)
👍
John6666 djuna JoeySalmons
🔥
John6666
➕
John6666
63cd4b6d1c8a5d1d7d76a778
https://cdn-avatars.huggingface.co/v1/production/uploads/63cd4b6d1c8a5d1d7d76a778/FAnsBh9h5-wG9Vb252ZNt.png
Concedo
concedo
user
66244d1a48e016b5ea1fda20
/avatars/51feb9d4617ba317e878dae64787d500.svg
Bill Winslarkakar
Y-A-R-K
user