no ai slop posted here today i just feel like posting what i did for today
text
no ai slop posted here today i just feel like posting what i did for today
new_line
new_line
wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.
text
wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.
new_line
new_line
code @ 
text
code @ 
https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
link
no ai slop posted here today i just feel like posting what i did for today

wrote a little framework for turning multiple dense models (llama based) into Sparse MoEs.. i found it fun, spent the whole day and a half on it.

code @ https://gist.github.com/cappuch/6a454ec8d2d349a27f9fd84f6ac90554
👍
John6666 iproskurina azettl