I propose "merge densification", a style of merger which attempts to transfer the benefits of a denser model to a base model. The model weight in this case is 0.02, which is atypically small for mergers, but high compared to the learning rate used during training. In this case, the expectation is more creative text-generation. More details below:
text
I propose "merge densification", a style of merger which attempts to transfer the benefits of a denser model to a base model. The model weight in this case is 0.02, which is atypically small for mergers, but high compared to the learning rate used during training. In this case, the expectation is more creative text-generation. More details below:
new_line
https://huggingface.co/grimjim/kunoichi-lemon-royale-v3-32K-7B
grimjim/kunoichi-lemon-royale-v3-32K-7B
model
resource
https://huggingface.co/grimjim/kunoichi-lemon-royale-v3-32K-7B
I propose "merge densification", a style of merger which attempts to transfer the benefits of a denser model to a base model. The model weight in this case is 0.02, which is atypically small for mergers, but high compared to the learning rate used during training. In this case, the expectation is more creative text-generation. More details below:
https://huggingface.co/grimjim/kunoichi-lemon-royale-v3-32K-7B
👀
osanseviero louisbrulenaudet victor SerialKicked
👍
SerialKicked