Can we align code generation models to be good at chat without compromising their base capabilities 🤔?
text
Can we align code generation models to be good at chat without compromising their base capabilities 🤔?
new_line
new_line
This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like 
text
This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like 
https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct
deepseek-ai/deepseek-coder-6.7b-instruct
model
resource
https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct
 and 
text
 and 
https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B
m-a-p/OpenCodeInterpreter-DS-33B
model
resource
https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B
 get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?
text
 get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?
new_line
new_line
It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!
text
It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!
new_line
new_line
The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.
text
The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.
new_line
new_line
Anyway, here's a demo of the model, along with all the code and datasets we used to train it:
text
Anyway, here's a demo of the model, along with all the code and datasets we used to train it:
new_line
new_line
* Demo: 
text
* Demo: 
https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
link
new_line
* Collection: 
text
* Collection: 
https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
collection
resource
https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
new_line
* Recipe: 
text
* Recipe: 
https://github.com/huggingface/alignment-handbook
https://github.com/huggingface/alignment-handbook
link
new_line
new_line
Hope it's useful to others!
text
Hope it's useful to others!
new_line
Can we align code generation models to be good at chat without compromising their base capabilities 🤔?

This was the question the H4 team asked itself when BigCode released StarCoder2 a bit over a week ago. We knew that code models like https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct and https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B get impressive scores on code benchmarks like HumanEval, but they tend to score poorly on chat benchmarks like MT Bench and IFEval. We also knew that the Zephyr recipe we applied to Mistral 7B produced a strong chat model, so we wondered -- could be tweaked to produce a strong coding assistant?

It turns out the answer is yes and I'm happy to share StarChat2, a DPO fine-tune of StarCoder2 15B that scores highly on both HumanEval and MT Bench / IFEval 🌟!

The most interesting lesson for me was that you get better models by blending in more code/math data than chat during the SFT step - in terms of tokens, we found a ratio of 3:1 worked best.

Anyway, here's a demo of the model, along with all the code and datasets we used to train it:

* Demo: https://huggingface.co/spaces/HuggingFaceH4/starchat2-playground
* Collection: https://huggingface.co/collections/HuggingFaceH4/starchat2-15b-65f068417b330fafad751fce
* Recipe: https://github.com/huggingface/alignment-handbook

Hope it's useful to others!

❤️
lvwerra kashif osanseviero smangrul jordangong Chunte sasikiran MexIvanov vicgalle medmac01
👀
victor
🔥
medmac01
60596ed1d4b07db522410800
/avatars/7769fb2bdb05116cd3ea8a112f0beeeb.svg
Sasi Kiran
sasikiran
user
5f0c746619cb630495b814fd
https://cdn-avatars.huggingface.co/v1/production/uploads/1594651707950-noauth.jpeg
Lewis Tunstall
lewtun
user