Reward Consistency Model
Collection
• 6 items • Updated
How to use naiweizi/dpo-harmless_saferlhf with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("RLHFlow/LLaMA3-SFT")
model = PeftModel.from_pretrained(base_model, "naiweizi/dpo-harmless_saferlhf")