Qwen3-0.6B -- Dataset B SFT (Trial T1)

LoRA supervised fine-tuning of Qwen/Qwen3-0.6B on nvidia/OpenMathReasoning, merged into a standalone full model (16-bit). Part of an IBA NLP Assignment 04 LoRA ablation study (trials T1-T5).

Trial T1: Data efficiency -- 25% of 500 (125 rows)

Hyperparameter Value
LoRA rank 16
Data fraction 25% of 500 rows
Learning rate 0.0002
Epochs 1
Target modules q_proj, k_proj, v_proj, o_proj
Max seq length 2048

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T1")
tok   = AutoTokenizer.from_pretrained("COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T1")
msgs  = [{"role": "user", "content": "What is 2+2?"}]
ids   = tok.apply_chat_template(msgs, return_tensors="pt",
                                add_generation_prompt=True, enable_thinking=True)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0]))
Downloads last month
6
Safetensors
Model size
0.6B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T1

Finetuned
Qwen/Qwen3-0.6B
Adapter
(506)
this model

Dataset used to train COMPUTATION-NOT-FOUND/qwen3-0.6b-datasetB-T1