pidgin_finetuned_oss_model

Overview

This repository contains a LoRA adapter that fine-tunes unsloth/gpt-oss-20b-unsloth-bnb-4bit — a 4-bit quantized build of OpenAI's gpt-oss-20b open-weight model — for Nigerian Pidgin English (Naija Pidgin, ISO 639-3: pcm).

Nigerian Pidgin is spoken by an estimated tens of millions of people across Nigeria and the wider West African diaspora, yet it remains a low-resource language for NLP: large language models are trained overwhelmingly on standard English and other high-resource languages, so they often struggle to generate or understand fluent, natural Pidgin. This project adapts an open-weight LLM towards Pidgin text generation, aiming to make LLM-based tools more accessible and natural for Pidgin-speaking users.

Training Details

Detail	Value
Base model	unsloth/gpt-oss-20b-unsloth-bnb-4bit (gpt-oss-20B, 4-bit quantized)
Fine-tuning method	LoRA (parameter-efficient fine-tuning via PEFT)
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Task type	Causal LM
Training framework	Unsloth (2x faster fine-tuning), Hugging Face transformers, trl, peft
License	Apache 2.0

According to the original author's notes, the adapter was trained on a dataset of roughly 10,000 examples. Detailed training logs (exact step/epoch counts, loss curves) were not published alongside the adapter and are therefore not restated here.

This repo hosts only the LoRA adapter weights (adapter_model.safetensors + adapter_config.json) — the base gpt-oss-20b weights must be loaded separately, as shown below.

Intended Use

• Generating conversational text in Nigerian Pidgin English
• Assisting with Pidgin-English translation and localization drafts
• Chatbot / assistant prototypes aimed at Pidgin-speaking users
• Research into low-resource African language adaptation of open-weight LLMs

This model is a research/community fine-tune and has not been evaluated for production or safety-critical use.

How to Use

Option A — with Unsloth (recommended, matches the training setup):

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Ephraimmm/pidgin_finetuned_oss_model",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

prompt = "Wetin dey happen for Naija today?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Option B — with transformers + peft directly:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/gpt-oss-20b-unsloth-bnb-4bit"
adapter_id = "Ephraimmm/pidgin_finetuned_oss_model"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Wetin dey happen for Naija today?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• Low-resource target language: Nigerian Pidgin has limited high-quality training data available, and Pidgin usage varies by region and speaker, so output fluency and correctness may be inconsistent.
• No formal benchmark evaluation: no quantitative accuracy, fluency, or benchmark scores are published for this adapter — outputs should be manually reviewed before use in any downstream application.
• Small adapter, limited training data: the LoRA adapter uses a modest rank (r=8) and was trained on an author-reported ~10k-example dataset, which may limit generalization to topics or phrasing not covered during fine-tuning.
• Inherits base model limitations: as a fine-tune of gpt-oss-20b, this model inherits the general limitations, potential biases, and safety considerations of the base model.
• Requires the base model: this repository only contains LoRA adapter weights; it must be combined with the unsloth/gpt-oss-20b-unsloth-bnb-4bit base model to run.

Author

Developed by Ephraimmm.