Gwimi-4-12B-IT-BF16

Gwimi-4-12B-IT-BF16 is a merged BF16 release of a Gemma 4 12B instruction model that underwent two consecutive post-training stages:

1. Supervised Fine-Tuning (SFT) on a 20,000-example curated reasoning mixture.
2. Group Sequence Policy Optimization (GSPO) on 12,000 frozen reinforcement-learning prompts with programmatic correctness, formatting, and anomaly rewards.

The final release contains the original BF16 base weights with the cumulative SFT + GSPO LoRA updates merged into them. No separate adapter is required for inference.

Model repository: trjxter/Gwimi-4-12B-IT-BF16

---

Model summary

Field	Value
Base model	unsloth/gemma-4-12b-it
Base revision used for the final merge	0f03bc8c5a5d620f219daea1ef93e05775ae9b74
Approximate parameter count	12B
Final precision	BF16
Post-training stages	SFT followed by GSPO
Adapter method during training	rank-stabilized LoRA
LoRA rank	128
LoRA alpha	256
rsLoRA	enabled
LoRA target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable adapter parameters	524,550,144
Final GSPO checkpoint	global step 2250
Final artifact	merged standalone BF16 model

The GSPO phase continued training the same cumulative adapter produced by SFT. This was not a second independent adapter stacked on top of the SFT adapter.

The final merge can be summarized as:

Exact Gemma 4 12B BF16 base
        +
Cumulative SFT + GSPO LoRA update
        =
Standalone Gwimi-4-12B-IT-BF16 model

---

Training pipeline

unsloth/gemma-4-12b-it
        ↓
20,000-row reasoning dataset
        ↓
Supervised Fine-Tuning
        ↓
SFT adapter
        ↓
12,000-prompt frozen RL dataset
        ↓
GSPO reinforcement learning
        ↓
Checkpoint 2250
        ↓
Merge into exact original BF16 base
        ↓
Gwimi-4-12B-IT-BF16

---

Stage 1: Supervised Fine-Tuning

SFT objective

The SFT stage trained the model to imitate high-quality reasoning responses across mathematics, science, coding, debugging, technical reasoning, general reasoning, and multilingual STEM material.

The model was trained for one epoch using a large rank-128 rsLoRA adapter while the base model was loaded in 8-bit precision.

SFT dataset composition

The final SFT corpus contained exactly 20,000 examples.

Curated sources

Dataset	Rows
trjxter/Kimi-K2.6-Technical-Reasoning-AddOn-3300x	3,301
trjxter/Kimi-K2.6-Reasoning-3300x-WandB	3,303
trjxter/Gemma-4-31B-Reasoning-1000x	995
Jackrong/Claude-opus-4.7-TraceInversion-5000x	4,761
Curated subtotal	12,360

Kimi K2.5 top-up source

The remaining 7,640 examples were selected from:

Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned

Category	Rows
General-Distillation	4,000
General-Math	1,500
PHD-Science	1,500
MultilingualSTEM	640
Top-up subtotal	7,640

Final SFT dataset arithmetic

12,360 curated rows
+ 7,640 Kimi K2.5 top-up rows
= 20,000 total rows

The dataset-building process used normalized-prompt deduplication. Curated rows were given priority over top-up rows when a duplicate was found.

The final top-up selection had zero normalized-prompt overlap with the curated set.

SFT train/evaluation split

Split	Rows
Training	18,000
Held-out evaluation	2,000
Total	20,000

SFT hyperparameters

Parameter	Value
Maximum sequence length	32,768
Base loading precision	8-bit
LoRA rank	128
LoRA alpha	256
rsLoRA	True
LoRA dropout	0
Bias	none
Microbatch size	2
Gradient accumulation	8
Effective optimizer batch	16
Epochs	1
Optimizer steps	1,125
Learning rate	2e-5
Warmup ratio	0.03
Scheduler	cosine
Optimizer	adamw_8bit
Maximum gradient norm	1.0

SFT evaluation design

The 2,000 held-out examples were divided into:

• 100 fixed anchor examples
• 1,900 examples in a rotating evaluation pool

During training, evaluation was scheduled every 250 optimizer steps using:

100 fixed anchor examples
+ 100 rotating examples
= 200 examples per scheduled evaluation

The fixed anchor set made checkpoint-to-checkpoint comparisons more consistent. The rotating slices increased coverage without requiring a full 2,000-example evaluation every 250 steps.

A final evaluation was then run over all 2,000 held-out examples.

Recorded SFT evaluation results

Checkpoint-1000 retrospective evaluation

Evaluation slice	Loss
Fixed anchor, 100 rows	0.636239767
Rotation 1, 100 rows	0.699529946
Rotation 2, 100 rows	0.659535289
Rotation 3, 100 rows	0.768394113
Rotation 4, 100 rows	0.665917516
Rotation 5, 100 rows	0.696190119
Mean rotating loss	0.697913396
Combined 600-row mean	0.687634458

These checkpoint-1000 results should not be confused with the final model evaluation.

Final full held-out evaluation

Metric	Result
Held-out rows	2,000
Final evaluation loss	0.6940310597419739
Final perplexity	2.001768540

These are teacher-forced SFT evaluation metrics. They measure prediction quality on held-out target responses and are not equivalent to free-generation benchmark accuracy.

---

Stage 2: GSPO reinforcement learning

What GSPO means in this run

GSPO stands for Group Sequence Policy Optimization.

For each prompt, the policy generated multiple candidate completions. Programmatic reward functions scored those completions, and the trainer compared their reward values within the prompt group.

The defining GSPO behavior in this run was:

importance_sampling_level = "sequence"

This uses one importance ratio per generated response sequence rather than a separate ratio for every token.

The implementation used TRL's GRPOTrainer and GRPOConfig, but sequence-level importance sampling made the optimization GSPO-style.

Frozen GSPO dataset suite

The RL dataset was frozen before training and stored as reproducible artifacts.

Training split

Source	Rows
dapo_math_en	8,400
openmath_reasoning_mini	1,800
sciq	1,800
Total GSPO training prompts	12,000

Anchor evaluation split

Source	Rows
dapo_math_en	100
openmath_reasoning_mini	100
sciq	100
Anchor total	300

Held-out evaluation split

Source	Rows
dapo_math_en	500
openmath_reasoning_mini	500
sciq	897
Held-out total	1,897

Protected SciQ test split

Split	Rows
Protected SciQ test	991

The frozen dataset suite had zero cross-split prompt overlap.

The GSPO training worker used only the 12,000-row training split. The anchor, held-out, and protected SciQ splits were reserved for evaluation and contamination control.

Reward functions

The training run used three frozen reward functions:

correctness_reward_func
format_reward_func
anomaly_reward_func

They were loaded from the frozen reward module:

gemma_native_rewards_v2_mathverify.py

Correctness reward

The correctness reward evaluated whether the generated answer matched the expected solution criteria. Mathematical answer comparison used a parser-oriented verification approach rather than relying only on literal string equality.

Format reward

The format reward encouraged the response structure expected by the training pipeline, including a valid final-answer structure and properly formed output.

Anomaly reward

The anomaly reward was designed to detect suspicious, malformed, repetitive, degenerate, or parser-exploiting output patterns.

The reward functions were frozen before the official run so that the optimization target could not silently change during training.

GSPO hyperparameters

Parameter	Value
Training prompts	12,000
Model loading precision	BF16
Learning rate	2e-6
Warmup steps	100
Importance-sampling level	sequence
Loss type	grpo
Reward scaling	group
KL coefficient beta	0.0
Lower clipping epsilon	3e-4
Upper clipping epsilon	4e-4
Generations per prompt	8
Unique prompts per rollout	3
Total completions per rollout	24
Per-device training batch	2
Gradient accumulation	4
Effective optimizer batch	8
Steps per generation	12
Optimizer updates per rollout	3
Maximum completion length	1,280
Maximum runtime sequence length	4,096
Temperature	1.15
Top-p	0.95
Repetition penalty	1.05
Mask truncated completions	True
Maximum gradient norm	1.0
Scheduler	cosine
Checkpoint cadence	every 10 optimizer steps
Final checkpoint	2250

The KL penalty was intentionally disabled. A zero KL value in this configuration does not mean that the policy was identical to a reference model; it means that KL regularization was not part of the objective.

GSPO stopping point

The official run stopped at:

global_step = 2250

The run had originally resumed from checkpoint 450, so the successful continuation added approximately 1,800 optimizer updates.

By the stopping point, the run had processed approximately 15.55 million tokens according to the training telemetry.

The decision to stop at 2250 was based on:

• stable entropy rather than continued collapse;
• controlled completion lengths;
• a learned and saturated format reward;
• active but non-saturated sequence clipping;
• continued reward variation;
• complete resumable checkpoints;
• and diminishing expected benefit relative to additional compute cost.

Near-terminal GSPO telemetry

The values below are a near-terminal W&B snapshot around global step 2250. They describe the final observed training batch or local window, not an external benchmark score.

Metric	Near-terminal value	Interpretation
Combined reward	0.2667	Mean aggregate reward for the observed batch
Correctness reward	0.1667	Correctness component for the observed batch
Format reward	0.1000	Format component was effectively saturated
Reward standard deviation	0.3086	The batch retained meaningful reward variation
Fraction of zero-variance reward groups	0.6667	Two of three prompt groups had equal rewards across their eight samples
Entropy	0.0893	Low but stable near the stopping point
Sequence clip ratio, region mean	0.125	Sequence-level clipping was active but not saturated
Mean completion length	762.875 tokens	Completions remained substantial and below the 1,280-token cap
Completion clipped ratio	0.25	Six of 24 completions reached the truncation condition in the observed rollout
Gradient norm	8.3416	Pre-clipping gradient magnitude; updates were constrained by max_grad_norm=1.0

How to interpret these metrics

Reward: Higher is generally better only when the reward implementation correctly reflects the desired behavior. Reward is not equivalent to standardized benchmark accuracy.

Reward standard deviation: Some variation is necessary because GSPO learns from differences among completions in the same prompt group.

Fraction of zero-variance groups: A high value means some prompt groups produced no relative reward signal. The metric was noisy because each rollout contained only three unique prompts, so possible values naturally moved in increments of one third.

Entropy: Entropy is not simply “higher is better.” In this run it declined and then stabilized around 0.09 rather than continuing toward zero. That was treated as evidence against an obvious late-stage policy collapse.

Sequence clip ratio: Some clipping is expected in GSPO. The near-terminal value showed that the trust region was active without clipping the entire rollout.

Completion clipped ratio: This is different from policy-ratio clipping. It measures completions that reached the maximum-length condition. Because truncated completions were masked from policy loss, a high value would reduce training efficiency.

Gradient norm: The logged value was above 1.0 before clipping. The configured maximum gradient norm constrained the actual optimizer update.

Important GSPO evaluation note

The online GSPO worker used:

eval_strategy = "no"

The frozen anchor, held-out, and protected SciQ splits were preserved, but they were not used as periodic online evaluations during the expensive generation loop.

Therefore:

• the SFT loss and perplexity results above are official recorded evaluation results;
• the GSPO values above are training telemetry;
• no claim is made here that the final model has already beaten the base model or SFT-only checkpoint on a standardized external benchmark;
• a proper comparison should evaluate the base, SFT-only checkpoint, and final SFT+GSPO model under the same prompts, decoding settings, and scoring implementation.

This distinction is intentional. Reward curves are useful for diagnosing RL training, but they are not substitutes for independent benchmark evaluation.

---

Validation and artifact integrity

Checkpoint validation

The final checkpoint contained:

adapter_config.json
adapter_model.safetensors
trainer_state.json
optimizer.pt
scheduler.pt
rng_state.pth
training_args.bin
tokenizer.json
tokenizer_config.json
processor_config.json
chat_template.jinja

trainer_state.json reported:

global_step = 2250

The final adapter had:

LoRA rank:       128
LoRA alpha:      256
rsLoRA:          true
Adapter size:    1.954 GiB
Trainable params: 524,550,144

Exact-base merge validation

The final adapter was merged into the exact untouched BF16 base revision:

0f03bc8c5a5d620f219daea1ef93e05775ae9b74

The original base and merged model had the same byte size because the merge changed existing BF16 tensor values rather than adding a second set of parameters:

Base size:   23,919,549,408 bytes
Merged size: 23,919,549,408 bytes

The SHA-256 hashes were different:

Base SHA-256:
5a84cb313260ac447237b890387116dfa8682e49a6b44bc585ae8353abbff18d

Merged SHA-256:
1e024792bf994c200fc7757621d202eb2bb2ba11593afcf6a6a98ab6bb9c4845

This verifies that the uploaded BF16 model contains modified cumulative SFT + GSPO weights and is not an unchanged copy of the original base model.

---

Intended uses

This model is intended for experimentation with:

• mathematical and scientific reasoning;
• technical question answering;
• coding and debugging assistance;
• long-form structured reasoning;
• instruction following;
• local inference research;
• comparison of SFT-only and SFT+RL post-training pipelines.

The model is best treated as a research and development release rather than a guaranteed production system.

---

Limitations

1. No independent final benchmark suite is reported yet.
   The README separates SFT evaluation loss, GSPO training telemetry, and future independent evaluation.

2. Reward optimization can inherit reward-function blind spots.
   Correctness, formatting, and anomaly rewards cannot measure every aspect of answer quality.

3. The GSPO stage focused heavily on verifiable mathematics, science, and structured reasoning prompts.
   Performance may vary outside those distributions.

4. Long-context behavior was not independently benchmarked after GSPO.
   SFT used a maximum sequence length of 32,768, while GSPO used a 4,096-token runtime window with a 1,280-token completion cap.

5. Text behavior was the focus of this post-training project.
   Any other capabilities inherited from the base model were not independently evaluated here.

6. The model can still hallucinate, make calculation errors, produce unsafe advice, or follow incorrect premises.
   Outputs should be verified for high-stakes use.

7. Training reward should not be interpreted as benchmark accuracy.
   The terminal reward values describe the model under the training reward system and rollout distribution.

---

Loading the BF16 model

This model was exported through Unsloth's Gemma 4 unified implementation.

At export time, the working stack included:

Unsloth:       2026.6.1
Unsloth Zoo:   2026.6.5
Transformers:  5.10.2 compatibility build
PyTorch:       2.10.0 + CUDA 12.8

A Transformers installation that does not recognize the gemma4_unified architecture may fail to load the model. Use a current compatible Unsloth and Transformers environment.

Example with Unsloth

import torch
from unsloth import FastLanguageModel

model_id = "trjxter/Gwimi-4-12B-IT-BF16"

model, processor = FastLanguageModel.from_pretrained(
    model_name=model_id,
    max_seq_length=4096,
    dtype=torch.bfloat16,
    load_in_4bit=False,
)

FastLanguageModel.for_inference(model)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Solve the problem carefully and provide a clear final answer: What is 17% of 240?"
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
    )

generated_ids = output_ids[0, inputs["input_ids"].shape[-1]:]

print(
    processor.decode(
        generated_ids,
        skip_special_tokens=True,
    )
)

For deterministic evaluation, use greedy decoding or a low-temperature configuration and keep decoding parameters identical across compared models.

---

Recommended evaluation approach

For a meaningful comparison, evaluate all three checkpoints under identical conditions:

1. Original Gemma 4 12B instruction base
2. Gwimi SFT-only checkpoint
3. Final Gwimi SFT + GSPO model

Keep the following fixed:

• prompt set;
• prompt formatting;
• chat template;
• maximum generated tokens;
• temperature and top-p;
• random seeds;
• answer extraction;
• reward and benchmark scoring;
• hardware and inference backend where practical.

Useful evaluation categories include:

• exact-answer mathematics;
• scientific multiple choice;
• code generation and debugging;
• instruction-format compliance;
• long-response termination behavior;
• repetition and anomaly rate;
• response length;
• pass@1 and sampled pass@k;
• qualitative reasoning review.

---

Training and release hardware

Stage	Hardware
SFT	NVIDIA RTX PRO 6000 Blackwell Server Edition
GSPO	NVIDIA H200 on Modal
Final BF16 merge	NVIDIA A100-SXM4-80GB

---

Acknowledgements

This release builds on:

• the Gemma model family;
• Unsloth;
• Hugging Face Transformers, PEFT, TRL, and Hub;
• Math-Verify;
• the dataset authors and teacher-model outputs represented in the SFT and GSPO data mixtures.

---

License

This model is a derivative of Gemma and remains subject to the applicable Gemma license and terms of use.

Users are responsible for reviewing the upstream license and ensuring that their intended use complies with it.