Estragon-9B — GGUF

GGUF quantizations of Estragon-9B, a Godot 4.7 / GDScript specialist (Qwen3.5-9B fine-tuned with SFT + GRPO using a headless-Godot judge as reward). Full model card, benchmarks, and method: bf16 repo · GitHub (harness, eval sets, ADRs).

Which file?

Every quant was run through the full 300-task GDScript-Eval v2 (headless Godot 4.7 as judge, greedy pass@1, deployment system prompt) — the quantization cost is measured, not guessed:

file bits size gdeval_v2 vs bf16 (248/300)
estragon-9b-Q5_K_M.gguf 5-bit 6.5 GB 249/300 (83.0%) +1 — statistically identical to bf16. Recommended.
estragon-9b-Q8_0.gguf 8-bit 9.5 GB 248/300 (82.7%) ±0 — indistinguishable from bf16
estragon-9b-Q4_K_M.gguf 4-bit 5.6 GB 229/300 (76.3%) −19 — the 8 GB VRAM pick; API-precision degrades most

SHA-256:

6803c3c115bdd4a5de2f54e860107cb8b7b384443f7a4e97eced1160c2415281  estragon-9b-Q4_K_M.gguf
db10164493835e3039b1cc4d931fd1e9e22b1c8dd99a4e34b52ab8968b51785d  estragon-9b-Q5_K_M.gguf
5acbb583d451e4486618faaf95d424fbc3a3d5b70dc0604492054e8c8e0be0cd  estragon-9b-Q8_0.gguf

Ollama

ollama run hf.co/Blugart/estragon-9b-gguf:Q5_K_M --think=false "A 2D platformer jump with coyote time, CharacterBody2D"

⚠️ Pass --think=false (API: "think": false). Ollama's qwen3.5 runtime manages <think> blocks itself and there is currently no model-level way to disable it (ollama#14617). Estragon is a non-thinking fine-tune: without the flag it burns tokens on reasoning and measured −15 tasks on the 300-task eval.

This repo ships template, system, and params files, so Ollama picks up the chat template, the deployment system prompt, and the recommended sampling parameters automatically. Alternatively, download a GGUF and use the included Modelfile:

ollama create estragon -f Modelfile
ollama run estragon --think=false "..."

llama.cpp

llama-server -m estragon-9b-Q5_K_M.gguf -ngl 99 --port 8080 \
  --chat-template-kwargs '{"enable_thinking":false}'

⚠️ enable_thinking:false is required. The embedded Qwen3.5 chat template otherwise opens a reasoning block; this is a non-thinking fine-tune and will produce empty or degraded answers with thinking enabled (measured: −15 tasks, and unclosed think blocks eat the whole token budget).

The system prompt is part of the model — all numbers were measured with it (see the system file in this repo, or the full card). Recommended sampling: temperature 0.7, top_p 0.8, top_k 20; do not enable reasoning/thinking mode.

License

Apache 2.0 (inherited from Qwen3.5-9B).

Downloads last month
1,383
GGUF
Model size
9B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Blugart/estragon-9b-gguf

Finetuned
Qwen/Qwen3.5-9B
Quantized
(2)
this model