Request access to Dwago
Dwago is released under the Dwago Evaluation License. Approved users may download the weights to evaluate, test and benchmark the model, or for non-commercial academic research. Any commercial or production use requires a separate paid license from Mohammedi Yunus; contact via the Hugging Face profile muhammediyunes. Requests are reviewed manually.
Log in or Sign Up to review the conditions and access this model content.
Dwago v8.1e: a 157M-parameter decision model trained from scratch
Created by Mohammedi Yunus. Open weights under the Dwago Evaluation License: evaluation and non-commercial research only; commercial use requires a paid license (see License and commercial use).
Dwago answers questions about a piece of content (the state: plain text or an object with named fields) and returns probabilities. It supports three question types: choice (pick one of N described options), yes/no, and score (an ordinal rating). v8.1e is an equal-weight ensemble of two Dwago networks, v7.2 (34M parameters) and v8.1b (123M). Every weight in both was trained from random initialisation in this project: its own tokenizer, encoders pretrained on 8.58 and 7.34 billion tokens of decontaminated text, and multi-task supervised fine-tuning. No pretrained weights, external embeddings or pretrained tokenizers were used, and no other model's predictions were used as labels (the only soft targets came from ensembles of Dwago's own from-scratch models).
Highlights
- Above Jev on 10 of 17 Jev-published benchmarks (5 with Holm-supported superiority of at least 2 points). Largest leads: Emotion (system test) 94.00 vs 55.00, Emotion (published) 87.75 vs 58.65, Security labels 88.00 vs 71.50, Finance topics 82.03 vs 66.99, Banking77 96.88 vs 90.63.
- Better than Dwago v7.2 on 9 of 17 (DailyDialog, Finance topics, Tweet topics, Emotion (system test), Moderation, Triage, Code correctness, Security labels, JevBench); lower on 3 (Emotion (published), MNLI, Security pair ranking).
- Above Laya on 14 of 17 of the same benchmarks.
- Sealed held-out sets (built for cycle v7, evaluated once): triage 82.27 (v7.2 76.82), multilingual 82.08 (77.14), JevBench-style 50.19 (48.44), code sealed C 84.18 (82.77).
- Fast: 4.89 ms median per request on an RTX 3070 for both networks together (157.2M parameters).
Benchmark results against Jev and Laya
Same inputs as Jev's public results; each benchmark's own (publisher) metric; every system evaluated once from a frozen checkpoint. Bold = v8.1e above Jev; * = Holm-supported superiority over Jev of at least 2 points (paired whole-group bootstrap, one Holm family of 17 per system).
| Benchmark | Jev 1.13.0 | Dwago v8.1e | Dwago v7.2 | Dwago v7 | Dwago v6.2 | Laya (shipped) |
|---|---|---|---|---|---|---|
| DailyDialog | 70.99 | 77.09* | 75.22* | 76.98* | 79.29* | 61.37 |
| Emotion (published) | 58.65 | 87.75* | 88.40* | 84.90* | 88.65* | 58.80 |
| Finance topics | 66.99 | 82.03* | 79.14* | 79.43* | 81.78* | 34.08 |
| Tweet topics | 79.33 | 82.40 | 79.80 | 79.15 | 79.21 | 63.20 |
| AG News | 91.00 | 92.00 | 92.00 | 93.00 | 92.00 | 94.00 |
| Banking77 | 90.63 | 96.88 | 96.88 | 95.83 | 94.79 | 80.21 |
| Emotion (system test) | 55.00 | 94.00* | 87.00* | 87.00* | 89.00* | 55.00 |
| Guardrails | 96.67 | 90.00 | 90.00 | 91.67 | 83.33 | 88.33 |
| MNLI | 86.67 | 70.00 | 71.67 | 73.33 | 78.33 | 98.33 |
| Moderation | 98.89 | 94.44 | 93.33 | 91.11 | 83.33 | 83.33 |
| Multilingual intent | 100.00 | 76.00 | 76.00 | 76.00 | 40.00 | 36.00 |
| Triage | 88.75 | 86.25 | 84.38 | 84.38 | 66.25 | 79.38 |
| Code correctness | 98.00 | 77.86 | 76.30 | 78.91 | 51.04 | 51.04 |
| Code quality | 99.48 | 100.00 | 100.00 | 100.00 | 67.45 | 54.43 |
| Security labels | 71.50 | 88.00* | 86.00* | 57.00 | 87.25* | 50.25 |
| Security pair ranking | 89.00 | 94.50 | 96.00* | 68.50 | 95.00 | 47.50 |
| JevBench | 86.58 | 48.92 | 47.62 | 43.72 | 44.16 | 54.55 |
| Above Jev | — | 10 of 17 | 10 of 17 | 7 of 17 | 8 of 17 | 3 of 17 |
Where Dwago is still behind Jev: Triage (86.25 vs 88.75); Moderation (94.44 vs 98.89); Guardrails (90.00 vs 96.67); MNLI (70.00 vs 86.67); Code correctness (77.86 vs 98.00); Multilingual intent (76.00 vs 100.00); JevBench (48.92 vs 86.58). Gaps range from 2.50 to 37.66 points. These are reported, not hidden.
Code review. The code-review numbers (correctness and quality) come from the full system: this ensemble plus Dwago's code-review tool layer (static checks, sandboxed execution of supplied tests, behaviour probes), which is not included in this package. The tool layer only handles requests whose state has a candidate_source field; every other request goes straight to the ensemble.
Full report with sealed held-out sets, size and speed, and method: benchmark/README.md. Machine-readable results: benchmark/results/.
How Dwago was built
From an evaluation suite fixed before training, through tokenizer training, pretraining on one RTX 3070, free Kaggle GPUs and a Colab A100, multi-task fine-tuning and self-distillation, to one-time evaluations recorded in append-only ledgers: benchmark/HOW_DWAGO_WAS_BUILT.md. The project was engineered with an AI coding agent (Claude Code) under the author's direction; the model's weights were all learned from random initialisation by these pipelines.
Quick start
from dwago_model import Dwago # dwago_model.py ships with the weights
model = Dwago.from_pretrained('.') # loads both members; CUDA if available, else CPU
model.predict({'message': 'Ich möchte heute Abend eine Pizza bestellen.'}, {
'intent': {'type': 'choice', 'instructions': 'What does the customer want?',
'criteria': {'takeaway_order': 'order takeaway food', 'transport_taxi': 'book a taxi', 'other': 'anything else'}},
'urgent': {'type': 'noul', 'instructions': 'Is there time pressure in `message`?'},
})
- Requirements:
torch,tokenizers,safetensors,numpy. - Check: this package's code reproduces the research engine on 200 development decisions (100% argmax agreement, max probability difference 3.0e-08;
results/package-check-v81e.json). - No ONNX export is included in this release.
Model
| Members | v7.2: 10-layer pre-LN Transformer encoder, width 384, 6 heads (33,925,633 parameters); v8.1b: 12 layers, width 768, 12 heads (123,314,689 parameters). Both with a schema-conditioned decision head. |
| Total parameters | 157,240,322 |
| Combination | Equal-weight average of the two members' probabilities (no weight tuning; fixed before selection) |
| Input | [CLS] question [SEP] state [SEP] (question ≤ 128 tokens, at most 512 positions in total). Each option description (≤ 62 tokens) is encoded separately and cached. |
| Tokenizer | 33,919-token lower-cased BPE with word-start markers: 24,576 tokens trained on the decontaminated English corpus plus 9,343 tokens for non-Latin scripts. Shared by both members. |
| Layout markers | Line breaks and indentation are encoded as ¶ and ¦, so code and multi-line fields keep their structure. |
| Precision | Weights fp32. GPU inference bf16. |
| Speed | 4.89 ms median, 6.99 ms p95 per request on an RTX 3070 (batch 1, end to end, CUDA graphs); v7.2 alone 1.70 ms |
Training data
Pretraining:
- English Wikipedia 2023-11-01 and FineWeb-Edu sample-10BT.
- Decontaminated against every evaluation text before the tokenizer was trained: 13-gram overlap, exact short sentences, and evaluation article titles.
- Two of the v6 teacher's members additionally saw 0.4B tokens of Python code (CodeSearchNet, codeparrot-clean validation, APPS; mixed licenses as stated by the sources).
Supervised (training splits only).
- The v5 pool: DBpedia-14, SNLI, MultiNLI, SQuAD 2.0 answerability, Banking77, BoolQ, AG News and STS-B, plus schema augmentations of these rows.
- v6 tasks: the task types below, plus request-format variants and execution-labelled code review built from MBPP and HumanEval.
- v6.2 additions:
- multilingual intent, scenario, topic, banking and NLI data (MASSIVE in 51 languages, SIB-200, MINDS-14, XNLI);
- code-review and code-execution data generated by this project, labelled by running the code and by syntax-tree checks.
- Screening: every training row was checked against every evaluation input (exact text and 5-gram containment ≥ 0.9). Non-Latin text uses a Unicode-aware normaliser and character 5-grams. Matches were removed.
| Task | Source | License (as stated by the source) | Training rows |
|---|---|---|---|
| abuse_severity | google/civil_comments | CC0 1.0 | 52,605 |
| anli | facebook/anli | CC BY-NC 4.0 | 60,000 |
| arc | allenai/ai2_arc | CC BY-SA 4.0 | 2,747 |
| code_exec | generated by this project (assertion outcomes, executed) | this project | 13,357 |
| code_review2 | generated by this project (function families, executed) | this project | 50,000 |
| code_review3 | generated by this project (function families, executed) | this project | 96,000 |
| code_vulnerable | CyberNative/Code_Vulnerability_Security_DPO | Apache-2.0 | 7,230 |
| copa | super_glue/copa | BSD-2 | 319 |
| csqa | tau/commonsense_qa | MIT | 9,115 |
| daily_dialog | DailyDialog (roskoN mirror) | CC BY-NC-SA 4.0 | 28,586 |
| emotion | dair-ai/emotion | other (research/education) | 17,273 |
| fin_topic | zeroshot/twitter-financial-news-topic | MIT | 15,603 |
| fmt_field | format variant of abuse_severity | CC0 1.0 | 56,899 |
| fmt_nli | format variant of snli | release | 59,200 |
| glue_cola | nyu-mll/glue | GLUE (per-task terms) | 7,894 |
| glue_mrpc | nyu-mll/glue | GLUE (per-task terms) | 2,850 |
| glue_qnli | nyu-mll/glue | GLUE (per-task terms) | 59,384 |
| glue_qqp | nyu-mll/glue | GLUE (per-task terms) | 59,378 |
| glue_rte | nyu-mll/glue | GLUE (per-task terms) | 1,990 |
| harm_severity | PKU-Alignment/PKU-SafeRLHF | CC BY-NC 4.0 | 37,974 |
| hellaswag | Rowan/hellaswag | MIT | 39,176 |
| identity_attack | google/civil_comments | CC0 1.0 | 25,297 |
| insult | google/civil_comments | CC0 1.0 | 57,615 |
| jailbreak | jackhhao/jailbreak-classification | Apache-2.0 | 792 |
| massive_intent | MASSIVE (mteb mirror) | CC BY 4.0 | 59,980 |
| ml_bank | MINDS-14 | CC BY 4.0 | 8,145 |
| ml_intent | MASSIVE | CC BY 4.0 | 201,710 |
| ml_nli | XNLI | CC BY-NC 4.0 | 74,990 |
| ml_scenario | MASSIVE | CC BY 4.0 | 40,746 |
| ml_topic | SIB-200 | CC BY-SA 4.0 | 61,500 |
| multirc | super_glue/multirc | other | 4,045 |
| openbookqa | allenai/openbookqa | unknown | 4,206 |
| prompt_injection | deepset/prompt-injections | Apache-2.0 | 7,932 |
| sciq | allenai/sciq | CC BY-NC 3.0 | 11,073 |
| sensitive_data | ai4privacy/pii-masking-200k | other | 57,632 |
| spam | ucirvine/sms_spam | unknown | 4,512 |
| sst2 | stanfordnlp/sst2 | unknown | 57,348 |
| sst5 | SetFit/sst5 (SST-5) | unknown | 8,945 |
| support_intent | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| threat | google/civil_comments | CC0 1.0 | 7,633 |
| ticket_urgent | Tobi-Bueck/customer-support-tickets | CC BY-NC 4.0 | 11,132 |
| toxic | google/civil_comments | CC0 1.0 | 57,582 |
| tweet_topic | cardiffnlp/tweet_topic_single | other | 4,287 |
| wants_cancel | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| wants_refund | bitext customer support | CDLA-Sharing-1.0 | 22,280 |
| wic | super_glue/wic | CC BY-NC 4.0 | 4,827 |
| winogrande | allenai/winogrande | CC BY | 8,615 |
Added in cycles v7 and v8:
- Public data (541,875 rows):
| Task | Sources (as named on the Hugging Face Hub) | Training rows |
|---|---|---|
| v7_injection | jayavibhav/prompt-injection, reshabhs/SPML_Chatbot_Prompt_Injection, databricks/databricks-dolly-15k, Lakera/mosscap_prompt_injection, TrustAIRLab/in-the-wild-jailbreak-prompts, Lakera/gandalf_ignore_instructions | 47,581 |
| v7_jailbreak | TrustAIRLab/in-the-wild-jailbreak-prompts, GuardrailsAI/detect-jailbreak, databricks/databricks-dolly-15k, lmsys/toxic-chat | 6,752 |
| v7_nli | alisawuffles/WANLI, pietrolesci/nli_fever, tasksource/lingnli | 205,889 |
| v7_scitail | allenai/scitail | 22,862 |
| v7_spam | thehamkercat/telegram-spam-ham, SetFit/enron_spam, Deysi/spam-detection-dataset | 41,682 |
| v7_threat | ucberkeley-dlab/measuring-hate-speech, jigsaw (Arsive mirror) | 11,930 |
| v7_toxic | jigsaw (Arsive mirror), ucberkeley-dlab/measuring-hate-speech, toxigen/toxigen-data, lmsys/toxic-chat | 53,621 |
| v7_toxic_ml | textdetox/multilingual_toxicity_dataset (es), textdetox/multilingual_toxicity_dataset (zh), textdetox/multilingual_toxicity_dataset (uk), textdetox/multilingual_toxicity_dataset (ar), textdetox/multilingual_toxicity_dataset (am), textdetox/multilingual_toxicity_dataset (ru), textdetox/multilingual_toxicity_dataset (tt), textdetox/multilingual_toxicity_dataset (hi), textdetox/multilingual_toxicity_dataset (it), textdetox/multilingual_toxicity_dataset (ja), textdetox/multilingual_toxicity_dataset (en), textdetox/multilingual_toxicity_dataset (fr), textdetox/multilingual_toxicity_dataset (de), textdetox/multilingual_toxicity_dataset (hin), textdetox/multilingual_toxicity_dataset (he) | 69,502 |
| v7_tri_churn | bitext/retail-banking, bitext/telco, bitext/insurance, bitext/retail-ecommerce, bitext/media, bitext/wealth-management, bitext/customer-support, bitext/mortgage-loans, bitext/travel, bitext/restaurants, bitext/events-ticketing, bitext/hospitality | 14,876 |
| v7_tri_intent | Tobi-Bueck/customer-support-tickets (en), Tobi-Bueck/customer-support-tickets (de), bitext/retail-ecommerce, bitext/insurance, bitext/restaurants, bitext/customer-support, bitext/hospitality, bitext/events-ticketing, bitext/media, bitext/mortgage-loans, bitext/telco, bitext/retail-banking, bitext/travel, bitext/wealth-management | 55,683 |
| v7_tri_refund | bitext/retail-ecommerce, bitext/events-ticketing, bitext/customer-support, bitext/travel, bitext/retail-banking, bitext/hospitality, bitext/insurance, bitext/telco, bitext/mortgage-loans, bitext/restaurants, bitext/media, bitext/wealth-management | 11,497 |
- Generated decision families (320,578 training rows): generator programs written by separate AI writer agents under a fixed contract; each generator fixes the answer first and writes content that supports it, so labels are true by construction (records:
results/dec-data.json). - Described yes/no rows (68,022): yes/no questions whose options are descriptions, in this project's own wording (
results/desc-noul.json). - Pretraining for v8.1b: additionally a 51-language Wikipedia corpus (
results/mlcorpus.json) and Python code. - Screening: every training row was checked against every benchmark input (exact text and 5-gram containment ≥ 0.9, Unicode-aware); matches were removed.
Data licenses. The training data includes sources under non-commercial, share-alike, research-only or unstated terms (for example CC BY-NC 4.0: ANLI, XNLI, PKU-SafeRLHF, WiC, ticket data; CC BY-NC-SA 4.0: DailyDialog; research/education-only: dair-ai/emotion). The licenses of the cycle v7 sources are as stated by each source and were not re-verified for this card. No dataset is redistributed here.
Evaluation
- Selection on development data only: the mean of nine development task groups. v8.1e scored 79.79 (v7.2: 79.15) and passed the code-review guard (
results/select-v81-ens-s1.json,results/tools-dev2-scores.json). - One-time evaluations of the frozen system:
sealed7: decision items from writer modules whose outputs were never used for training or selection (aggregates only). Code sealed C and blind A: code-review sets held out from all training and selection; family-balanced correctness at the frozen threshold 0.5.
| System | sealed7 triage | sealed7 multilingual | sealed7 JevBench-style | Code sealed C | Blind A |
|---|---|---|---|---|---|
| v8.1e | 82.27 | 82.08 | 50.19 | 84.18 | 79.49 |
| v7.2 | 76.82 | 77.14 | 48.44 | 82.77 | 79.05 |
- Protocol for the Jev comparison: paired whole-group bootstrap (10,000 replicates) with one Holm family across all 17 comparisons. A win needs Holm-supported superiority and an estimated difference of at least 2 points. Jev's numbers are its published per-item outcomes (Jev 1.13.0), not new API calls; Dwago ran on exactly the same inputs.
- Post-hoc disclosure: cycles v7 and v8 were designed after earlier benchmark results showed which task types were weak, so their data choices are benchmark-informed. No benchmark item was ever used for training, tuning or selection.
Intended use and limitations
- Intended for research and prototyping of fast, local, structured decisions (routing, triage, tagging, moderation, intent and entailment-style checks) with human review.
- Not validated for consequential decisions (medical, legal, financial, employment) or safety-critical use. It is not a security scanner or a code reviewer.
- Weak spots: MNLI-style entailment, multilingual intent, multi-step JevBench-style decisions, and code correctness (which needs the tool layer that is not part of this package); long documents are truncated at 512 tokens.
- Calibration can degrade under distribution shift.
Reproducibility
results/ holds the pre-registrations (with every addendum), the selection records, the append-only evaluation ledger with the SHA-256 of every results file, the Jev comparisons, the sealed-set scores, the data records, the latency record and the package check. The training code is not published with this release; the pipeline is described step by step in the build write-up.
License and commercial use
- Released under the Dwago Evaluation License (LICENSE). Copyright (c) 2026 Mohammedi Yunus. Open weights, not open source.
- Allowed after access approval: evaluating, testing and benchmarking the model; non-commercial academic research, with credit to "Dwago by Mohammedi Yunus".
- Not allowed without a separate written license: any commercial or production use, redistribution of the weights or of modified versions, and using Dwago or its outputs to train other models.
- Commercial licenses (fees, royalties or revenue sharing) are available from Mohammedi Yunus: contact via the Hugging Face profile muhammediyunes.
- Some training datasets carry their own non-commercial or research-only terms (see Training data); this license grants no rights in them.
Citation
@misc{yunus2026dwago,
author = {Mohammedi Yunus},
title = {Dwago: a compact structured-decision model trained from scratch},
year = {2026}, howpublished = {\url{https://huggingface.co/muhammediyunes/dwago}}
}
- Downloads last month
- -