Instructions to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with Ollama:
ollama run hf.co/DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with Docker Model Runner:
docker model run hf.co/DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
- Lemonade
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Linnaeus-0.1.0-2B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use DreamBlooms/Linnaeus-0.1.0-2B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "DreamBlooms/Linnaeus-0.1.0-2B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Linnaeus-0.1.0-2B
Built on Qwen/Qwen3.5-2B, this checkpoint returns decision distributions from text and images. It supports candidate selection (choice), truth estimates (noul), and ordered scores (score) without generating reasoning or free-form responses.
Benchmarks (measured)
- Held-out macro accuracy 80.78% (26 groups, 180k examples; upstream Dohnuts-0.1.0-0.8B: 78.21%)
- JevBench v1.2.2 public 231 tasks: 73.16% (easy 100 / standard 91.7 / hard 49.6) โ top of the ~2B local-model class, ahead of all sub-1B classifiers
- Laya app suites: beats upstream on 7/11 groups (spam, routing, typed-decisions strongest)
- XNLI 15-language: 76.0% โ recovers the suite upstream lost to Laya multilingual (73.8%)
On-device exports (Apple Silicon)
Merged + quantized builds for macOS/iOS (text MLX builds are text-only; the VLM builds keep the vision tower):
| Repo | Size | JevBench | Target |
|---|---|---|---|
| Linnaeus-0.1.0-2B-merged | 4.3 GB | 71.0% (MPS) | Mac dev / conversion source |
| Linnaeus-0.1.0-2B-MLX-8bit | 1.9 GB | 70.56% | Mac + iPhone |
| Linnaeus-0.1.0-2B-MLX-4bit | 1.0 GB | 67.53% | iPhone size-optimized |
| Linnaeus-0.1.0-2B-MLX-VLM-8bit | 2.5 GB | text 70.56% + images | Mac + iPhone multimodal |
| Linnaeus-0.1.0-2B-MLX-VLM-4bit | 1.6 GB | text ~67% + images | iPhone multimodal, size pick |
The merged checkpoint embeds the decision head as an extra lm_head row
(score_row_id), so any stock LM runtime produces decision scores at
<|fim_suffix|> marker positions. See linnaeus-runtime.json and
github.com/pi-dal/Linnaeus src/linnaeus/mlx_predictor.py.
Model details
| Property | Value |
|---|---|
| Base model | Qwen/Qwen3.5-2B |
| Selection | Seed 42, update 2,800 |
| Inference | merged LoRA, BF16, fused operations, shared-prefix parallel candidate scoring |
| Inputs | Text and one PIL image; 2โ128 candidates; 4,096 tokens per question |
Questions reuse a shared input prefix and compute their suffixes in parallel. Additional questions still require computation.
Training
The training recipe combines RLCD and auxiliary cross-entropy, following the pinned Laya and Laya Vision references. Temperature calibration uses an independent partition after LoRA merging. Calibration quality is measured below.
Training recipe ยท RLCD implementation and upstream attribution
Evaluation
Held-out macro accuracy: 80.78%.
| Dataset | N | Accuracy | NLL before / after calibration | ECE before / after |
|---|---|---|---|---|
| ag_news | 7600 | 90.04% | 0.8728 / 0.5378 | 0.0832 / 0.0710 |
| aokvqa | 1138 | 83.66% | 0.6347 / 0.4746 | 0.0930 / 0.0513 |
| banking77 | 3080 | 73.70% | 1.0536 / 1.1711 | 0.0275 / 0.2001 |
| boolq | 3270 | 87.71% | 0.3854 / 0.3288 | 0.0610 / 0.0648 |
| clevr_attribute | 53734 | 98.92% | 0.1687 / 0.1230 | 0.0100 / 0.0095 |
| clevr_count | 35422 | 89.72% | 0.7159 / 0.2652 | 0.0793 / 0.0077 |
| clevr_exist | 20196 | 98.58% | 0.2526 / 0.0955 | 0.0137 / 0.0127 |
| contract_nli | 1173 | 85.59% | 0.5014 / 0.4020 | 0.0888 / 0.0338 |
| emotion | 2000 | 77.00% | 0.7237 / 0.6812 | 0.0777 / 0.0488 |
| esci_es | 1482 | 60.26% | 0.9241 / 0.9775 | 0.0342 / 0.1266 |
| esci_jp | 1633 | 63.81% | 0.8878 / 0.9531 | 0.0349 / 0.1410 |
| esci_us | 1145 | 57.64% | 0.9185 / 0.9712 | 0.0489 / 0.0868 |
| mail_phishing | 1050 | 98.95% | 0.1327 / 0.0467 | 0.0097 / 0.0094 |
| mail_spam | 854 | 98.95% | 0.1658 / 0.0580 | 0.0105 / 0.0098 |
| massive_en-US | 2970 | 78.89% | 0.7713 / 0.7944 | 0.0532 / 0.1141 |
| massive_zh-CN | 2921 | 76.82% | 0.8801 / 0.8552 | 0.0664 / 0.0904 |
| scienceqa | 2017 | 92.66% | 0.2942 / 0.2049 | 0.0489 / 0.0342 |
| screenqa_choice | 848 | 22.05% | 2.3429 / 2.4277 | 0.0464 / 0.0603 |
| screenqa_noul | 2148 | 72.35% | 0.5747 / 0.6116 | 0.0341 / 0.1288 |
| sharc | 8276 | 73.19% | 0.6671 / 0.6700 | 0.0705 / 0.0547 |
| sms_spam | 794 | 99.37% | 0.0910 / 0.0319 | 0.0062 / 0.0061 |
| typed_decisions | 2000 | 73.05% | 0.9068 / 0.9944 | 0.1073 / 0.2750 |
| vqav2_yesno | 8102 | 85.88% | 0.3962 / 0.4835 | 0.0248 / 0.1920 |
| wikiqa | 6160 | 96.17% | 0.4844 / 0.1710 | 0.0374 / 0.0322 |
| xnli_en | 5009 | 87.20% | 0.3602 / 0.3727 | 0.0307 / 0.0601 |
| xnli_zh | 5009 | 78.20% | 0.5986 / 0.5582 | 0.0824 / 0.0297 |
Inference speed
Warm RTX 4090 end-to-end predict latency, including preprocessing and transfers. Three warmups and 20 synchronized repetitions; network and queueing excluded.
| Engine | Workload | p50 ms | p95 ms | Decisions/s |
|---|---|---|---|---|
| Linnaeus | vision_protocol_text_1q | 44.53 | 45.29 | 22.4 |
| Linnaeus | vision_protocol_text_3q | 47.81 | 48.29 | 62.9 |
| Linnaeus | vision_protocol_image_1q | 49.88 | 51.27 | 19.9 |
| Linnaeus | vision_protocol_image_3q | 100.83 | 102.87 | 29.6 |
| Linnaeus | distinct_text_1q | 46.83 | 48.00 | 21.3 |
| Linnaeus | distinct_text_5q | 94.32 | 95.89 | 52.8 |
| Linnaeus | distinct_text_10q | 94.83 | 96.16 | 105.1 |
| Linnaeus | distinct_text_50q | 122.42 | 124.17 | 407.7 |
Benchmark coverage
Use
from linnaeus.predictor import Predictor
model = Predictor.from_checkpoint("runs/2b/checkpoint")
Installation, question definitions and response fields.
Limitations
For choice and score, confidence is 1 - H(p) / log(K). For noul, it is max(p, 1 - p). These distribution summaries are not empirical correctness guarantees; calibration metrics use maximum probability and observed correctness.
Raw predictions, resource samples, calibration bins and quality diagnostics accompany this report.
- One seed; variation across seeds is unmeasured. Development selects weights; calibration fits temperatures; test never selects either.
- Laya references use their own templates, FP32 CPU weights and published temperatures; Linnaeus uses merged BF16 weights.
- Laya Vision has VQAv2 source-pool and A-OKVQA selection exposure. Backbone pretraining exposure is unverified.
- Bub acceptance verifies local decision-tool calls, not autonomous planning quality.
- Source model, dataset and image terms apply; this report does not assign a new weight license.
Reproducibility
Base revision: 15852e8c16360a2fea060d615a32b45270f8a8fc.
Weight SHA-256: 0424555feea3126ee02d8d1b12636fad8cf491a5964a00a04b2fec67bcc0f602.
Recipe SHA-256: d0017a70f1b6c789318700339cd454a74a05a3cac9f0687c1c322bd6eb031c62.
Recorded source-file hashes identify the code snapshot used for the run. A training Git revision is not recorded.
GGUF
CPU inference is available through
dohnuts.cpp, a native C++ port on
llama.cpp. It runs the model on CPU,
reads the scalar decision head at each answer slot, and serves the same
POST /v1/systemone wire format. No GPU or Python runtime is needed. The
checkpoint is quantized from pi-dal/Linnaeus-0.1.0-2B
(Qwen/Qwen3.5-2B plus a rank-8 LoRA) and uses the dohnuts profile.
| File | Contents | Size |
|---|---|---|
Linnaeus-0.1.0-2B-F16.gguf |
f16 weights, reference | 3.8 GB |
Linnaeus-0.1.0-2B-Q8_0.gguf |
Q8_0, conservative default | 2.0 GB |
Linnaeus-0.1.0-2B-Q4_K_M.gguf |
Q4_K_M, smallest | 1.3 GB |
mmproj-Linnaeus-0.1.0-2B-bf16.gguf |
vision tower (bf16), for image input | 671 MB |
head.f32 |
scalar decision head, required at runtime | 8.2 kB |
linnaeus.json |
profile, calibrated temperatures, flags | 1.3 kB |
The three text GGUFs are text-only; pair mmproj-Linnaeus-0.1.0-2B-bf16.gguf
with any of them to enable image inputs. head.f32 and linnaeus.json are
required at runtime; linnaeus.json carries the calibrated choice / noul /
score temperatures from the upstream checkpoint.
This conversion is structural and has been smoke-tested with dohnuts-cli: it
loads and answers choice / noul / score, and the F16, Q8_0 and Q4_K_M
builds agree on every argmax on the sample set. It has not been benchmarked
against the PyTorch reference. Q8_0 is the conservative default.
git clone https://github.com/DreamBlooms/dohnuts.cpp
cd dohnuts.cpp
git submodule update --init --depth 1
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON
cmake --build build -j --target dohnuts-cli
build/dohnuts-cli --server --port 8080 \
--model Linnaeus-0.1.0-2B-Q8_0.gguf \
--head head.f32 --metadata linnaeus.json
Add --mmproj mmproj-Linnaeus-0.1.0-2B-bf16.gguf for image inputs.
Ask one state several questions (same request shape as the PyTorch server):
curl http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' \
-d '{"state":"My card was charged twice for the same purchase.",
"questions":{"department":{"type":"choice","instructions":"Which team should handle this?",
"criteria":{"billing":null,"technical support":null,"sales":null}},
"refund_requested":{"type":"noul","instructions":"Is a refund requested?"}}}'
The answer keeps the core fields (type, choice, probabilities, noul,
score, legend) in the same POST /v1/systemone wire format.
Rebuild these files from the upstream adapter with
scripts/build_linnaeus_gguf.sh (merge the rank-8 LoRA into Qwen/Qwen3.5-2B,
export the head, then convert with --no-mtp; the base config declares an MTP
layer but ships no MTP weights).
- Downloads last month
- 65
4-bit
8-bit
16-bit