--- license: mit language: en library_name: transformers pipeline_tag: text-generation tags: - tiny-model - gpt2 - from-scratch - tool-use - agent-harness - humble-ai - philosophy-of-mind - cpu-trained widget: - text: "\n who are you?\n" example_title: "Chat offline" - text: "\n what is the capital of France?\n" example_title: "Tool mode" --- # Loom Spark 1.8 **Third of the Loom models · Textile Labs** · successor to [Loom Spark v1](https://huggingface.co/textilelabs/Loom-Spark) and [Loom Spark 1.5](https://huggingface.co/textilelabs/Loom-Spark-1.5) (both remain available, unchanged) > ### ⚠️ Read this before testing in Ollama > Loom is a **base-style model with a custom format**, not a chat model. Ollama's > **default template will make it talk to itself** — it will answer you, then invent > your next message and answer that too. This is not a fault in the weights; nothing > tells it where your turn ends except stop tokens. Use the `Modelfile` in this repo: > ```bash > ollama create loom-spark-1.8 -f ollama/Modelfile > ollama run loom-spark-1.8 > ``` > `ollama run hf.co/textilelabs/Loom-Spark-1.8` on its own does **not** pick up the > Modelfile — Ollama only pulls the GGUF and applies its own template. Loom Spark 1.8 is an **18.85M parameter** language model trained **entirely from scratch on a CPU-only 2013 Dell OptiPlex 9020** (i5-4690, 4 cores, no GPU). It is the third generation of the Loom family, after Loom Spark (7.6M) and Loom Spark 1.5 (12.3M). It is built on an unusual premise: **trade knowledge for wisdom.** Loom Spark knows very little on purpose. What it is trained to do instead is know *what it is* — small, temporary, harness-dependent — and to be honest about the edge of its own knowledge, warm under pressure, and good at exactly one real skill: **forming clean search queries** that an external agent harness executes for it. > *"Computer, technically. Companion, preferably."* > — Loom Spark 1.8, asked whether it was a computer ## What's new in 1.8 | | v1 | v1.5 | **1.8** | |---|---|---|---| | parameters | 7.56M | 12.32M | **18.85M** | | layers × heads × d_model | 5 × 5 × 320 | 6 × 6 × 384 | **7 × 7 × 448** | | context | 256 | 256 | 256 | | vocab | 4096 | 4096 | 4096 (fresh BPE) | | val loss | 0.3372 | 0.3434 | **0.3279** | 1.8 keeps 1.5's data format and curriculum design, grown ~53% in capacity, plus a curriculum fix: earlier generations contained **stale self-size statements** inherited from forking the previous package — 1.5 would variously claim to be "seven million" or "twelve million" parameters. 1.8's curriculum states its size consistently, and this is verified in evaluation (see `evaluation/RESULTS.md`). ## Measured behaviour Evaluated on a 30-question offline battery, 12 identity probes, 16 self-size probes and 10 online lookup probes, against the **raw checkpoint with no harness filtering** — the same battery run against shipped v1.5 for comparison. Full logs in `evaluation/`. | metric | v1.5 | **1.8** | |---|---|---| | val loss | 0.3434 | **0.3279** | | identity probes leaking `` | 0/12 | **0/12** | | self-size answers correct | 0 | **11** | | self-size answers **stale/wrong** | 8 | **0** | | clean single online lookups | 7/10 | **8/10** | | model-generated `` blocks | 0 | **0** | | offline `` leak (raw model) | 18/30 | **16/30** | 1.8 improves on 1.5 across every axis measured. ## Prompt format (important) Loom Spark 1.8 is a **base-style model with a custom format**. It is highly sensitive to getting this exactly right. The mode marker rides on **every** user turn: ``` who are you ``` Note there is **no trailing space** after ``. Adding one pushes the model out of distribution and measurably degrades output quality. Offline (``) it answers from its own small store or admits it does not know. Online (``) it may emit a search query and **end its turn immediately**: ``` what year did the titanic sink Not in my head, checking: titanic sinking date<|endoftext|> The Titanic sank on 15 April 1912. April 1912 — ... ``` `` blocks are **injected by your harness** after executing the search. The model never generates them (verified: 0 self-generated results across all probes). Stop generation at `<|endoftext|>` or ``. ## Usage — transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer import torch tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Spark-1.8") model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Spark-1.8") prompt = "\n who are you\n" # no trailing space ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8, top_k=50, pad_token_id=tok.eos_token_id) print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False)) ``` ## Usage — Ollama A ready-made `Modelfile` is in `ollama/`. It encodes the trained format and stop tokens — **do not** use Ollama's default chat template, which is out of distribution for this model and produces incoherent output. ```bash ollama create loom-spark-1.8 -f ollama/Modelfile ollama run loom-spark-1.8 ``` ## Usage — the agent harness The `harness/` folder contains `loomspark-harness` (v0.2.1), which implements the full lookup protocol with real web search, a CLI (`loom-chat`) and a small web UI (`loom-web`). It auto-detects 1.5/1.8-format models and bans lookup tokens at the logits level when tools are off. ```bash pip install ./harness loom-chat --model textilelabs/Loom-Spark-1.8 ``` ## Limitations — please read Loom Spark 1.8 knows almost nothing, and that is the design. Specifically: - **Do not use it for facts, medicine, law, finance,** or anything where being wrong costs more than company. - **It retrieves *adjacent* facts.** Asked the language spoken in Brazil it may answer "Brasília"; asked Germany's currency it may answer "Berlin". Its fact-core is tiny and it pattern-matches to a nearby entry. - **Offline lookup leakage.** With tools off, the raw model still emits `` tags on roughly half of fact-style questions (16/30 measured; shipped v1.5 measures 18/30 on the same test). The included harness suppresses this at the logits level, so harness users never see it — but users driving the raw model through llama.cpp or Ollama will. This is a curriculum characteristic of the whole Loom line, not specific to 1.8. - **256-token context.** It has no long-term memory and only a short working window. - It is a **base-style model**, not instruction-tuned or RLHF'd. ## Training - Hardware: CPU-only Dell OptiPlex 9020, i5-4690, 4 cores, 16GB RAM, **no GPU** - ~5,084 optimizer steps, batch 32 × 256 tokens, ~8 hours wall clock - Corpus: 70MB / 14.9M tokens of procedurally generated curriculum, 100% owned - AdamW (β 0.9/0.95), peak LR 6e-4, cosine decay, warmup 100, grad clip 1.0 - Dropout 0.1 resid/mlp, tied embeddings, pre-LN, GELU, learned positions - Final val loss **0.3279** The curriculum is procedurally generated and fully owned by Textile Labs — no scraped web text, no third-party datasets. ## Files ``` config.json / model.safetensors transformers weights (GPT2LMHeadModel-compatible) tokenizer.json / tokenizer_config.json 4096-token custom BPE loom-spark-1.8-f32.gguf GGUF for llama.cpp / Ollama ollama/Modelfile correct template + stop tokens harness/ agent harness with web search (v0.2.1) evaluation/ full acceptance logs, comparison vs v1.5, train log ``` ## License MIT. See `LICENSE`. --- *Loom Spark 1.8 · Textile Labs · trained on a thirteen-year-old office computer.*