qwen3.6-a3b-blackhole
Qwen3.6-35B-A3B (hybrid Gated-DeltaNet + full attention, 256-expert MoE, 8 active) served on a single Tenstorrent Blackhole P150a through vLLM. Multi-turn prefix reuse keeps time-to-first-token roughly flat as a conversation grows, instead of re-prefilling the whole transcript each turn.
Runs on p150 (mesh P150) β 262,144-token context, up to 1 concurrent sequences.
Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).
Quickstart
tt-model pull Adartras/qwen3.6-a3b-blackhole --with-weights
tt-model serve Adartras/qwen3.6-a3b-blackhole
pull --with-weights downloads the Docker image and the Qwen/Qwen3.6-35B-A3B weights at 995ad96eacd98c81ed38be0c5b274b04031597b0 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.
Serve
tt-model serve Adartras/qwen3.6-a3b-blackhole
Then point any OpenAI-compatible client at the port it prints (tt-model serve picks one from
20000 upward; a bare docker run binds 8000).
What to expect on one P150a
- decode ~37 tok/s, prefill ~600-880 tok/s
- multi-turn TTFT ~0.3 s steady on a growing chat, 96-99% of each prompt reused
- up to 4 conversations stay warm at once, so a client that interleaves prompt streams
(a chat UI's auto-title requests alongside your conversation) keeps its prefix. Slots age
out least-recently-used;
QWEN36_CONV_SLOTS=1restores single-conversation behaviour - 256K context ceiling (
QWEN36_MAX_SEQ), bf4 experts, bf8 KV. KV is paged, one pool shared by the conversation slots, so this costs the same memory as a single flat cache
Reasoning ("thinking") blocks are parsed out via --reasoning-parser qwen3, and tool calls via
--tool-call-parser qwen3_coder.
Provenance
The exact sources the image was built from β code/ in this repo is byte-identical to the model code inside the image:
| component | built from |
|---|---|
| tt-metal | d53c2a71fb0f1830077f1fea488fe3e85297f5a6 |
| vLLM | v0.26.0 |
| vllm-tt-plugin | bef89e429e202caa38d3e3c8a24da4e8cc02405a |
code/ digest |
62042a7925bfbf60 (sha256, first 16 hex digits) |
| built | 2026-09-09T00:31:23+00:00 by tt-model 0.1.0 |