qwen3.6-a3b-blackhole

Qwen3.6-35B-A3B (hybrid Gated-DeltaNet + full attention, 256-expert MoE, 8 active) served on a single Tenstorrent Blackhole P150a through vLLM. Multi-turn prefix reuse keeps time-to-first-token roughly flat as a conversation grows, instead of re-prefilling the whole transcript each turn.

Runs on p150 (mesh P150) β€” 262,144-token context, up to 1 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  Adartras/qwen3.6-a3b-blackhole --with-weights
tt-model serve Adartras/qwen3.6-a3b-blackhole

pull --with-weights downloads the Docker image and the Qwen/Qwen3.6-35B-A3B weights at 995ad96eacd98c81ed38be0c5b274b04031597b0 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Serve

tt-model serve Adartras/qwen3.6-a3b-blackhole

Then point any OpenAI-compatible client at the port it prints (tt-model serve picks one from 20000 upward; a bare docker run binds 8000).

What to expect on one P150a

  • decode ~37 tok/s, prefill ~600-880 tok/s
  • multi-turn TTFT ~0.3 s steady on a growing chat, 96-99% of each prompt reused
  • up to 4 conversations stay warm at once, so a client that interleaves prompt streams (a chat UI's auto-title requests alongside your conversation) keeps its prefix. Slots age out least-recently-used; QWEN36_CONV_SLOTS=1 restores single-conversation behaviour
  • 256K context ceiling (QWEN36_MAX_SEQ), bf4 experts, bf8 KV. KV is paged, one pool shared by the conversation slots, so this costs the same memory as a single flat cache

Reasoning ("thinking") blocks are parsed out via --reasoning-parser qwen3, and tool calls via --tool-call-parser qwen3_coder.

Provenance

The exact sources the image was built from β€” code/ in this repo is byte-identical to the model code inside the image:

component built from
tt-metal d53c2a71fb0f1830077f1fea488fe3e85297f5a6
vLLM v0.26.0
vllm-tt-plugin bef89e429e202caa38d3e3c8a24da4e8cc02405a
code/ digest 62042a7925bfbf60 (sha256, first 16 hex digits)
built 2026-09-09T00:31:23+00:00 by tt-model 0.1.0
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support