qwen3.8-27b on Tenstorrent p150x4

Serves Qwen/Qwen3.8-27B by the Qwen team at revision 1d4bf0f2. See the original model card for training and evaluation details.

Qwen3.8-27B, a 27B-parameter dense hybrid model (Gated DeltaNet and attention layers), served with vLLM for chat, reasoning and tool calling on text.

Runs on Tenstorrent p150x4 (mesh P150x4) — 131,584-token context, up to 1 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150x4
Context	131,584 tokens
License	apache-2.0

Intended use

Chat, reasoning, coding and tool calling through an OpenAI-compatible API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150x4

Quickstart

tt serve tt-hous/qwen3.8-27b

tt serve downloads the Docker image and the Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/qwen3.8-27b.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "Qwen/Qwen3.8-27B" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser qwen3_coder): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Reasoning output is separated (--reasoning_parser qwen3): the thinking text arrives in reasoning, apart from content.

The model card's sampling for non-thinking mode is temperature 0.7, top_p 0.8, top_k 20.

Expected performance

eval	this package	reference	prompts	notes
GPQA Diamond, exact match, thinking off, 4,096-token output cap	65.0%	89.2%	40 of 198	14 of 40 responses hit the cap; standard error 7.6
LiveCodeBench v6, thinking off, 4,096-token output cap	48.0%	90.3%	50 of 175 (2025 Jan to Apr problems)	22 of 50 responses hit the cap; reference: the full v6 set

Evals ran on 2026 Sep 29 and 30 against this package's image with lm-evaluation-harness through the chat API: thinking off, temperature 0.7, top_p 0.8, top_k 20, no presence penalty, one request at a time, on the subsets and output caps in the table. The reference column is the figure on the Qwen model card, which states no recipe, subset or output cap for these rows.

profile	ISL	OSL	concurrency	N	TTFT (ms)	prefill tok/s/u	decode tok/s/u	E2EL (s)
default	128	128	1	1	144	888	33.3	3.96
default	1,024	256	1	1	227	4,515	33.2	7.91
default	4,096	256	1	1	690	5,937	33.0	8.41
default	16,384	256	1	1	2,806	5,838	32.5	10.65
default	32,768	256	1	1	5,865	5,587	32.3	13.75
default	65,536	256	1	1	12,903	5,079	31.6	20.97
default	131,072	256	1	1	30,395	4,312	30.3	38.80

Means of 4 requests per row up to 4,096 input tokens, 2 at 16,384 and 1 above, measured on 2026 Sep 10 on four P150 chips against this package's image: random-token prompts of exactly ISL tokens sent to /v1/completions and streamed, output pinned to OSL tokens with ignore_eos, greedy. Prefill tok/s/u is ISL divided by TTFT.

Limitations

• One request at a time: concurrent requests queue. Tested on four P150 chips only.
• Image and video input were not tested.
• Prefix caching is off, so every turn of a multi-turn chat re-prefills the whole prompt: a 131,072-token prompt takes about 30 s to first token.
• Not for long reasoning under small output budgets: with a 4,096-token cap, 14 of 40 GPQA Diamond and 22 of 50 LiveCodeBench responses ran out of budget before answering.
• Accuracy was measured on subsets (40 GPQA Diamond and 50 LiveCodeBench prompts) with thinking off only; thinking mode was not evaluated.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/qwen3.8-27b/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published (dirty tree — the image includes uncommitted changes)
vLLM	v0.24.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	96b494af6e2fb8f2 (sha256, first 16 hex digits)
built	2026-09-01T21:55:25+00:00 by tt-model 0.1.0