clm-v0.1-8b on Tenstorrent p150 and p150x4

Serves Contrastive-LM/CLM-v0.1-8B by the Contrastive-LM project (Kwok et al.): two projection heads on a frozen Qwen/Qwen3-8B encoder at revision b968826d. See the original model card for training and evaluation details.

CLM-v0.1-8B is a System One verifier and ranker: a frozen Qwen3-8B encoder (36 layers, last-token pooling) with two 9.4M-parameter MLP projection heads (4096-1536-1536-512), a state head and an action head, trained with a bidirectional InfoNCE loss. It scores the candidates it is given and never generates text. This package serves its System One API (typed noul, choice and score questions, and free-form ranking) and an OpenAI-compatible embeddings endpoint through the vendored clm-serve. The weights pointer is the Qwen3-8B encoder; the head checkpoint (CLM_v0.1-8B.pt) ships inside the image.

Runs on Tenstorrent p150 or p150x4 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150, p150x4
License	apache-2.0

Intended use

Zero-shot action selection, best-of-N ranking and typed decisions over text states, and Qwen3-8B text embeddings, through the System One API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150 or p150x4

Quickstart

tt serve tt-hous/clm-v0.1-8b

tt serve downloads the Docker image and the Qwen/Qwen3-8B at b968826d9c46dd6066d109eabc6255188de91218 weights (into your HF cache; they are not in the image), then starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/clm-v0.1-8b.

The playground UI is served at / and the T-Rex demo at /demo on the port your serve command reported. The clm Python client from https://github.com/Contrastive-LM/CLM works unchanged with CLM_BASE_URL=http://<host>:<port>.

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	description
p150 (default)	p150	P150	One Blackhole p150 chip, accuracy_lofi_mlp policy (bf16 attention weights with HiFi4 attention math, bfp8 MLP weights with LoFi math) plus the measured program-config overrides (QKV block shape at 128 tokens, 11x10 MinimalMatmul grid above 128 tokens, block-sharded RMSNorm up to 512 rows). Selected by the datatype sweep: passes every gate (98.9 percent decision agreement with the fp32 reference on confident decisions) with the lowest served-workload latency.
p150-accuracy	p150	P150	One Blackhole p150 chip, the stock accuracy policy (bf16 attention, HiFi4 attention math, bfp8 MLP with HiFi2 math) on the stock program configs; the previous default. Passes every gate (98.9 percent confident-decision agreement in the standalone sweep); about 9 percent slower than p150 at 128 tokens batch 1 and 19 to 22 percent slower in the other measured cells (host bench, both profiles).
p150-fast	p150	P150	One Blackhole p150 chip, the stock bfp8 default policy (bfp8 attention weights, HiFi2 linears) on the stock program configs. Fastest at 128 tokens; fails the port's decision-agreement gate (97.3 percent of confident decisions agree with the fp32 reference in the standalone sweep, 95.7 percent measured from a served package).
p150x4	p150x4	P150x4	Four Blackhole p150-class chips as a 1x4 tensor-parallel mesh (validated on two p300c boards); about 1.9x lower latency at 128 tokens than one chip; accuracy_lofi_mlp policy (99.5 percent confident-decision agreement on this mesh); the single-chip program-config overrides do not apply.

Capabilities

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes for the payload it expects.

POST /v1/systemone takes {"state": ..., "questions": {id: {"type": "noul"|"choice"|"score", "instructions": ..., "criteria": ...}}, "temperature": 1.0} and returns a probability distribution per question. POST /v1/rank takes {"context", "question", "answers": [...]} and returns the candidates ranked with probabilities. POST /v1/embeddings is OpenAI-compatible (model qwen3-8b; 4096-dimensional, L2-normalised, last-token pooled Qwen3-8B embeddings). Every response carries an X-CLM-Latency-Ms header.

The T-Rex demo at /demo runs the CLM repository's game headless inside the container at 60 FPS; the model chooses jump, duck or run through this server's own POST /v1/systemone, and the page streams the game and every decision (the three option texts with probabilities, the executed action, shield interventions, round trip and X-CLM-Latency-Ms) over a WebSocket at /demo/ws. The table at the end of a run has the same fields as the T-Rex rows under Expected performance.

Expected performance

Evaluations

eval	profile	this package	reference	items
LocalLLaMA/typed-decisions, zero-shot accuracy	p150 (default)	0.345	—	400 cases, 2,000 decisions
LocalLLaMA/typed-decisions, zero-shot accuracy	p150-accuracy	0.364	—	400 cases, 2,000 decisions
typed-decisions, KL divergence	p150	2.04	—	2,000 decisions
typed-decisions, Brier	p150	0.63	—	2,000 decisions
T-Rex game, courses survived	p150	3 of 5	5 of 5 (CLM GitHub README, RTX 4090)	5 courses
T-Rex game, decisions per course	p150	2,195	3,342 (CLM GitHub README, RTX 4090)	5 courses

The CLM model card publishes no typed-decisions figure (it reports fine-tuned verifier results, not this checkpoint zero-shot); the T-Rex reference figures are the authors' in the CLM GitHub README.

Parity against an fp32 CPU run of the same weights:

parity	profile	this package	items
embedding cosine to the fp32 reference, mean / min	p150	0.9992 / 0.9960	308 texts (host run of the same encoder code)
identical argmax, typed-decisions subset (served)	p150	96.0 percent	200 decisions
identical argmax where the reference top-2 margin is above 0.10 (served)	p150	98.4 percent	188 decisions
typed-decisions accuracy, this package / fp32 reference	p150	0.355 / 0.370	40-case subset
confident-decision agreement with the fp32 reference	p150	98.9 percent	standalone sweep
confident-decision agreement with the fp32 reference	p150-accuracy	98.9 percent	standalone sweep
confident-decision agreement with the fp32 reference	p150-fast	95.7 percent served (97.3 standalone)	200 decisions served
confident-decision agreement with the fp32 reference	p150x4	99.5 percent	standalone sweep

Latency

p50, client on the same host. The reference figures are the authors' on one RTX 4090 in the CLM GitHub README: 58.1 ms for a new state against cached option vectors (38 tokens embedded), and the vector-cache table's 28.0 ms for a new state and 0.6 ms for a cached one.

profile	request	N	median (ms)	p95 (ms)	reference (RTX 4090)
p150 (default)	README customer example: 3 questions, 98 tokens embedded, cold cache	—	220	—	—
p150	same, warm (all vectors cached)	—	0.1	—	0.6
p150	new state against a fixed action set, server-side	—	56	—	58.1 (28.0 in the vector-cache table)
p150	typed-decisions case, 5 questions	—	258	—	—
p150	embeddings, one 128-token text	—	56.6	—	—
p150	embeddings, one 512-token text	—	83	—	—
p150	embeddings, one 1,024-token text	—	149	—	—
p150	embeddings, one 2,048-token text	—	272	—	—
p150	embeddings, eight 128-token texts	—	149	—	—
p150	embeddings, eight 512-token texts	—	530	—	—
p150-accuracy	README customer example, cold cache	—	263	—	—
p150-accuracy	new state against a fixed action set	—	60	—	58.1
p150-fast	README customer example, cold cache	—	234	—	—
p150-fast	new state against a fixed action set	—	56	—	58.1
p150x4	README customer example, cold cache	—	169	—	—
p150x4	new state against a fixed action set	—	34	—	58.1

The embeddings endpoint reaches 7.7k tokens/s at batch 32 on p150. p150-accuracy is about 9 percent slower than p150 at 128 tokens, batch 1, and 19 to 22 percent slower in the other measured cells; p150x4 is about 1.9 times faster than one chip at 128 tokens.

Measured from this package served on one Blackhole p150 chip (one chip of a p300c board) for the p150* profiles and on four chips across two p300c boards for p150x4, client on the same host, p50 latencies; the T-Rex rows are 5 courses of the demo game.

Limitations

• Encoder-only: it produces scores and embeddings, never text. Not for text generation. Not for zero-shot typed decisions where accuracy matters: zero-shot accuracy on typed-decisions (0.345) is below the dataset's prior baseline on both CPU and this package, and the CLM authors treat that benchmark as a fine-tuning target.
• Tested on one p150 chip (of a p300c board) and on a four-chip p150x4 mesh (two p300c boards). No two-chip profile is offered.
• Inputs longer than 2,048 tokens are truncated to their last 2,048 tokens (the upstream clm-serve default); the encoder itself supports 40,960.
• Identical requests give identical outputs, but a text embedded alongside different batch-mates can differ at the cosine 0.999 level: between single-text and batched embeddings of the same 200 typed-decisions decisions, 4 to 5 argmax answers change, all near ties. Against the fp32 CPU reference the default profile reverses 8 of those 200 decisions, 5 of them where the reference's own top-2 margin is below 0.10 (3 of the 188 confident ones).
• The example probabilities in the CLM README (0.410 / 0.939 / 1.984) are not what this package returns: the default profile gives 0.844 / 0.991 / 2.000 on the single-text path and 0.875 / 0.986 / 2.000 when the request is embedded as one batch; the CPU fp32 reference of the same weights gives 0.842 / 0.988 / 2.000.
• p150-fast agrees with the fp32 reference on 95.7 percent of confident decisions as served (default profile: 98.4 percent); choose it only where latency matters more than agreement.
• Single-text latency is about twice an RTX 4090 (56 ms against 28.0 to 58.1 ms for a new state); latency-sensitive loops such as the T-Rex game lose decisions (3 of 5 courses survived, 2,195 decisions per course, against the authors' 5 of 5 and 3,342).
• One model process per container; concurrent requests are serialised on the device.

Licensing

Qwen3-8B weights under Apache-2.0; CLM-v0.1-8B head weights under Apache-2.0 (Contrastive-LM); the vendored clm serving code under Apache-2.0; the TTNN port code under Apache-2.0.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/clm-v0.1-8b/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	2641f871d7c031866dba01db8eb52946f2772861
code/ digest	949aa72e5600889b (sha256, first 16 hex digits)
built	2026-10-02T18:37:16+00:00 by tt-model 0.1.0