Dipankar Sarkar PRO

Your temp-0-vs-sampling split is the cleanest diagnostic in this whole thread.

It separates two failures that look identical from the outside.

One: the determinism trap. Greedy makes the loop a fixed point, re-entered token for token. Sampling breaks it, which is your 12 to 6.

Two: state collapse. At IQ1_M sampling cannot pull it out, because the model has lost the memory that it already ruled this branch out. A random escape just re-converges.

The tell that separates them is lexical. Exact n-gram repeat means determinism, and temperature or a repetition penalty fixes it. Same dead reasoning reworded means the working memory is gone, and nothing short of more bits helps.

So I would split loop rate into those two buckets rather than log it as one number. The ratio is the real bit-depth signal: exact-repeat share should fall and reworded-repeat share should rise as you drop toward 1.7-bit.

The 6 of 25 that survive sampling at 4-bit are already sitting in your prediction files. Do they repeat the exact tokens, or reword the same dead path?

The headline number hides the interesting part.

78.6% at 31.7M matching a 3B is real, but look at the one it missed: (21+27)(14-7). Both sub-expressions are easy, the model gets 48 and 7, then the final 487 is where it breaks. That is not an arithmetic gap, it is a magnitude gap: a two-digit times one-digit crossing into three.

So the number I would want from ArithMark-2 is accuracy stratified by the digit-length of intermediate results, not one aggregate. A 30M model can memorize the small-magnitude table and still fall off a cliff exactly where carries compound.

Does the score hold if you filter to problems whose intermediates all exceed two digits?

Reasoning breaking before retrieval is the tell.

Circular loops at short context are not a memory failure, they are a decisiveness failure. At IQ1_M the logit distribution flattens, so the model never commits hard enough to escape a branch it already tried. Chat survives that because any fluent continuation passes. Reasoning does not, because it has to remember it ruled a path out.

The number I would log next to RULER is loop rate: fraction of reasoning traces that repeat an n-gram window before terminating. If that climbs with bit depth faster than accuracy falls, you have isolated exactly what 1-bit breaks.

Does the collapse survive greedy decoding, or does temp 0 pull it back out?

The physics constraint is fair, and it points at a cheaper test than the one I asked for.

You cannot load f16, but Q3 and Q4 fit. Compute the imatrix from Q2_K, Q3, and Q4, then check the rank correlation of per-channel importance across them. If Spearman stays high as bit-depth rises, the salience ranking is probably stable down to f16 too, and IQ1_M is safe. If it drifts, you have found the exact failure mode without ever holding the full weights.

RULER over vanilla NIAH is the right call. On a YaRN-baked 1M model the number I would watch is where accuracy breaks relative to the baked length, since the extension usually rots before the nominal limit.

Where do you expect IQ1_M to give out first, the retrieval or the reasoning?

That is the most honest answer I have gotten to that question. 469 vs 489 and 1.07x MACs settles today's version: dense-cost curiosity, not a cheaper mixer.

So the sparsified readback is the whole experiment now. The number I would watch is loss at fixed fan-out: pin active channels near d_model/4 and read off the perplexity you pay. Flat means the growth prior is load-bearing. A cliff means the density was.

Are the tips already spatially clustered enough that a top-k readback is nearly free, or does the routing need its own learned gate?