arashakb/FASTWAM-Unitree-G1

FastWAM Unitree-G1 two-task — W4A4 real packed INT4 at group 32, two arms

The Unitree-G1 two-task FastWAM fine-tune (LGG100/fastwam-sc-10k, step 10000) quantised to INT4, group 32 for GPUs with 4-bit tensor cores, by two methods calibrated identically and packed identically — so they differ in their mathematics and not in their plumbing.

Real quantisation, not a simulation of it. The weight ships PACKED at 4 bits, two output channels per byte, in the [K, N/2] layout the fused Triton kernels contract on. A fake quantiser stores 4-bit values in a 16-bit tensor and runs an ordinary fp16 GEMM: right for measuring what 4 bits cost, wrong for deploying, because it reads 2 bytes per weight and performs no integer arithmetic.

	sc_step10000_asp_w4a4_g32.pt	sc_step10000_svdquant_w4a4_g32.pt
method	AFQ / ASP, deflated form	SVDQuant (Li et al., ICLR 2025)
rotation	block Hadamard	none
low-rank branch	rank 32 on the action expert, from the action metric	rank 32 on every layer, from the quantisation residual
BPW	4.6117	4.8298
file	3.38 GiB	3.53 GiB
vs bf16, mean	0.0016	0.0055
vs bf16, worst frame	0.0031	0.0140
vs recorded teleoperation	0.0910	0.0920

bf16's own distance to the demonstrator is 0.0910, so ASP matches the unquantised model to four decimals while spending 0.218 fewer bits per weight. It moves the action 3.47x less than SVDQuant.

Both quantise the same 600 block Linears, 5.914 B parameters, against the bf16 checkpoint's 11.2 GiB.

The two contracts

SVDQuant — s is the SmoothQuant migration factor s_j = act_absmax_j^alpha / w_absmax_j^beta; L2 L1 is the rank-32 fp16 branch absorbing the weight outliers. Both paths see the FULL smoothed activation: nothing is deflated, nothing is rotated.

y = Q4(x/s) Q4(R)  +  ((x/s) L1^T) L2^T  +  bias,          R = (W*s) - L2 L1

Deflated ASP — the activation is rotated by a block Hadamard, and the protected subspace is removed from the 4-bit path on both sides. V is the top-32 eigenbasis of G~ = H diag(s) G diag(s) H, the action metric in the coordinates actually being quantised.

y = Q4((I - VV^T) x~) Q4(W~(I - Pi))  +  (x~ V)(W~V)^T  +  bias,
x~ = (x/s)H,   W~ = (W*s)H,   Pi = V V^T

They produce the same shapes, so nothing but the recorded lowrank_mode distinguishes them. Run either checkpoint through the other's class and you get a well-formed GEMM of the wrong bilinear form: it loads, it runs, and it emits a plausible action chunk. install_svdquant dispatches on the mode and cross-checks requires_runtime_fwht against what the layers carry.

Where the action metric points

G_l = E[J_l^T J_l], J_l = d(action chunk)/d(x_l) differentiated through all ten denoising steps, estimated with random probes over 905 frames x 12 probes of the calibration set. The mass concentrates in cross_attn.o — 13.8% in block 0 alone — and the top-32 eigenvectors capture 98.75% of tr(G) on the dominant layer. The joint (alpha, beta, V) search buys 1.387x over no subspace at all; SVDQuant's residual-SVD branch is a different object entirely and is fitted to the weight rather than to the action.

Calibration

30 episodes: 15 per task from item-classification-new and sort-tools-new, seed 42, ALL frames — 14,469 observations. Activation absmax feeds SmoothQuant; a 1024-row-per-layer reservoir of real layer inputs feeds the smoothing search, whose objective is SVDQuant's own OutputsError: each layer's output MSE against bf16, with the INT4 quantiser and the refitted low-rank branch inside the scored loop.

SVDQuant's families: 419 layers (alpha, 0), 178 (alpha, 1-alpha), 3 none; split-half agreement 56.0%, and on held-out rows the search removes 3.9% of the error a fixed alpha = 0.5 leaves (the in-sample figure is 5.0%; the held-out one is the honest one). ASP's joint search: split-half 58.0% on the action expert, 45.7% on the video expert.

Verification

Three checks, because they fail in different ways.

1. Does the packing compute the right bilinear form? Per layer, on random inputs, against the fake-quant formula written out directly. For SVDQuant: relative error 1.1e-2, 0.078% of codes differ by exactly 1 LSB, scales bit-identical; repack fidelity out of the exporter was cleaner still, 0.0000% of codes off. This is the decisive test — a wrong nibble pairing or a transposed scale shows up here and nowhere else.

For ASP the criterion had to change, and the reason is worth stating. Bit-identical scales are achievable for SVDQuant, whose prologue is an elementwise multiply torch reproduces exactly, and are not achievable for ASP, whose prologue is a tensor-core Hadamard: the rotated activation rounds differently, so its per-group amax does too. The honest criterion is distance to the unquantised layer, and ASP's kernel is at worst +0.23% further from the true bf16 layer than a fp32 reference is — on layers where both sit at ~1.4e-1, because that is W4A4's own error. A structurally wrong kernel would be far further, not 0.23%. Codes are still within 1 LSB and scales within 5%.

2. How far did quantisation move the action? On 32 observations from 8 episodes disjoint from the calibration selection: 0.0016 for ASP, 0.0055 for SVDQuant.

3. Is it still tracking the recorded actions?

arm	vs recorded teleoperation
bf16	0.0910
ASP deflated	0.0910
SVDQuant	0.0920

No closed-loop success rate has been measured. Open-loop agreement is necessary and not sufficient.

Running it

Needs triton and a GPU with INT4-on-INT8 tensor cores (built and verified on sm_89). The file is self-contained — the 600 quantised Linears, every unquantised mot tensor and the proprio encoder — so the 11.2 GiB bf16 checkpoint is not needed at inference. You still need the Wan2.2 VAE to encode the camera image, dataset_stats.json for proprio z-scoring and action denormalisation, and the pre-encoded prompt.

from fastwam_w4a4_runtime import load_quantized_model
model, cfg = load_quantized_model("sc_step10000_asp_w4a4_g32.pt", build_model=my_builder)

install_svdquant raises unless every name in the checkpoint resolves to an nn.Linear: a partial swap still runs and still emits actions, and the numbers it produces belong to no scheme at all.

fastwam_w4a4_runtime.py and w4a4_triton.py here are standalone copies, verified bit-identical to the in-repo versions on real exported layers of both contracts.

Four things that will bite you

1. This robot is 16-D, not the 14-D of the UR3 FastWAM checkpoints. configs/data/robotwin.yaml fixes action_output_dim and proprio_output_dim at 14 and the model config reads them through interpolation in three places (proprio_dim, action_dit_config.action_dim, video_dit_config.action_dim). Instantiate it unchanged and you build the wrong widths.
2. Both datasets declare the same instruction. Every episode of item-classification-new AND of sort-tools-new carries "pick up cube." in its own metadata, despite the repo names. That is what conditions the model; the repo names are not an instruction source it ever saw. There is exactly one distinct prompt, and it is pre-encoded here.
3. The mosaic is 384x320: cam_left_high resized to 320x256 on top, cam_left_wrist and cam_right_wrist at 160x128 side by side underneath, order fixed, pixels mapped to (-1, 1).
4. Use dataset_stats.json from this repo, not statistics pooled from the datasets. Training used 190 of the 200 episodes; pooling all of them reproduces a published stats file only to 0.2-0.5% on the mean and 1.5-2.6% on the std, and a few percent of error in a z-score mean is a silent uniform bias on every observation the model ever sees.

Tasks

Both recorded on Unitree_G1_Dex1_Sim, 100 episodes each, 30 fps, three cameras at 480x640.

• item-classification-new — 66,933 frames
• sort-tools-new — 32,746 frames