litert-community/kitten-tts-nano-0.8

LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

KittenTTS nano 0.8 — LiteRT, dynamic length + streaming

KittenTTS nano (15M params, StyleTTS2 + ISTFTNet + mini-ALBERT, 8 voices, 24 kHz) converted to LiteRT CPU/XNNPACK graphs with a dynamic sequence length — any sentence length runs on the same graphs, no padding buckets. Built for small-CPU targets (Raspberry Pi class): RTF 0.187 measured on a Raspberry Pi 5 (~1.6× faster than the ONNX Runtime baseline on this device class).

Listen (golden sentence, same inputs): samples/litert_golden.wav — this port (fp32, deterministic) · samples/ref_onnx.wav — the official ONNX (stochastic reference).

Upstream ships ONNX only (Apache-2.0). This port re-authors the model in TF/Keras from the ONNX weights and converts with the official TFLiteConverter, which emits fused dynamic-length TFLite LSTM kernels for the five BiLSTMs — the piece that torch-path conversions cannot keep dynamic (see "Why TF re-authoring" below).

Graphs

Graph	Inputs	Outputs	fp32	fp16
kitten_predictor.tflite	input_ids [1,N] int32, style [1,256], speed [1]	d [1,N,256], t_en [1,N,128], durations [N] int32	33.8 MB	17.0 MB
kitten_prosody.tflite	en [1,T,256], style [1,256]	f0 [1,2T], n [1,2T], har [1,120T+1,22]	3.3 MB	1.7 MB
kitten_vocoder.tflite	asr [1,T,128], f0, n, har, style	wav [1,600T] @ 24 kHz	26.4 MB	13.4 MB

Host glue between graphs is ~10 lines of numpy: en = repeat(d, durations), asr = repeat(t_en, durations) (equivalent to the in-graph Loop alignment of the ONNX — verified bit-exact), then slice har/f0/n per vocoder call. style comes from voices.npz exactly as in the pip package (voices[voice][min(len(text), 399)]).

Measured on a real Raspberry Pi 5 (2026-08-06)

Pi 5 8 GB, Raspberry Pi OS 64-bit, Python 3.13.5, ai-edge-litert 2.1.6, 4 threads. vcgencmd get_throttled = 0x0 before/after each run (no undervoltage/throttling during measurement).

Sentence	N tokens	audio	predictor	prosody	vocoder	sentence	RTF
[0]	41	2.77 s	59.9 ms	20.2 ms	403.8 ms	483.8 ms	0.174
[1]	63	4.08 s	87.0 ms	30.1 ms	610.5 ms	727.6 ms	0.179
[2]	138	7.03 s	188.8 ms	53.2 ms	1141.7 ms	1383.7 ms	0.197

Overall RTF 0.187 (fp32; durations bit-OK, log-spec corr ≥ 0.9954). fp16 is speed-identical on the Pi (XNNPACK unpacks fp16 weights to fp32 compute) with slightly lower corr — deploy fp32. For reference, the same model on ONNX Runtime was measured at RTF 0.30 on this device class, so the LiteRT path is ~1.6× faster.

GPU (v3dv WebGPU) status — measured on the Pi 5, 2026-08-06

With Mesa built from git (v3dv Vulkan 1.3, driver 26.2.99, V3D_WEBGPU_OVERRIDE=1): the static-chunk vocoder (kitten_vocoder_static80.tflite) compiles and runs fully accelerated (is_fully_accelerated=True) with output corr 0.9997 vs CPU (fp16-class divergence). The dynamic vocoder does not compile (the GPU delegate requires static shapes), and the fused-LSTM predictor/prosody graphs are CPU-only. The CPU is faster on this board; the all-CPU config above is the recommended deployment.

Verification (vs. the official ONNX, same inputs)

The reference model is stochastic (SineGen draws a random initial harmonic phase and additive noise every run), so the fair bar is the ONNX's own run-to-run variability. All numbers on the golden sentence, Mac M-series CPU:

Comparison	log-mel corr	spec-conv
ONNX vs ONNX (two runs, same inputs)	0.98327	0.0949
LiteRT fp32 vs ONNX (deterministic)	0.98388	0.1242
LiteRT fp16 vs ONNX (deterministic)	0.98231	—

i.e. the port sits inside the model's intrinsic noise floor. Predicted durations are bit-identical to the ONNX output. The decoder/vocoder chain is float-exact in isolation (corr 1.000000 against the deterministic reference when fed the reference harmonics). int8 dynamic-range quantization was tried and rejected (log-mel corr 0.913, durations change).

Raw-waveform correlation is not a meaningful metric here: sub-0.5 % f0 differences de-correlate the waveform via accumulated sine phase while being inaudible and spectrally identical.

Speed (Mac M-series, 4 threads, XNNPACK)

Sentence	N tokens	frames	audio	predictor	prosody	vocoder	RTF
golden	85	182	4.55 s	16 ms	8 ms	52 ms	0.017
short	27	80	2.00 s	7 ms	4 ms	25 ms	0.018
long	112	195	4.88 s	19 ms	8 ms	53 ms	0.016

In a python:3.12-slim linux/arm64 container (same aarch64 ai-edge-litert 2.1.6 wheel the Pi uses): durations still bit-identical, log-spec corr ≥ 0.9955.

Snapdragon NPU (Hexagon)

• kitten_predictor.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: LiteRtException: Failed to load model from file.
• kitten_predictor_fp16.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: LiteRtException: Failed to load model from file.
• kitten_prosody.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: LiteRtException: Failed to load model from file.
• kitten_prosody_fp16.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: LiteRtException: Failed to load model from file.
• kitten_vocoder.tflite — neither accelerator produced a usable row on the S26. NPU: LiteRtException: ERROR: [./third_party/odml/litert/litert/cc/litert_compiled_model.h:2070]. GPU: LiteRtException: Failed to compile model.
• kitten_vocoder_fp16.tflite — neither accelerator produced a usable row on the S26. NPU: LiteRtException: ERROR: [./third_party/odml/litert/litert/cc/litert_compiled_model.h:2070]. GPU: LiteRtException: Failed to compile model.
• kitten_vocoder_static80.tflite — the GPU is faster: 11.32 ms against 75.92 ms on the NPU, a factor of 6.70. The NPU still loads 11.42x faster (175 ms against 1997 ms).

file	backend	compiled	inference (median / min)	load
kitten_vocoder_static80.tflite	NPU (Hexagon v81)	on-device JIT	75.92 ms / 75.22 ms	175 ms
kitten_vocoder_static80.tflite	GPU (Adreno)	—	11.32 ms / 10.43 ms	1997 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.77–0.78, where 1.0 is the throttling threshold.

The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. That first compile took 30 s here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.

GPU wiring: GPU guide.

Quickstart

hf download litert-community/kitten-tts-nano-0.8 --local-dir kitten-tts-litert
cd kitten-tts-litert

One-command benchmark (no espeak needed on the device — inputs are pre-tokenized in bench_inputs.npz):

pip install numpy ai-edge-litert
python bench.py --models-dir .                # fp32, 4 threads
python bench.py --models-dir . --precision fp16 --write-wavs

Reports per-graph latency, per-sentence synthesis latency, RTF, and an output identity check (durations bit-compare + log-spectrogram correlation vs the bundled reference).

Drop-in synthesis (Piper replacement; frontend tokenization verified byte-identical to the official pip package):

pip install numpy ai-edge-litert phonemizer espeakng-loader
python say.py "Hello! How can I help you today?" --models-dir . -o hello.wav

from say import KittenTTS

tts = KittenTTS(models_dir=".", voice="Jasper")
for sentence, pcm in tts.stream(text):   # float32 @ 24 kHz per sentence
    play(pcm)

G2P options: the phonemizer library (exact reference tokenization; espeak-ng GPL-3.0 in-process — same situation as Piper) or --g2p cli to call the espeak-ng binary as a subprocess (GPL isolation).

Deployment note — LSTM state: the fused TFLite LSTM kernels keep their hidden state in variable tensors that persist across invoke(). Call interpreter.reset_all_variables() (after allocate_tensors()) before every utterance, or the second synthesis on a reused interpreter is corrupted. bench.py/say.py already do this.

Streaming

• Sentence-level (exact, recommended) — synthesize per sentence and play while the next sentence synthesizes; this is the same granularity the official pip package uses (chunk_text). First-audio latency = one short sentence (≈0.4–0.7 s on the Pi 5).
• Intra-sentence chunked vocoder (approximate) — the vocoder alone can be run on overlapping frame chunks (~1 s chunks, 20-frame overlap). It is not exact because StyleTTS2's AdaIN InstanceNorms take statistics over the whole utterance: chunked output measures log-mel corr 0.970 against the full decode. Use sentence-level unless latency demands force this mode.

Files

File	Purpose
kitten_{predictor,prosody,vocoder}.tflite	fp32 graphs (recommended)
kitten_{predictor,prosody,vocoder}_fp16.tflite	fp16-weight variants (half size; same speed on XNNPACK)
kitten_vocoder_static80.tflite	static 80-frame vocoder chunk (GPU-delegate experiment; CPU deployment recommended)
voices.npz	the 8 voice style vectors, same lookup as the pip package
bench.py + bench_inputs.npz	one-command device benchmark (numpy + ai-edge-litert only)
make_bench_inputs.py	regenerate bench inputs (needs espeak on the host)
say.py	drop-in say(text) / stream(text) synthesis module + CLI
samples/	output samples: this port vs the official ONNX, same inputs

Why TF re-authoring (and not litert-torch / onnx2tf)

• litert-torch (0.9.2) cannot export a dynamic-length LSTM: torch.export's LSTM decomposition specializes the time axis. Worse, even conv-only graphs exported with a dynamic axis bake the example length into internal RESHAPEs, so the resulting model only runs at the trace length. Dynamic shapes effectively require the TF converter path today.
• The TF path also dodges a constant-dedup hazard that corrupts the ISTFTNet cos/sin inverse-DFT ConvTranspose pair on the torch path — here the iSTFT converts cleanly in-graph.
• onnx2tf was not used (accuracy hazards on attention models).

Conversion notes that generalize: ONNX exporters value-deduplicate identical initializers and CSE-merge whole InstanceNorm nodes (two AdaINs normalizing the same tensor shared one node here) — map norm parameters by graph connectivity, not by module name. tf.nn.leaky_relu defaults to α=0.2 while torch defaults to 0.01. SineGen's rad % 1 matters because unvoiced frames carry small negative f0. TFLite has no ATAN builtin — atan2 is the supported route to the harmonic STFT phase.

Text frontend / licensing

The model consumes espeak-ng IPA phoneme IDs (same 178-symbol table as Kokoro/StyleTTS2). The pip package phonemizes with espeak-ng (GPL-3.0) — fine as a separate process, or swap in an Apache-licensed neural G2P such as litert-community/Kokoro-G2P-en-US (same symbol table) for a GPL-free stack.

Model weights: Apache-2.0 (KittenML). Deterministic SineGen: the random initial phase / noise of the reference are fixed to zero (the reference itself produces a different waveform every run; zeroing selects one deterministic sample).