--- license: other license_name: mit-and-apache-2.0 license_link: https://huggingface.co/leuconoe/litert-lm-unity-quantized#known-caveats-disclose-in-model-cards tags: - litert - tflite - litertlm - whisper - qwen2.5 - asr - text-generation - quantized - int8 - int4 base_model: - openai/whisper-tiny - openai/whisper-base - openai/whisper-medium - openai/whisper-large-v3 - openai/whisper-large-v3-turbo - Qwen/Qwen2.5-0.5B-Instruct - Qwen/Qwen2.5-1.5B-Instruct --- # Community Release — Project-Quantized LiteRT Models (2026-07-23) Quantized ASR/LLM artifacts produced by the LiteRT-LM-Unity project, prepared for upstream contribution (e.g., litert-community on Hugging Face). All files were validated before inclusion; validation data below. ## Toolchain / provenance - Quantizer: `ai-edge-quantizer` 0.8.0 (post-training, dynamic-range). - Recipes: - **i8** = `recipe.dynamic_wi8_afp32()` (int8 weights channelwise, fp32 activations). - **i4** = `recipe.dynamic_wi4b64_afp32()` (int4 weights, blockwise-64, fp16 scales) **with i8 overrides on sensitive scopes** (encoder / embedding-logits table / selected projections). Pure full-scope int4 was tested and REJECTED for several models (Korean transcription errors); the mixed recipes below are what passed. - int4 channelwise (`wi4c`) and blockwise-32 (`wi4b32`) are not used in this collection: in our tests `wi4c` degraded accuracy sharply and `wi4b32` produced an immediate EOS. Do not regenerate with them. - Whisper medium / large-v3 / large-v3-turbo tflite graphs were exported by this project from the openai checkpoints (30 s fixed window, encode/decode split signatures matching the litert-community whisper-tiny/base graph interface; large-v3 family = 128 mel bins / 51866 vocab). f32 source exports (3–6 GB) are retained separately and available on request. - Qwen2.5 `.litertlm` files were produced by unpack → quantize → repack: `litert-lm-peek` dump → `dynamic_wi4b64_afp32` on the f32 tflite section → rebuild with `litert-lm-builder` 0.14.0 (tokenizer + LlmMetadata mirrored from the official q8 release). Source: official litert-community f32 releases. - Runtime compatibility: `.litertlm` format 1.5.0; validated on LiteRT-LM v0.14.0 (Windows x86_64 CPU/GPU and Android arm64, Snapdragon 865-class device). ## Files and validation > **Accuracy-claim scope**: "exact" below means the specific gate clips used during quantization validation (sentence-length Korean/English test audio), not general Korean capability. Per-tier CER across the full 10-clip matrix — including short voice commands, where small tiers degrade sharply — is published in the project repo's `docs/benchmarks/asr-model-matrix.md`. Validation clips: Korean "2025년 3월 5일 전술평가 결과 보고" sentence + English equivalent + short Korean voice commands ("볼륨 업" etc.). CER computed against reference transcripts (punctuation-normalized). Full matrix: `docs/benchmarks/asr-model-matrix.md` in the LiteRT-LM-Unity repo. | File | Size | Base model | Recipe | Validation | | --- | ---: | --- | --- | --- | | `whisper-tiny/whisper_tiny_30s_i4.tflite` | 35 MB | openai/whisper-tiny (via litert-community f32) | wi4b64 + i8 encoder+embed table | en gate clips exact; **lower Korean accuracy — 3/9 exact, CER ko 0.281** (misses `볼륨 업`→`별념`, `음량 증가`→`능량 증가`). Not recommended for Korean; see the ASR matrix in the project repo | | `whisper-base/whisper_base_30s_i8.tflite` | 74 MB | openai/whisper-base (via litert-community f32) | wi8 | transcripts byte-identical to f32; ~1.6× faster | | `whisper-base/whisper_base_30s_i4.tflite` | 44 MB | same | wi4b64 (full scope OK for base) | ko/en gate clips exact (sentence-length audio); sub-1.2 s clips remain unreliable at this tier | | `whisper-medium/whisper_medium_30s_i8.tflite` | 794 MB | openai/whisper-medium (project export) | wi8 (DRQ) | 7/9 exact, CER ko 0.042 / en 0.000 | | `whisper-medium/whisper_medium_30s_i4.tflite` | 634 MB | same | wi4b64 mixed (L1) | same transcripts as i8 | | `whisper-large-v3/whisper_large_v3_30s_i8.tflite` | 1.6 GB | openai/whisper-large-v3 (project export, 128-mel) | wi8 (DRQ) | CER 0.000/0.000, 7/9 exact (spacing-only misses) | | `whisper-large-v3/whisper_large_v3_30s_i4.tflite` | 1.1 GB | same | wi4b64 mixed (mixD) | CER 0.000/0.000; ~20 % faster than i8 | | `whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i8.tflite` | 1.1 GB | openai/whisper-large-v3-turbo (project export, 128-mel) | wi8 | 5–6/9 exact, CER ko 0.000 | | `whisper-large-v3-turbo/whisper_large_v3_turbo_30s_i4.tflite` | 721 MB | same | wi4b64 mixed (mixD) | **best tier overall: 8/9 exact, CER 0.000/0.000**; device-validated (Snapdragon 865, shape-based decode binding) | | `qwen2.5-0.5b-instruct/Qwen2.5-0.5B-Instruct_wi4b64_ekv1280.litertlm` | 265 MB | litert-community/Qwen2.5-0.5B-Instruct f32 | wi4b64 full scope | inference-validated Windows + Android (35.5 tok/s device CPU, +38 % vs official q8) | | `qwen2.5-1.5b-instruct/Qwen2.5-1.5B-Instruct_wi4b64_ekv4096.litertlm` | 790 MB | litert-community/Qwen2.5-1.5B-Instruct f32 | wi4b64 full scope | inference-validated Windows CPU (59.7 prefill / 11.8 decode tok/s, Korean QA correct) | ## Related: ACFT short-window Whisper models Fixed 5 s/10 s/30 s short-window TFLite graphs (encode-signature-sized mel window; method: [futo-org/whisper-acft](https://github.com/futo-org/whisper-acft)) live in two dedicated repos: - **Pure ACFT** (TFLite conversions of the futo-org checkpoints, no retraining; tiny / base / small × multilingual / .en, dynamic-range int8): **[litert-community/whisper-acft](https://huggingface.co/litert-community/whisper-acft)**. - **Korean ACFT** (ACFT-KO, Korean-trained for on-device Korean voice commands; tiny / base / medium / large-v3-turbo, dynamic-range int8): **[leuconoe/whisper-acft-ko](https://huggingface.co/leuconoe/whisper-acft-ko)**. Training data zeroth-korean (OpenSLR SLR40, CC-BY-4.0) + google/fleurs en_us (CC-BY-4.0). ## Known caveats (disclose in model cards) - Whisper 30 s graphs; decoder is fixed-length re-run (no KV cache) matching the litert-community tiny/base interface. large-v3 family needs a 128-mel frontend and vocab 51866 (`<|transcribe|>`=50360, `<|notimestamps|>`=50364). - tiny/base i8 showed sub-1.2 s clip sensitivity on one Snapdragon 865 device (mel/STFT numerics, ~0.1 % energy delta vs desktop); turbo/large-v3 unaffected. - Qwen2.5 i4 litertlm metadata carries lowercase `model_type` (`tf_lite_prefill_decode`) vs uppercase in official files — loads fine on v0.14.0, noted for completeness. - Licenses: whisper weights MIT (OpenAI); Qwen2.5 Apache-2.0. Quantized derivatives inherit the base licenses.