jaichang/embeddinggemma2-korean-birds-620-litert

📱⚡ EmbeddingGemma 2 — Korean 620 Bird Species On-Device (LiteRT .tflite + WebGPU .onnx) & Binary Index

English | 한국어

---

🌍 English Overview

This repository provides the knowledge-distilled, ultra-compact on-device encoders (LiteRT .tflite and ONNX Runtime WebGPU .onnx) and precomputed Float32 binary vector databases (.f32) distilled from the full 740M teacher model (jaichang/embeddinggemma2-korean-birds-620).

Designed for 100% client-side, zero-server inference in Flutter Mobile (Android / iOS via tflite_flutter) and Flutter Web (WebGPU / WebGL / WASM via onnxruntime-web).

🧠 Knowledge Distillation Architecture & Accuracy

Instead of downloading the 1.2 GB model.safetensors onto a phone or browser, we trained lightweight student networks to directly regress the exact 768-dimensional L2-normalized embedding coordinates of EmbeddingGemma 2 (740M) using Cosine + InfoNCE Distillation Loss:

Student Encoder	Backbone Architecture	Input Tensor Shape	Output Tensor	File Size	Retrieval Accuracy (vs. Teacher Space)
embeddinggemma2_vision_768d.onnx	MobileNetV3-Small + 768-d L2 Projection (Distilled Student)	float32[1, 3, 160, 160] ([-1, 1] RGB)	float32[1, 768] (L2-norm)	9.24 MB	100.0% Top-1 (mean_cos = 0.8841)
embeddinggemma2_audio_768d.onnx	1D Multi-Scale Spectral ConvNet + 768-d L2 Projection (Distilled Student)	float32[1, 80000] (5s @ 16kHz)	float32[1, 768] (L2-norm)	14.70 MB	99.1% Top-1 (mean_cos = 0.9086)
embeddinggemma2_undistilled_vision_int8.{onnx,tflite}	Un-distilled Gemma4Vision Transformer Tower (model.safetensors INT8)	float32[1, 3, 160, 160] ([-1, 1] RGB)	float32[1, 768] (L2-norm)	22.38 MB	93.5% Top-1 (100% on field test set)
embeddinggemma2_undistilled_audio_int8.{onnx,tflite}	Un-distilled Gemma4Audio Conformer Tower (model.safetensors INT8)	float32[1, 80000] (5s @ 16kHz)	float32[1, 768] (L2-norm)	30.90 MB	100.0% Top-1 on field audio test set

📦 File Inventory

Filename	Target Platform / Runtime	Purpose & Size
embeddinggemma2_vision_768d.onnx	Flutter Web (WebGPU $\rightarrow$ WebGL $\rightarrow$ WASM) & ONNX Runtime	Knowledge-distilled 160x160 RGB bird photo $\rightarrow$ 768-d embedding encoder (9.24 MB)
embeddinggemma2_audio_768d.onnx	Flutter Web (WebGPU $\rightarrow$ WebGL $\rightarrow$ WASM) & ONNX Runtime	Knowledge-distilled 16kHz bird audio $\rightarrow$ 768-d embedding encoder (14.70 MB)
embeddinggemma2_undistilled_vision_int8.onnx / .tflite	ONNX Runtime (WASM SIMD) & LiteRT (tflite_flutter)	Un-distilled Fine-Tuned Gemma4Vision Tower INT8 (22.38 MB)
embeddinggemma2_undistilled_audio_int8.onnx / .tflite	ONNX Runtime (WASM SIMD) & LiteRT (tflite_flutter)	Un-distilled Fine-Tuned Gemma4Audio Conformer Tower INT8 (30.90 MB)
embeddinggemma2_vision_768d.tflite	Android / iOS (tflite_flutter) & LiteRT	Mobile LiteRT vision encoder (160x160 RGB $\rightarrow$ 768-d)
embeddinggemma2_audio_768d.tflite	Android / iOS (tflite_flutter) & LiteRT	Mobile LiteRT audio encoder (16kHz $\rightarrow$ 768-d)
korean_620_birds_audio_index.f32	Mobile & Web (Float32List)	Precomputed 768-d audio reference vectors for 620 species (1.8 MB)
korean_620_birds_photo_index.f32	Mobile & Web (Float32List)	Precomputed 768-d photo reference vectors for 620 species (1.8 MB)
korean_620_birds_catalog.json	Mobile & Web	Offline encyclopedia metadata (Korean/English/Scientific names, descriptions, image & audio URLs)

🚀 Quick Start (ONNX Runtime in Python / Web)

import json
import numpy as np
import onnxruntime as ort
from PIL import Image
from huggingface_hub import hf_hub_download

repo_id = "jaichang/embeddinggemma2-korean-birds-620-litert"
onnx_path = hf_hub_download(repo_id=repo_id, filename="embeddinggemma2_vision_768d.onnx")
f32_path = hf_hub_download(repo_id=repo_id, filename="korean_620_birds_photo_index.f32")
cat_path = hf_hub_download(repo_id=repo_id, filename="korean_620_birds_catalog.json")

# 1. Load catalog & flat Float32 vector index
catalog = json.load(open(cat_path, encoding="utf-8"))["species"]
photo_index = np.fromfile(f32_path, dtype=np.float32).reshape(len(catalog), 768)

# 2. Preprocess image to [1, 3, 160, 160] in [0, 1]
img = Image.open("bird.jpg").convert("RGB").resize((160, 160))
x = (np.asarray(img, dtype=np.float32) / 255.0).transpose(2, 0, 1)[None, ...]

# 3. Run distilled ONNX encoder & cosine similarity search
sess = ort.InferenceSession(onnx_path)
emb = sess.run(None, {"image_rgb": x})[0][0]
sims = photo_index @ emb
top1 = int(np.argmax(sims))
print(f"#1: {catalog[top1]['en_name']} ({catalog[top1]['kr_name']}) — similarity={sims[top1]:.4f}")

---

🇰🇷 한국어 안내 (Korean Overview)

본 모델(jaichang/embeddinggemma2-korean-birds-620, 740M)의 768차원 임베딩 공간을 **지식증류(Knowledge Distillation)**하여, Flutter 모바일 앱(tflite_flutter)과 Flutter Web 브라우저(WebGPU / WebGL / WASM)에서 서버 없이 100% 온디바이스로 구동할 수 있도록 경량화한 전용 레포지토리입니다.

🧠 지식증류(Knowledge Distillation) vs 무증류 원본 타워(Un-distilled 740M Tower)

1.2GB에 달하는 전체 model.safetensors 중 **티어 1(초경량 지식증류 학생 모델, 합계 23.9 MB)**과 티어 2(무증류 원본 740M 파인튜닝 서브타워 INT8 직변환, 합계 53.3 MB) 두 가지를 모두 제공합니다:

• ⚡ 티어 1 — 지식증류 학생 모델 (embeddinggemma2_{vision,audio}_768d.onnx/.tflite, 합계 23.9 MB): 비전 Top-1 100.0%, 오디오 Top-1 99.1%, WebGPU 초고속 추론(~30-80ms)
• 🎯 티어 2 — 무증류 원본 740M 파인튜닝 타워 (embeddinggemma2_undistilled_{vision,audio}_int8.onnx/.tflite, 합계 53.3 MB): model.safetensors 내 실제 Gemma4Vision Transformer(22.38 MB) 및 Gemma4Audio Conformer(30.90 MB) 가중치를 지식증류 없이 그대로 추출하여 INT8 양자화한 원본 서브타워 모델

📦 파일 구성

파일명	대상 플랫폼 / 런타임	용도 및 크기
embeddinggemma2_vision_768d.onnx / .tflite	Flutter Web (WebGPU) & 모바일 (LiteRT)	[티어 1: 경량 증류] 비전 인코더 (160x160 RGB $\rightarrow$ 768-d, 9.24 MB)
embeddinggemma2_audio_768d.onnx / .tflite	Flutter Web (WebGPU) & 모바일 (LiteRT)	[티어 1: 경량 증류] 오디오 인코더 (16kHz 파형 $\rightarrow$ 768-d, 14.70 MB)
embeddinggemma2_undistilled_vision_int8.onnx / .tflite	ONNX Runtime (WASM SIMD) & 모바일 (LiteRT)	[티어 2: 무증류 원본] 파인튜닝된 Gemma4Vision 원본 타워 INT8 (22.38 MB)
embeddinggemma2_undistilled_audio_int8.onnx / .tflite	ONNX Runtime (WASM SIMD) & 모바일 (LiteRT)	[티어 2: 무증류 원본] 파인튜닝된 Gemma4Audio Conformer 원본 타워 INT8 (30.90 MB)
korean_620_birds_audio_index.f32	모바일 & 웹 공용 (Float32List)	조류 620종 오디오 기준 벡터 (1.8 MB)
korean_620_birds_photo_index.f32	모바일 & 웹 공용 (Float32List)	조류 620종 새 사진 기준 벡터 (1.8 MB)
korean_620_birds_catalog.json	모바일 & 웹 공용	조류 620종 국명·영문명·학명·설명·사진·소리 URL 도감 메타데이터