Load the decision fine-tune (ONNX served from the Space), trained prompt format and scale, Egg check tab

.gitattributes

@@ -42,3 +42,8 @@ examples/textvqa-mc.jpg filter=lfs diff=lfs merge=lfs -text
 examples/vqav2-yesno.jpg filter=lfs diff=lfs merge=lfs -text
 models/egemma2-decide/onnx/audio_encoder_q4.onnx_data filter=lfs diff=lfs merge=lfs -text
 models/egemma2-decide/onnx/audio_encoder_q4f16.onnx_data filter=lfs diff=lfs merge=lfs -text
+models/egemma2-decide/onnx/model_q4.onnx_data filter=lfs diff=lfs merge=lfs -text
+models/egemma2-decide/onnx/model_q4f16.onnx_data filter=lfs diff=lfs merge=lfs -text
+models/egemma2-decide/onnx/vision_encoder_q4.onnx_data filter=lfs diff=lfs merge=lfs -text
+models/egemma2-decide/onnx/vision_encoder_q4f16.onnx_data filter=lfs diff=lfs merge=lfs -text
+models/egemma2-decide/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,65 +1,68 @@
----
-title: Decide Live
-emoji: 🎯
-colorFrom: green
-colorTo: gray
-sdk: static
-pinned: false
-license: apache-2.0
-short_description: On-device decisions from camera and microphone
-models:
-  - google/embeddinggemma-2
-  - FINAL-Bench/EmbeddingGemma2-ZTC-ONNX
----
-
-# Decide Live: on-device decisions from camera and microphone
-
-A static page that makes zero-token decisions in the browser. A multimodal embedding model,
-[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) (Apache-2.0) fine-tuned for decisions,
-runs locally with [transformers.js](https://github.com/huggingface/transformers.js) 4.3.1 on WebGPU,
-or on the CPU (WASM) when WebGPU is not available (slower, and it uses the larger fp16 files).
-
-The ONNX files (from the private model repo `FINAL-Bench/EmbeddingGemma2-ZTC-ONNX`, same file layout as
-[onnx-community/embeddinggemma-2-ONNX](https://huggingface.co/onnx-community/embeddinggemma-2-ONNX)) are served
-from this Space under `models/egemma2-decide/`, so the page needs no access to any other repository.
-There is no server and no API call per decision. The weights are downloaded once and cached by the browser.
-Camera frames, microphone audio, images and text are processed on the device and are never uploaded.
-
-## What it does
-
-- **Live**: embeds camera frames (or a demo feed made from the sample images) and scores up to 4 decision
-  cards. By default each card embeds the frame together with its own question, in the trained format
-  (one pass per card). The "frame alone" setting embeds the frame once for all cards (faster, not the
-  trained format). Each card shows probability bars, the current decision, and a confidence gate.
-- **Microphone / sound file**: the same cards for 2 second sound windows (16 kHz mono). Windows
-  quieter than -55 dBFS are not sent to the model. The audio encoder is the unchanged base one.
-- **Single input**: one image, text, audio clip or short video, a question, an optional context,
-  and 2 to 8 options written as `key: description` (or a yes / no question).
-- **Egg check**: 8 held-out candling photos (not used in training) and your own photo, decided with
-  the trained egg prompt.
-- **Sample tests**: 18 image tests with gold answers, run in the browser with an accuracy report,
-  plus a comparison of prompt formats.
-
-## How a decision is computed (trained format)
-
-1. Query: the media placeholders first, then `task: classification | query: {question}`, then
-   `\nContext: {context}` when a context is given. The model embeds media and text together.
-2. Each option: `title: none | text: {description}`, embedded once and cached.
-3. Embedding: mean pooling over the attention mask, then L2 normalization (768 numbers).
-4. P(option) = softmax(scale * cosine). The scale was learned in training: 19.96
-   (temperature 1 / scale, about 0.0501), stored in `models/egemma2-decide/decision.json`.
-
-Yes / no questions keep the question in the query and score the options `Yes` and `No`.
-The older prompt formats of the first version of this page remain selectable for comparison.
-
-## Files
-
-- `index.html`, `style.css`, `app.js`: the page.
-- `worker.js`: loads the model in a Web Worker and returns embeddings.
-- `models/egemma2-decide/`: ONNX model (q4, q4f16 and fp16 text and vision graphs, 4-bit audio graphs),
-  config, processor and tokenizer files, and `decision.json` (scale and prompt formats).
-- `examples/`: sample images and `samples.json` (name, files, state, type, question, options, gold);
-  `examples/egg-test/`: held-out egg candling photos.
-- `CREDITS.md`: sources and licenses of the sample images.
-
-Code: Apache-2.0. Model: derived from google/embeddinggemma-2 (Apache-2.0).
+---
+title: Decide Live
+emoji: 🎯
+colorFrom: green
+colorTo: gray
+sdk: static
+pinned: false
+license: apache-2.0
+short_description: On-device decisions from camera and microphone
+models:
+  - google/embeddinggemma-2
+  - FINAL-Bench/EmbeddingGemma2-ZTC-ONNX
+---
+
+# Decide Live: on-device decisions from camera and microphone
+
+A static page that makes zero-token decisions in the browser. A multimodal embedding model,
+[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) (Apache-2.0) fine-tuned for decisions,
+runs locally with [transformers.js](https://github.com/huggingface/transformers.js) 4.3.1 on WebGPU,
+or on the CPU (WASM) when WebGPU is not available (much slower).
+
+The ONNX files (from the private model repo `FINAL-Bench/EmbeddingGemma2-ZTC-ONNX`, same file layout as
+[onnx-community/embeddinggemma-2-ONNX](https://huggingface.co/onnx-community/embeddinggemma-2-ONNX)) are served
+from this Space under `models/egemma2-decide/`, so the page needs no access to any other repository.
+There is no server and no API call per decision. The weights are downloaded once and cached by the browser.
+Camera frames, microphone audio, images and text are processed on the device and are never uploaded.
+
+## What it does
+
+- **Live**: embeds camera frames (or a demo feed made from the sample images) and scores up to 4 decision
+  cards. By default each card embeds the frame together with its own question, in the trained format
+  (one pass per card). The "frame alone" setting embeds the frame once for all cards (faster, not the
+  trained format). Each card shows probability bars, the current decision, and a confidence gate.
+- **Microphone / sound file**: the same cards for 2 second sound windows (16 kHz mono). Windows
+  quieter than -55 dBFS are not sent to the model. The audio encoder is the unchanged base one.
+- **Single input**: one image, text, audio clip or short video, a question, an optional context,
+  and 2 to 8 options written as `key: description` (or a yes / no question).
+- **Egg check**: 8 held-out candling photos (not used in training) and your own photo, decided with
+  the trained egg prompt.
+- **Sample tests**: 18 image tests with gold answers, run in the browser with an accuracy report,
+  plus a comparison of prompt formats.
+
+## How a decision is computed (trained format)
+
+1. Query: the media placeholders first, then `task: classification | query: {question}`, then
+   `\nContext: {context}` when a context is given. The model embeds media and text together.
+2. Each option: `title: none | text: {description}`, embedded once and cached.
+3. Embedding: mean pooling over the attention mask, then L2 normalization (768 numbers).
+4. P(option) = softmax(scale * cosine). The scale was learned in training: 19.96
+   (temperature 1 / scale, about 0.0501), stored in `models/egemma2-decide/decision.json`.
+
+Yes / no questions keep the question in the query and score the options `Yes` and `No`.
+The older prompt formats of the first version of this page remain selectable for comparison.
+
+## Files
+
+- `index.html`, `style.css`, `app.js`: the page.
+- `worker.js`: loads the model in a Web Worker and returns embeddings.
+- `models/egemma2-decide/`: ONNX model, config, processor and tokenizer files, and `decision.json`
+  (scale and prompt formats). Text and vision graphs in `q4f16` (as in the model repo) and in a portable
+  `q4` (the model repo's `onnx_portable/` files: 4-bit weights, with plain 8-bit gathers for the embedding
+  and position tables, so the same files also run on the WASM runtime, which lacks GatherBlockQuantized),
+  plus the 4-bit audio encoder. fp16 and fp32 files are not copied here because Space storage is limited to 1 GB.
+- `examples/`: sample images and `samples.json` (name, files, state, type, question, options, gold);
+  `examples/egg-test/`: held-out egg candling photos.
+- `CREDITS.md`: sources and licenses of the sample images.
+
+Code: Apache-2.0. Model: derived from google/embeddinggemma-2 (Apache-2.0).

index.html

@@ -184,9 +184,8 @@
         <label class="col">Precision (applies on next load)
           <select id="dtype">
             <option value="auto">Auto: 4-bit (q4f16 on WebGPU with fp16 support, else q4)</option>
-            <option value="q4">q4 (4-bit, works everywhere)</option>
+            <option value="q4">q4 (4-bit, works everywhere, also on the CPU)</option>
             <option value="q4f16">q4f16 (4-bit weights, fp16 math; WebGPU with fp16 only)</option>
-            <option value="fp16">fp16 (largest; WebGPU only)</option>
           </select>
         </label>
         <label class="col">Backend (applies on next load)
@@ -270,7 +269,7 @@
       <ul>
         <li>An embedding model compares meanings; it does not read, count or reason step by step. Questions about exact counts, small text or negation are hard for it.</li>
         <li>Probabilities depend on the temperature. They are comparable between options of one card, not calibrated against ground truth.</li>
-        <li>WebGPU needs a recent Chrome, Edge or Safari. Without it the page falls back to the CPU (WASM): it works, but it downloads a larger fp16 model (about 880 MB, because the 4-bit graphs need an operator the CPU runtime lacks) and takes several seconds per image, so the live rate drops to a frame every few seconds.</li>
+        <li>WebGPU needs a recent Chrome, Edge or Safari. Without it the page falls back to the CPU (WASM): it works with the same 4-bit files (about 340 MB), but it takes several seconds per image (about 6 s at 70 image tokens, about 30 s at 280 in our test), so the live rate drops to a frame every few seconds.</li>
         <li>The fine-tuning targeted image decisions (including egg candling). The audio encoder is the unchanged base one, so sound decisions are not tuned.</li>
       </ul>
       <p class="note">Sample image credits: <a href="./CREDITS.md" target="_blank">CREDITS.md</a>. Code: Apache-2.0.</p>

models/egemma2-decide/onnx/model_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb6b3c8cae2c7a65272c6211313f34f70941df1d40869d4021be22ec27aabdee
+size 490555

models/egemma2-decide/onnx/model_q4.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bdc17b2daccb303a367a306f914d9de3781ba58ac1d2c44f3ba7d0a626851e5
+size 224360448

models/egemma2-decide/onnx/model_q4f16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53feeced79582d661e30adeaa9829cea90d92b1c78347e26fd123773580f71d0
+size 495298

models/egemma2-decide/onnx/model_q4f16.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e293b62bf40b405a9f0e88985dc4a25a5bb05572ca314da4baad9400be3eae8
+size 156862464

models/egemma2-decide/onnx/vision_encoder_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:daea777fe94c02d19749c286fdd0f52d2cea56697a0f093c9b7f4652e9646a75
+size 164811

models/egemma2-decide/onnx/vision_encoder_q4.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6823b98325ae0b4e8108fb28f995f9dfb5496b43c8432c65c8c7d881eeb39fb3
+size 114774016

models/egemma2-decide/onnx/vision_encoder_q4f16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:572258f9700e2a23edaa478a717d4da29618206e24ccd4fb34c822181ab49e27
+size 152866

models/egemma2-decide/onnx/vision_encoder_q4f16.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27a86792e1d3572b4667cf09cb59165e54a18e3e9bc59bf22f2c35b845ad052a
+size 97614336

models/egemma2-decide/preprocessor_config.json

@@ -0,0 +1,22 @@
+{
+  "dither": 0.0,
+  "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+  "feature_size": 128,
+  "fft_length": 512,
+  "fft_overdrive": false,
+  "frame_length": 320,
+  "hop_length": 160,
+  "input_scale_factor": 1.0,
+  "max_frequency": 8000.0,
+  "mel_floor": 0.001,
+  "min_frequency": 0.0,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "per_bin_mean": null,
+  "per_bin_stddev": null,
+  "preemphasis": 0.0,
+  "preemphasis_htk_flavor": true,
+  "return_attention_mask": true,
+  "sampling_rate": 16000,
+  "processor_class": "EmbeddingGemma2Processor"
+}

models/egemma2-decide/processor_config.json

@@ -0,0 +1,78 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 280,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "EmbeddingGemma2Processor",
+  "video_processor": {
+    "add_timestamps": false,
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 1,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_frames": 32,
+    "max_soft_tokens": 140,
+    "overflow_strategy": "uniform",
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "EmbeddingGemma2VideoProcessor"
+  }
+}

models/egemma2-decide/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d777ef5bdc1aa36227abdfb77c3e49e7b9c892d16e1b6bda41c393504828be4
+size 32170510

models/egemma2-decide/tokenizer_config.json

@@ -0,0 +1,57 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>",
+    "video_token": "<|video|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "processor_class": "EmbeddingGemma2Processor",
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "video_token": "<|video|>"
+}

style.css

@@ -180,3 +180,17 @@ td.no { color: var(--bad); font-weight: 700; }
   .hud { top: 44px; left: 8px; right: auto; font-size: 11.5px; }
   .overlay { left: 8px; right: 8px; bottom: 8px; gap: 10px; padding: 8px 10px; }
 }
+
+/* egg check */
+.egggrid { display: grid; grid-template-columns: repeat(auto-fill, minmax(120px, 1fr)); gap: 10px; margin: 12px 0; }
+.eggthumb { padding: 6px; display: flex; flex-direction: column; gap: 4px; align-items: stretch; font-size: 12px; color: var(--muted); }
+.eggthumb img { width: 100%; aspect-ratio: 1; object-fit: cover; border-radius: 8px; background: #000; display: block; }
+.btnrow .drop.inline { display: inline-block; margin: 0; padding: 8px 12px; }
+.eggresult { display: grid; grid-template-columns: minmax(120px, 220px) 1fr; gap: 16px; align-items: start; margin: 12px 0; }
+.eggresult img { width: 100%; border-radius: 10px; background: #000; }
+@media (max-width: 560px) { .eggresult { grid-template-columns: 1fr; } }
+.okc { color: var(--good); font-weight: 600; margin-left: 6px; }
+.noc { color: var(--bad); font-weight: 600; margin-left: 6px; }
+img.mini { width: 32px; height: 32px; object-fit: cover; border-radius: 4px; vertical-align: middle; margin-right: 6px; }
+.dctx { display: block; font-size: 12px; font-weight: 400; color: var(--muted); }
+.dedit input + input { margin-top: 6px; }

worker.js

@@ -1,200 +1,211 @@
-// SPDX-License-Identifier: Apache-2.0
-// Embedding worker: loads google/embeddinggemma-2 (ONNX build) with transformers.js
-// and returns L2-normalized 768-d sentence embeddings. Runs entirely in the browser.
-// Loading approach follows the transformers.js usage in the model card of
-// onnx-community/embeddinggemma-2-ONNX (Apache-2.0).
-import {
-  AutoConfig,
-  AutoModel,
-  AutoProcessor,
-  RawImage,
-  RawVideo,
-  RawVideoFrame,
-  env,
-} from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.3.1/dist/transformers.min.js";
-
-const MODEL_ID = "onnx-community/embeddinggemma-2-ONNX";
-env.allowLocalModels = false;
-if (env.backends?.onnx?.wasm) env.backends.onnx.wasm.numThreads = 1; // static hosting is not cross-origin isolated
-
-let processor = null;
-let model = null;
-let loaded = null; // { device, dtype, audio }
-let queue = Promise.resolve();
-
-const post = (msg, transfer) => self.postMessage(msg, transfer || []);
-const state = (phase, label, extra = {}) => post({ type: "state", phase, label, ...extra });
-
-let gpuF16 = false;
-async function hasWebGPU() {
-  try {
-    if (!("gpu" in navigator)) return false;
-    const adapter = await navigator.gpu.requestAdapter();
-    gpuF16 = !!adapter?.features?.has("shader-f16");
-    return !!adapter;
-  } catch {
-    return false;
-  }
-}
-
-async function loadModel({ device = "auto", dtype = "q4", audio = false }) {
-  if (loaded && (device === "auto" || device === loaded.device) && dtype === loaded.requested && (!audio || loaded.audio)) {
-    return loaded;
-  }
-  if (model) {
-    try { await model.dispose(); } catch {}
-    model = null;
-    loaded = null;
-  }
-  let candidates;
-  if (device === "auto") candidates = (await hasWebGPU()) ? ["webgpu", "wasm"] : ["wasm"];
-  else candidates = [device];
-
-  if (!processor) {
-    state("loading", "Loading processor");
-    processor = await AutoProcessor.from_pretrained(MODEL_ID);
-  }
-  let lastErr = null;
-  const requested = dtype;
-  if (candidates.includes("webgpu") && device !== "auto") await hasWebGPU();
-  for (const dev of candidates) {
-    // "auto": 4-bit weights; fp16 math on WebGPU adapters that support shader-f16 (faster), else q4
-    if (dev === "wasm") {
-      // The quantized text and vision graphs use GatherBlockQuantized, which the WASM (CPU) runtime
-      // does not implement, so on WASM the text and vision parts run in fp16. The audio graphs have no such op.
-      const aq = requested === "auto" || requested === "q4f16" || requested === "fp16" ? "q4" : requested;
-      dtype = audio ? { model: "fp16", vision_encoder: "fp16", audio_encoder: aq } : "fp16";
-    } else {
-      dtype = requested === "auto" ? (gpuF16 ? "q4f16" : "q4") : requested;
-    }
-    try {
-      const config = await AutoConfig.from_pretrained(MODEL_ID);
-      if (!audio) config.audio_config = null; // text + vision only (video uses the vision encoder)
-      state("loading", `Downloading model (${dev}${audio ? ", with audio encoder" : ""})`, { progress: 0, device: dev });
-      const t0 = performance.now();
-      model = await AutoModel.from_pretrained(MODEL_ID, {
-        config,
-        device: dev,
-        dtype,
-        progress_callback: (p) => {
-          if (p.status === "progress_total") state("loading", `Downloading model (${dev})`, { progress: p.progress, device: dev, loadedBytes: p.loaded, totalBytes: p.total });
-        },
-      });
-      state("loading", "Warming up", { progress: 100, device: dev });
-      const warm = await processor(["task: classification | query: warm up"]);
-      const out = await model(warm);
-      out.sentence_embedding.dispose?.();
-      const label = typeof dtype === "string" ? dtype : Object.entries(dtype).map(([k, v]) => `${k.replace("_encoder", "")} ${v}`).join(", ");
-      loaded = { device: dev, dtype: label, requested, audio, loadMs: performance.now() - t0 };
-      state("ready", `Ready on ${dev.toUpperCase()} (${label})`, { device: dev, audio });
-      return loaded;
-    } catch (e) {
-      lastErr = e;
-      console.warn(`Loading on ${dev} failed`, e);
-      if (model) { try { await model.dispose(); } catch {} model = null; }
-      state("loading", `Loading on ${dev} failed, trying next backend`, { device: dev });
-    }
-  }
-  state("error", `Could not load the model: ${lastErr?.message || lastErr}`);
-  throw lastErr;
-}
-
-function rows(tensor) {
-  const [n, d] = tensor.dims;
-  const data = tensor.data;
-  const out = [];
-  for (let i = 0; i < n; i++) {
-    const v = new Float32Array(d);
-    let s = 0;
-    for (let j = 0; j < d; j++) { v[j] = data[i * d + j]; s += v[j] * v[j]; }
-    s = Math.sqrt(s) || 1;
-    for (let j = 0; j < d; j++) v[j] /= s;
-    out.push(v);
-  }
-  return out;
-}
-
-let lastModelMs = 0;
-async function run(inputs) {
-  const tm = performance.now();
-  const out = await model(inputs);
-  lastModelMs = performance.now() - tm;
-  const emb = rows(out.sentence_embedding);
-  try { out.sentence_embedding.dispose?.(); } catch {}
-  return emb;
-}
-
-// texts: array of strings, embedded in small batches
-async function embedTexts(texts) {
-  const res = [];
-  const B = 8;
-  for (let i = 0; i < texts.length; i += B) {
-    const batch = texts.slice(i, i + B);
-    res.push(...(await run(await processor(batch))));
-  }
-  return res;
-}
-
-// One input that may interleave text with an image, audio clip or video.
-// With empty text the media is embedded on its own (no prefix, as the model card advises for media).
-async function embedInput(input) {
-  const { kind } = input;
-  const text = (input.text || "").trim();
-  if (kind === "text" || !kind) {
-    return (await run(await processor([text])))[0];
-  }
-  if (kind === "image" || kind === "rgba") {
-    if (processor.image_processor) processor.image_processor.max_soft_tokens = input.budget || 280;
-    const img = kind === "rgba"
-      ? new RawImage(new Uint8ClampedArray(input.data), input.width, input.height, 4)
-      : await RawImage.fromBlob(input.blob);
-    if (!text) return (await run(await processor(null, img)))[0];
-    const t = text.includes("<|image|>") ? text : `${text} <|image|>`;
-    return (await run(await processor([t], [[img]])))[0];
-  }
-  if (kind === "audio") {
-    if (!loaded?.audio) throw new Error("The audio encoder is not loaded.");
-    const audio = input.audio instanceof Float32Array ? input.audio : new Float32Array(input.audio);
-    if (!text) return (await run(await processor(null, null, audio)))[0];
-    const t = text.includes("<|audio|>") ? text : `${text} <|audio|>`;
-    return (await run(await processor([t], null, [audio])))[0];
-  }
-  if (kind === "video") {
-    if (processor.image_processor) processor.image_processor.max_soft_tokens = input.budget || 280;
-    const frames = input.frames.map((f) => new RawVideoFrame(new RawImage(new Uint8ClampedArray(f.data), f.width, f.height, 4), f.timestamp));
-    const video = new RawVideo(frames, input.duration);
-    if (!text) return (await run(await processor(null, null, null, video)))[0];
-    const t = text.includes("<|video|>") ? text : `${text} <|video|>`;
-    return (await run(await processor([t], null, null, [video])))[0];
-  }
-  throw new Error(`Unknown input kind: ${kind}`);
-}
-
-async function handle(msg) {
-  const { id, op } = msg;
-  try {
-    if (op === "load") {
-      const info = await loadModel(msg.options || {});
-      post({ type: "result", id, result: info });
-      return;
-    }
-    if (!model) throw new Error("Model is not loaded yet.");
-    const t0 = performance.now();
-    if (op === "embedTexts") {
-      const embs = await embedTexts(msg.texts);
-      post({ type: "result", id, result: { embeddings: embs, ms: performance.now() - t0 } }, embs.map((e) => e.buffer));
-    } else if (op === "embedInput") {
-      const emb = await embedInput(msg.input);
-      post({ type: "result", id, result: { embedding: emb, ms: performance.now() - t0, modelMs: lastModelMs } }, [emb.buffer]);
-    } else {
-      throw new Error(`Unknown op ${op}`);
-    }
-  } catch (e) {
-    post({ type: "error", id, message: e?.message || String(e) });
-  }
-}
-
-self.addEventListener("message", ({ data }) => {
-  // serialize all requests: one model, one queue
-  queue = queue.then(() => handle(data));
-});
+// SPDX-License-Identifier: Apache-2.0
+// Embedding worker: loads a decision-tuned EmbeddingGemma 2 (ONNX, same graph layout as
+// onnx-community/embeddinggemma-2-ONNX) with transformers.js and returns L2-normalized
+// 768-d embeddings (mean pooling over the attention mask, then L2 normalization).
+// The model files are served next to this page (./models/...), so nothing is fetched from elsewhere.
+import {
+  AutoConfig,
+  AutoModel,
+  AutoProcessor,
+  RawImage,
+  RawVideo,
+  RawVideoFrame,
+  env,
+} from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.3.1/dist/transformers.min.js";
+
+const MODEL_ID = "egemma2-decide";
+env.allowLocalModels = true;
+env.allowRemoteModels = false;
+env.localModelPath = new URL("./models/", self.location.href).pathname; // same-origin path, e.g. "/models/"
+if (env.backends?.onnx?.wasm) env.backends.onnx.wasm.numThreads = 1; // static hosting is not cross-origin isolated
+
+let processor = null;
+let model = null;
+let loaded = null; // { device, dtype, audio }
+let decision = null; // { scale } from decision.json
+let queue = Promise.resolve();
+
+const post = (msg, transfer) => self.postMessage(msg, transfer || []);
+const state = (phase, label, extra = {}) => post({ type: "state", phase, label, ...extra });
+
+let gpuF16 = false;
+async function hasWebGPU() {
+  try {
+    if (!("gpu" in navigator)) return false;
+    const adapter = await navigator.gpu.requestAdapter();
+    gpuF16 = !!adapter?.features?.has("shader-f16");
+    return !!adapter;
+  } catch {
+    return false;
+  }
+}
+
+async function loadModel({ device = "auto", dtype = "q4", audio = false }) {
+  if (loaded && (device === "auto" || device === loaded.device) && dtype === loaded.requested && (!audio || loaded.audio)) {
+    return loaded;
+  }
+  if (model) {
+    try { await model.dispose(); } catch {}
+    model = null;
+    loaded = null;
+  }
+  let candidates;
+  if (device === "auto") candidates = (await hasWebGPU()) ? ["webgpu", "wasm"] : ["wasm"];
+  else candidates = [device];
+
+  if (!decision) {
+    try { decision = await (await fetch(`${env.localModelPath}${MODEL_ID}/decision.json`)).json(); } catch { decision = {}; }
+  }
+  if (!processor) {
+    state("loading", "Loading processor");
+    processor = await AutoProcessor.from_pretrained(MODEL_ID);
+  }
+  let lastErr = null;
+  const requested = dtype;
+  if (candidates.includes("webgpu") && device !== "auto") await hasWebGPU();
+  for (const dev of candidates) {
+    // Files shipped with this page (Space storage is limited to 1 GB):
+    //   q4: 4-bit weights with plain 8-bit gathers for the embedding and position tables, so the same files
+    //       run on WebGPU and on the WASM (CPU) runtime, which lacks GatherBlockQuantized;
+    //   q4f16: 4-bit weights with fp16 math (WebGPU with shader-f16 only);
+    //   audio: one 4-bit audio encoder used with both.
+    const d = dev === "wasm" ? "q4" : requested === "auto" ? (gpuF16 ? "q4f16" : "q4") : requested;
+    dtype = audio ? { model: d, vision_encoder: d, audio_encoder: "q4" } : d;
+    try {
+      const config = await AutoConfig.from_pretrained(MODEL_ID);
+      if (!audio) config.audio_config = null; // text + vision only (video uses the vision encoder)
+      state("loading", `Downloading model (${dev}${audio ? ", with audio encoder" : ""})`, { progress: 0, device: dev });
+      const t0 = performance.now();
+      model = await AutoModel.from_pretrained(MODEL_ID, {
+        config,
+        device: dev,
+        dtype,
+        progress_callback: (p) => {
+          if (p.status === "progress_total") state("loading", `Downloading model (${dev})`, { progress: p.progress, device: dev, loadedBytes: p.loaded, totalBytes: p.total });
+        },
+      });
+      const sessionMs = performance.now() - t0;
+      state("loading", "Warming up", { progress: 100, device: dev });
+      const tw = performance.now();
+      const warm = await processor(["task: classification | query: warm up"]);
+      const out = await model(warm);
+      out.sentence_embedding.dispose?.();
+      const warmMs = performance.now() - tw;
+      const label = typeof dtype === "string" ? dtype : Object.entries(dtype).map(([k, v]) => `${k.replace("_encoder", "")} ${v}`).join(", ");
+      loaded = { device: dev, dtype: label, requested, audio, loadMs: performance.now() - t0, sessionMs, warmMs, scale: decision?.scale || null };
+      state("ready", `Ready on ${dev.toUpperCase()} (${label})`, { device: dev, audio });
+      return loaded;
+    } catch (e) {
+      lastErr = e;
+      console.warn(`Loading on ${dev} failed`, e);
+      if (model) { try { await model.dispose(); } catch {} model = null; }
+      state("loading", `Loading on ${dev} failed, trying next backend`, { device: dev });
+    }
+  }
+  state("error", `Could not load the model: ${lastErr?.message || lastErr}`);
+  throw lastErr;
+}
+
+function rows(tensor) {
+  const [n, d] = tensor.dims;
+  const data = tensor.data;
+  const out = [];
+  for (let i = 0; i < n; i++) {
+    const v = new Float32Array(d);
+    let s = 0;
+    for (let j = 0; j < d; j++) { v[j] = data[i * d + j]; s += v[j] * v[j]; }
+    s = Math.sqrt(s) || 1;
+    for (let j = 0; j < d; j++) v[j] /= s;
+    out.push(v);
+  }
+  return out;
+}
+
+let lastModelMs = 0;
+async function run(inputs) {
+  const tm = performance.now();
+  const out = await model(inputs);
+  lastModelMs = performance.now() - tm;
+  const emb = rows(out.sentence_embedding);
+  try { out.sentence_embedding.dispose?.(); } catch {}
+  return emb;
+}
+
+// texts: array of strings, embedded in small batches (right padding; padded tokens are masked out)
+async function embedTexts(texts) {
+  const res = [];
+  const B = 8;
+  for (let i = 0; i < texts.length; i += B) {
+    const batch = texts.slice(i, i + B);
+    res.push(...(await run(await processor(batch))));
+  }
+  return res;
+}
+
+// One input that may interleave text with an image, audio clip or video.
+// If the text already holds a placeholder (<|image|>, <|audio|>, <|video|>) it is used as is;
+// otherwise the placeholder is put first, as in the trained prompt format. Empty text embeds the media alone.
+async function embedInput(input) {
+  const { kind } = input;
+  const text = (input.text || "").trim();
+  const withPh = (ph) => (text.includes(ph) ? text : `${ph}${text}`);
+  if (kind === "text" || !kind) {
+    return (await run(await processor([text])))[0];
+  }
+  if (kind === "image" || kind === "rgba") {
+    if (processor.image_processor) processor.image_processor.max_soft_tokens = input.budget || 280;
+    const img = kind === "rgba"
+      ? new RawImage(new Uint8ClampedArray(input.data), input.width, input.height, 4)
+      : await RawImage.fromBlob(input.blob);
+    if (!text) return (await run(await processor(null, img)))[0];
+    return (await run(await processor([withPh("<|image|>")], [[img]])))[0];
+  }
+  if (kind === "audio") {
+    if (!loaded?.audio) throw new Error("The audio encoder is not loaded.");
+    const audio = input.audio instanceof Float32Array ? input.audio : new Float32Array(input.audio);
+    if (!text) return (await run(await processor(null, null, audio)))[0];
+    return (await run(await processor([withPh("<|audio|>")], null, [audio])))[0];
+  }
+  if (kind === "video") {
+    if (processor.image_processor) processor.image_processor.max_soft_tokens = input.budget || 280;
+    const frames = input.frames.map((f) => new RawVideoFrame(new RawImage(new Uint8ClampedArray(f.data), f.width, f.height, 4), f.timestamp));
+    const video = new RawVideo(frames, input.duration);
+    if (!text) return (await run(await processor(null, null, null, video)))[0];
+    return (await run(await processor([withPh("<|video|>")], null, null, [video])))[0];
+  }
+  throw new Error(`Unknown input kind: ${kind}`);
+}
+
+async function handle(msg) {
+  const { id, op } = msg;
+  try {
+    if (op === "load") {
+      const info = await loadModel(msg.options || {});
+      post({ type: "result", id, result: info });
+      return;
+    }
+    if (!model) throw new Error("Model is not loaded yet.");
+    const t0 = performance.now();
+    if (op === "embedTexts") {
+      const embs = await embedTexts(msg.texts);
+      post({ type: "result", id, result: { embeddings: embs, ms: performance.now() - t0 } }, embs.map((e) => e.buffer));
+    } else if (op === "embedInput") {
+      const emb = await embedInput(msg.input);
+      post({ type: "result", id, result: { embedding: emb, ms: performance.now() - t0, modelMs: lastModelMs } }, [emb.buffer]);
+    } else if (op === "embedInputs") {
+      // several prompts over the same media (one pass each); used by live cards in the trained format
+      const embs = [];
+      for (const inp of msg.inputs) embs.push(await embedInput(inp));
+      post({ type: "result", id, result: { embeddings: embs, ms: performance.now() - t0 } }, embs.map((e) => e.buffer));
+    } else {
+      throw new Error(`Unknown op ${op}`);
+    }
+  } catch (e) {
+    post({ type: "error", id, message: e?.message || String(e) });
+  }
+}
+
+self.addEventListener("message", ({ data }) => {
+  // serialize all requests: one model, one queue
+  queue = queue.then(() => handle(data));
+});