litert-community

/

MiniCPM5-2B

Like 8

Follow

LiteRT Community (FKA TFLite) 9.03k

Thanks for pulling the two CPU+GPU files into this repo and for the card update — this adds the details behind them, as asked in the Space thread.

What the PR adds to the README, keeping your structure and the OpenBMB sections as they are:

• Correctness — the 8-question gate on CPU and GPU (Mac, Galaxy S26 with full OpenCL delegation, iPhone 17 Pro), GSM8K n=100 with thinking off against the bf16 reference (92 / int8 91 / int4 86–87), and the thinking-mode caveat: int4's reasoning chains run ~4× longer than int8's and hit the budget; int8 is the file when the reasoning has to finish.
• Controlling thinking — the model thinks by default; how enable_thinking reaches the template (ThinkingConfig / extra context / CLI flags), the output budget it needs, and the sampling OpenBMB recommends.
• Conversion notes — what makes these files run on both backends: the checkpoint's chat_template.jinja embedded verbatim with the thought channel declared; the int4 zero-scale fix (13 all-zero rows in layer 0's MLP → XNNPACK refuses scale 0, the GPU does not); the int8 file's prefer_activation_type = "fp32" declaration (fp16 GPU activations let one gate question run 2000+ tokens without closing </think>). Tool versions and the reproduction command.
• License → Changes from the original work — the Apache-2.0 attribution line for the converted files.
• litertlm_manifest.json — sha256, sizes, sections, verified backends and the measured rows for the two recommended files, for tooling that picks a file per device (same schema as the other litert-community LLM cards that carry one).

Two small edits to existing text: "compatable" → "compatible", and a GPU example line under Try It. Nothing changes in the model files or in the OpenBMB sections. If a different length or shape fits the org's cards better, tell me once and I'll rework it.