About

Mixed precision modelopt of https://huggingface.co/maldv/Gemma-4-31B-Isometry-Fabled-Persona

Full conversion script in the model files: https://huggingface.co/Legislat/Gemma-4-31B-Isometry-Fabled-Persona-nvfp4/blob/main/quantize_gemma4-stagepass.py

Weights: 19.81 GiB memory

VLLM serve example (RTX PRO 5000 72gb):

vllm serve ~/ai/models/Gemma-4-31B-Isometry-Fabled-Persona-nvfp4
--quantization modelopt_fp4
--kv-cache-dtype fp8
--max-model-len 16384
--max-num-seqs 1
--gpu-memory-utilization 0.42
--trust-remote-code
--reasoning-parser gemma4
--tool-call-parser gemma4
--enable-auto-tool-choice
--chat-template ~/ai/models/Gemma-4-31B-Isometry-Fabled-Persona-nvfp4/chat_template.jinja

Great model, big thanks to maldv.