zhiyaowang/MiniCPM-SALA-EAGLE3-Draft-NVFP4

MiniCPM-SALA EAGLE3 Draft NVFP4

English

This repository contains the NVFP4-quantized EAGLE3 draft model for MiniCPM-SALA speculative decoding. It is a draft-only checkpoint and was trained for the MiniCPM-SALA GPTQ/NVFP4 target model. Use it with a compatible NVFP4 target in SGLang; it is not a standalone language model.

中文

本仓库是 MiniCPM-SALA 的 EAGLE3 draft 权重。该 draft 权重已做 NVFP4 量化，并且是为 MiniCPM-SALA GPTQ/NVFP4 target model 训练和配套使用的。它只能作为 SGLang speculative decoding 的 draft checkpoint 使用，不是独立语言模型。

Format

• NVFP4 draft attention/MLP weights.
• BF16 model.fc.weight and lm_head.weight.
• Fused qkv/gate_up weight and weight_scale tensors for SGLang GEMM modules.
• Split scalar input_scale and weight_scale_2 source keys:
  • model.midlayer.self_attn.{q,k,v}_proj.*
  • model.midlayer.mlp.{gate,up}_proj.*

No post-export split-scale conversion is required. The checkpoint is meant for the upstream-style SGLang EAGLE3 loader that uses stacked-parameter mapping.

格式说明

• draft attention/MLP 权重为 NVFP4。
• model.fc.weight 与 lm_head.weight 保持 BF16。
• qkv/gate_up 的 weight 与 weight_scale 为 SGLang GEMM 模块使用的 fused tensor。
• q/k/v 和 gate/up 的 input_scale、weight_scale_2 使用拆分后的 source key。

导出后不需要再做 split-scale 转换；权重目录可直接被当前 upstream-style SGLang EAGLE3 loader 加载。

Usage

python -m sglang.launch_server \
  --model-path /path/to/MiniCPM-SALA-NVFP4-target \
  --trust-remote-code \
  --quantization modelopt_fp4 \
  --fp4-gemm-backend flashinfer_cutlass \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path /path/to/this/repo

The tokenizer files are copied from the MiniCPM-SALA target model for loader compatibility.

使用方式

--model-path 应指向 MiniCPM-SALA NVFP4 target model；--speculative-draft-model-path 指向本 draft 权重目录。tokenizer 文件随 target 复制，仅用于加载兼容。

Training Metadata

The exported draft is from the MiniCPM-SALA EAGLE3 draft training pipeline with NVFP4 target-regenerated data, 32K draft vocabulary, three auxiliary target hidden layers [1, 10, 22], and ttt_steps_trained=7. See conversion_meta.json for the recorded checkpoint metadata.

训练元信息

训练数据由 MiniCPM-SALA NVFP4 target model 重新生成；draft vocab 为 32K，使用三层 target aux hidden [1, 10, 22]，ttt_steps_trained=7。精确导出信息见 conversion_meta.json。