Where is the gguf?
#1
by OracleToes - opened
The Qwen3.6-27B-PRISM-PRO-DQ page references this repo, saying to load the gguf of the eagle-3 drafter, but I don't see any gguf to download. Additionally there is no one-shot llama.cpp patch, there is only a patch for sglang
# 3. EAGLE-3 chain (needs the WIP PR #18039 patches + the RS-rollback fix --
# a one-shot llama.cpp patch script is documented alongside the drafter:
# https://huggingface.co/Ex0bit/Qwen3.6-27B-PRISM-EAGLE3)
./llama-server --model Qwen3.6-27B-PRISM-PRO-DQ.gguf \
--spec-type draft-eagle3 --model-draft <eagle3-drafter.gguf> \
--spec-draft-n-max 2
I can upload one but llama.cpp does not support eagle-3 yet - you can easily convert the tensors to GGUF using llama.cpp convert
can't wait to try Qwen3.6-27B_Q8 with eagle-3 on llama.cpp ❤︎
I generated some to investigate a llama.cpp issue that was not anymore: gelim/Qwen3.6-27B-PRISM-EAGLE3-GGUF
Not worth yet to replace unsloth's MTP GGUF