Experimental global target bits‑per‑weight quantization of allenai/Olmo-3-7B-Instruct and allenai/Olmo-3-7B-Think
text
Experimental global target bits‑per‑weight quantization of allenai/Olmo-3-7B-Instruct and allenai/Olmo-3-7B-Think
new_line
new_line
Unlike standard llama.cpp quantizations that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global file size target.
text
Unlike standard llama.cpp quantizations that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global file size target.
new_line
new_line
Key Advantages:
text
Key Advantages:
new_line
- VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM).
text
- VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM).
new_line
- Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.
text
- Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.
new_line
new_line
Full benchmarks (PPL, KLD, ARC, MMLU, etc.) and methodology in the models' cards
text
Full benchmarks (PPL, KLD, ARC, MMLU, etc.) and methodology in the models' cards
new_line
new_line
https://huggingface.co/eaddario/Olmo-3-7B-Instruct-GGUF
eaddario/Olmo-3-7B-Instruct-GGUF
model
resource
https://huggingface.co/eaddario/Olmo-3-7B-Instruct-GGUF
new_line
https://huggingface.co/eaddario/Olmo-3-7B-Think-GGUF
eaddario/Olmo-3-7B-Think-GGUF
model
resource
https://huggingface.co/eaddario/Olmo-3-7B-Think-GGUF
Experimental global target bits‑per‑weight quantization of allenai/Olmo-3-7B-Instruct and allenai/Olmo-3-7B-Think

Unlike standard llama.cpp quantizations that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global file size target.

Key Advantages:
- VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM).
- Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.

Full benchmarks (PPL, KLD, ARC, MMLU, etc.) and methodology in the models' cards

https://huggingface.co/eaddario/Olmo-3-7B-Instruct-GGUF
https://huggingface.co/eaddario/Olmo-3-7B-Think-GGUF
🤗
unmodeled-tyler John6666 ddh0