Qwen3-Coder-Next-oQ
Collection
5 items • Updated • 1
How to use from
PiConfigure the model in Pi
# Install Pi:
npm install -g @mariozechner/pi-coding-agent# Add to ~/.pi/agent/models.json:
{
"providers": {
"mlx-lm": {
"baseUrl": "http://localhost:8080/v1",
"api": "openai-completions",
"apiKey": "none",
"models": [
{
"id": "bearzi/Qwen3-Coder-Next-oQ4"
}
]
}
}
}Run Pi
# Start Pi in your project directory:
piQuick Links
Qwen3-Coder-Next-oQ4
oQ4 mixed-precision MLX quantization produced via oMLX.
- Quantization: oQ4 (sensitivity-driven mixed precision, group_size=64)
- Format: MLX safetensors
- Compatible with: mlx-lm, mlx-vlm, oMLX on Apple Silicon
Usage
from mlx_lm import load, generate
model, tokenizer = load("bearzi/Qwen3-Coder-Next-oQ4")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Hello"}],
add_generation_prompt=True,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))
About oQ
oQ measures per-layer quantization sensitivity through calibration and allocates bits where they matter most — critical layers stay at higher precision, tolerant layers compress aggressively. Target averages of 2/3/4/6/8 bits are provided; actual per-layer bits vary by measured sensitivity.
See oQ documentation.
Comparative benchmarks and feedback welcome — please open a discussion.
- Downloads last month
- 1,420
Model size
13B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
Model tree for bearzi/Qwen3-Coder-Next-oQ4
Base model
Qwen/Qwen3-Coder-Next
Start the MLX server
# Install MLX LM: uv tool install mlx-lm# Start a local OpenAI-compatible server: mlx_lm.server --model "bearzi/Qwen3-Coder-Next-oQ4"