Text Generation
GGUF
llama.cpp
conversational
on-device
mobile
korean
korean-llm
cpu
local-llm
edge
gemma
gemma4
mixture-of-experts
Mixture of Experts
pocket
vidraft
imatrix
Instructions to use FINAL-Bench/POCKET-26B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FINAL-Bench/POCKET-26B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama cli -hf FINAL-Bench/POCKET-26B-GGUF:Q2_K
Use Docker
docker model run hf.co/FINAL-Bench/POCKET-26B-GGUF:Q2_K
- Llama new
- LM Studio
- Jan
- vLLM
How to use FINAL-Bench/POCKET-26B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FINAL-Bench/POCKET-26B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FINAL-Bench/POCKET-26B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FINAL-Bench/POCKET-26B-GGUF:Q2_K
- Ollama
How to use FINAL-Bench/POCKET-26B-GGUF with Ollama:
ollama run hf.co/FINAL-Bench/POCKET-26B-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use FINAL-Bench/POCKET-26B-GGUF with Docker Model Runner:
docker model run hf.co/FINAL-Bench/POCKET-26B-GGUF:Q2_K
- Lemonade
How to use FINAL-Bench/POCKET-26B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FINAL-Bench/POCKET-26B-GGUF:Q2_K
Run and chat with the model
lemonade run user.POCKET-26B-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
POCKET family: add POCKET-Qwen3.8-Flash-Next (180B on a laptop), cross-link all 8 POCKET repos
Browse files
README.md
CHANGED
|
@@ -22,6 +22,9 @@ tags:
|
|
| 22 |
- pocket
|
| 23 |
- vidraft
|
| 24 |
---
|
|
|
|
|
|
|
|
|
|
| 25 |
> ### π **[POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)** β photoreal **images** in **46 s** on a **CPU only**. No GPU, no CUDA, no Python.
|
| 26 |
> [](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) [](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) []() <!-- POCKET-Zimage-CPU-BADGE -->
|
| 27 |
|
|
@@ -40,7 +43,7 @@ tags:
|
|
| 40 |
|
| 41 |
[](https://www.apache.org/licenses/LICENSE-2.0) [](https://github.com/ggml-org/llama.cpp) []() [](https://huggingface.co/google/gemma-4-26B-A4B-it)
|
| 42 |
|
| 43 |
-
**Pick your build β** [](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [ β Korean, Android
|
| 108 |
- π [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) β Korean, iPhone / Mac
|
| 109 |
- π [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) β English, phone / PC
|
|
|
|
| 110 |
- πΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) β character-perfect text in any image
|
|
|
|
| 111 |
|
| 112 |
**Demos & tools (Spaces)**
|
| 113 |
- π¨ [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio) β text-in-image, generate in-page
|
| 114 |
- π₯οΈ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) β 35B answering on a CPU
|
| 115 |
- π₯οΈ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β 26B on a CPU
|
|
|
|
| 116 |
|
| 117 |
π [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)
|
| 118 |
|
|
|
|
| 22 |
- pocket
|
| 23 |
- vidraft
|
| 24 |
---
|
| 25 |
+
> ### π **[POCKET-Qwen3.8-Flash-Next](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF)** β a **180B** model running on a **laptop with 8 GB VRAM + 32 GB RAM** Β· **4.17 tok/s measured**.
|
| 26 |
+
> [](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) []() []() []() <!-- POCKET-FLASHNEXT-BADGE -->
|
| 27 |
+
|
| 28 |
> ### π **[POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)** β photoreal **images** in **46 s** on a **CPU only**. No GPU, no CUDA, no Python.
|
| 29 |
> [](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) [](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) []() <!-- POCKET-Zimage-CPU-BADGE -->
|
| 30 |
|
|
|
|
| 43 |
|
| 44 |
[](https://www.apache.org/licenses/LICENSE-2.0) [](https://github.com/ggml-org/llama.cpp) []() [](https://huggingface.co/google/gemma-4-26B-A4B-it)
|
| 45 |
|
| 46 |
+
**Pick your build β** [](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) [](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) [](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) [](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) [](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU)
|
| 47 |
|
| 48 |
## Why this one?
|
| 49 |
|
|
|
|
| 110 |
- π°π· [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) β Korean, Android
|
| 111 |
- π [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) β Korean, iPhone / Mac
|
| 112 |
- π [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) β English, phone / PC
|
| 113 |
+
- π» [POCKET-Qwen3.8-Flash-Next-GGUF](https://huggingface.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF) β **180B on a laptop** (8 GB VRAM + 32 GB RAM)
|
| 114 |
- πΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) β character-perfect text in any image
|
| 115 |
+
- π₯οΈ [POCKET-Zimage-CPU](https://huggingface.co/FINAL-Bench/POCKET-Zimage-CPU) β photoreal images on a CPU only
|
| 116 |
|
| 117 |
**Demos & tools (Spaces)**
|
| 118 |
- π¨ [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio) β text-in-image, generate in-page
|
| 119 |
- π₯οΈ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) β 35B answering on a CPU
|
| 120 |
- π₯οΈ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β 26B on a CPU
|
| 121 |
+
- πΌοΈ [POCKET-Zimage-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-Zimage-CPU) β image generation on a CPU
|
| 122 |
|
| 123 |
π [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)
|
| 124 |
|