Instructions to use Arki05/Grok-1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Arki05/Grok-1-GGUF with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Arki05/Grok-1-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Arki05/Grok-1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Arki05/Grok-1-GGUF:IQ1_M # Run inference directly in the terminal: llama cli -hf Arki05/Grok-1-GGUF:IQ1_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Arki05/Grok-1-GGUF:IQ1_M # Run inference directly in the terminal: llama cli -hf Arki05/Grok-1-GGUF:IQ1_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Arki05/Grok-1-GGUF:IQ1_M # Run inference directly in the terminal: ./llama-cli -hf Arki05/Grok-1-GGUF:IQ1_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Arki05/Grok-1-GGUF:IQ1_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Arki05/Grok-1-GGUF:IQ1_M
Use Docker
docker model run hf.co/Arki05/Grok-1-GGUF:IQ1_M
- LM Studio
- Jan
- Ollama
How to use Arki05/Grok-1-GGUF with Ollama:
ollama run hf.co/Arki05/Grok-1-GGUF:IQ1_M
- Unsloth Desktop
- Docker Model Runner
How to use Arki05/Grok-1-GGUF with Docker Model Runner:
docker model run hf.co/Arki05/Grok-1-GGUF:IQ1_M
- Lemonade
How to use Arki05/Grok-1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Arki05/Grok-1-GGUF:IQ1_M
Run and chat with the model
lemonade run user.Grok-1-GGUF-IQ1_M
List all available models
lemonade list
- Atomic Chat
IQ3_XS
#6
by Arki05 - opened
- grok-1-IQ3_XS-split-00001-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00002-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00003-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00004-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00005-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00006-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00007-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00008-of-00009.gguf +3 -0
- grok-1-IQ3_XS-split-00009-of-00009.gguf +3 -0
grok-1-IQ3_XS-split-00001-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:114f9c9b22429648f694e1993d7b5121a869acb56ce2fce37133064cbfa71f2f
|
| 3 |
+
size 17531846528
|
grok-1-IQ3_XS-split-00002-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac8b3768f544ff9049c99956aa556eaf77b717eca1dd79c3e8631e74c32090fb
|
| 3 |
+
size 15301623488
|
grok-1-IQ3_XS-split-00003-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:85d6a7faed876f832a567267b8746ab701e6dfd403e170f9d6ffb82ce2fc1b3d
|
| 3 |
+
size 15188377344
|
grok-1-IQ3_XS-split-00004-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f1406dcfbdd5caec6591a9d0ee4cabc0707b0bec04a47e0631e0b22e1cd77dcd
|
| 3 |
+
size 15625215744
|
grok-1-IQ3_XS-split-00005-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a6beb801fe4c592c831d675f27be58d40d54a996205e768b2c4efa02a18613f7
|
| 3 |
+
size 15428951840
|
grok-1-IQ3_XS-split-00006-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:906ebb35eb00b148bb172982535773768e444b955e89da8dbc682b3782040c9d
|
| 3 |
+
size 15197814528
|
grok-1-IQ3_XS-split-00007-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:66164f3bda7ed7d70bb7adb98183e178b804235595a5568a86e660a2b842a902
|
| 3 |
+
size 15188377344
|
grok-1-IQ3_XS-split-00008-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e3680cb383ae25525f58e01d1ef2309628d46593c2bc1a0e02470a1a9834597c
|
| 3 |
+
size 15991717632
|
grok-1-IQ3_XS-split-00009-of-00009.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:598e36389478d0f5ac56799cfff1846612220ab121cc32676b1bc5698003a9ae
|
| 3 |
+
size 4184830144
|