Instructions to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama cli -hf Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Use Docker

docker model run hf.co/Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Llama new
LM Studio
Jan

vLLM

How to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Ollama
How to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with Ollama:
```
ollama run hf.co/Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16
```
Unsloth Desktop
Docker Model Runner
How to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with Docker Model Runner:
```
docker model run hf.co/Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16
```

Lemonade

How to use Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF:F16

Run and chat with the model

lemonade run user.SmolLM2-1.7B-telegram-chats-GGUF-F16

List all available models

lemonade list

Atomic Chat

SmolLM2-1.7B-telegram-chats — GGUF

GGUF quantizations of Offlin33er/SmolLM2-1.7B-telegram-chats — a full SFT of SmolLM2-1.7B on 3,000 synthetic Telegram Desktop export conversations, trained with TRL. The model writes chat histories in the Telegram Desktop export (result.json) format.

Live demo: Offlin33er/telegram-chat-generator runs the Q4_K_M file on a free CPU Space.

Files

File	Quant	Size
`smollm2-1.7b-telegram-chats-f16.gguf`	f16 (full precision)	3.4 GB
`smollm2-1.7b-telegram-chats-q4_k_m.gguf`	Q4_K_M	1.1 GB

Usage

Drop a file into LM Studio or any llama.cpp-based runner and prompt it with a Telegram-export style conversation. Typical use: generating synthetic chat data for fine-tuning or testing chat-rendering pipelines.

Training

SFT with TRL on Offlin33er/telegram-chat-export-synthetic — see the full-precision model card for the training procedure and framework versions.

Downloads last month: 72

GGUF

Model size

2B params

Architecture

llama

Hardware compatibility

4-bit

16-bit

Model tree for Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF

Base model

HuggingFaceTB/SmolLM2-1.7B

Quantized

(60)

this model

Offlin33er
/

SmolLM2-1.7B-telegram-chats-GGUF

SmolLM2-1.7B-telegram-chats — GGUF

Files

Usage

Training

Model tree for Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF

Space using Offlin33er/SmolLM2-1.7B-telegram-chats-GGUF 1