Instructions to use KIEFERSA/Qwen3.5-0.8B-Greek with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KIEFERSA/Qwen3.5-0.8B-Greek with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="KIEFERSA/Qwen3.5-0.8B-Greek") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("KIEFERSA/Qwen3.5-0.8B-Greek") model = AutoModelForMultimodalLM.from_pretrained("KIEFERSA/Qwen3.5-0.8B-Greek", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use KIEFERSA/Qwen3.5-0.8B-Greek with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "KIEFERSA/Qwen3.5-0.8B-Greek" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KIEFERSA/Qwen3.5-0.8B-Greek", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/KIEFERSA/Qwen3.5-0.8B-Greek
- SGLang
How to use KIEFERSA/Qwen3.5-0.8B-Greek with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "KIEFERSA/Qwen3.5-0.8B-Greek" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KIEFERSA/Qwen3.5-0.8B-Greek", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "KIEFERSA/Qwen3.5-0.8B-Greek" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KIEFERSA/Qwen3.5-0.8B-Greek", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use KIEFERSA/Qwen3.5-0.8B-Greek with Docker Model Runner:
docker model run hf.co/KIEFERSA/Qwen3.5-0.8B-Greek
Qwen3.5-0.8B-Greek was made for the PyCon workshop. It is Qwen/Qwen3.5-0.8B fine-tuned for natural Greek chat and long conversations. Because of its small size, it has some limits in longer conversations (it can lose track of details or repeat itself); this is a limit of the model size.
It keeps the vision capabilities of Qwen3.5-0.8B: it also understands images and video.
- Creator: Kiefer SA
- Base model: Qwen3.5-0.8B
- Language: Greek
- GGUF (llama.cpp): KIEFERSA/Qwen3.5-0.8B-Greek-GGUF (BF16, Q8_0, Q6_K, Q4_K_M)
MTP (self-drafting)
The model keeps its Multi-Token Prediction (MTP) layer, so it can draft its own next tokens for faster generation (speculative decoding, no extra draft model). The MTP layer was retrained on our chat data so its drafts match the fine-tuned model:
vllm serve KIEFERSA/Qwen3.5-0.8B-Greek --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'
vLLM, 40 Greek prompts, greedy decoding, 2 draft tokens per step.
| MTP | Original MTP layer | Retrained MTP layer |
|---|---|---|
| Acceptance rate | 32.3% | 58.4% |
| 1st draft token accepted | 47.1% | 79.0% |
| 2nd draft token accepted | 17.5% | 37.9% |
| Tokens per step | 1.65 | 2.17 |
Evaluation
Greek benchmarks, 0-shot, accuracy (%). Qwen3.5-0.8B-Greek highlighted.
| Benchmark | Qwen3.5-0.8B (base) | Qwen3.5-0.8B-Greek |
|---|---|---|
| Average (all 6) | 34.83 | 40.23 |
| HellaSwag | 32.04 | 33.58 |
| Medical MCQA | 19.68 | 28.94 |
| ARC-Challenge | 34.25 | 44.35 |
| Belebele | 48.22 | 52.78 |
| TruthfulQA | 24.11 | 31.21 |
Forgetting check
Benchmarks where the fine-tuned model scores slightly below the base model, kept here to track forgetting.
| Benchmark | Qwen3.5-0.8B (base) | Qwen3.5-0.8B-Greek |
|---|---|---|
| Winogrande | 50.67 | 50.51 |
GreekMMLU
| GreekMMLU | Qwen3.5-0.8B (base) | Qwen3.5-0.8B-Greek |
|---|---|---|
| Overall | 40.98 | 46.55 |
| Humanities | 41.35 | 46.05 |
| Art professional | 48.33 | 52.00 |
| Art secondary school | 30.77 | 20.51 |
| Art university | 58.33 | 58.33 |
| Greek history primary school | 36.64 | 43.32 |
| Greek history professional | 25.64 | 31.62 |
| Greek history secondary school | 38.71 | 45.16 |
| Greek literature | 35.71 | 42.86 |
| Greek mythology | 37.39 | 44.96 |
| Greek traditions | 45.48 | 48.14 |
| Prehistory | 55.56 | 63.49 |
| World history | 35.00 | 45.00 |
| World religions | 34.84 | 38.06 |
| Social sciences | 40.53 | 47.13 |
| Accounting | 35.33 | 38.59 |
| Economics professional | 30.61 | 42.86 |
| Economics university | 54.65 | 65.12 |
| Education professional | 52.17 | 51.78 |
| Education university | 31.71 | 24.39 |
| Geography primary school | 39.44 | 51.39 |
| Geography secondary school | 37.10 | 33.87 |
| Government and politics primary school | 57.14 | 76.07 |
| Government and politics secondary school | 38.67 | 44.00 |
| Law | 28.63 | 34.51 |
| Management professional | 51.79 | 54.76 |
| Management university | 52.00 | 56.00 |
| Modern greek language primary school | 35.86 | 40.60 |
| Modern greek language secondary school | 47.05 | 56.70 |
| Stem | 40.56 | 46.53 |
| Agriculture professional | 43.95 | 49.26 |
| Agriculture university | 38.57 | 42.86 |
| Biology | 28.47 | 38.04 |
| Chemistry | 33.33 | 39.51 |
| Civil engineering | 46.81 | 49.28 |
| Clinical knowledge | 44.08 | 47.87 |
| Computer networks and security | 33.72 | 33.72 |
| Computer science professional | 59.84 | 71.65 |
| Computer science university | 34.62 | 42.31 |
| Electrical engineering | 45.11 | 46.01 |
| Mathematics | 31.75 | 37.57 |
| Medicine professional | 42.42 | 43.51 |
| Medicine university | 34.72 | 41.67 |
| Physics primary school | 39.34 | 65.57 |
| Physics professional | 41.92 | 46.88 |
| Physics university | 33.80 | 33.80 |
| Other | 43.07 | 45.62 |
| Driving rules | 44.15 | 45.72 |
| General knowledge | 36.47 | 45.58 |
| Qa maritime safety and rescue operations | 46.62 | 44.59 |
License
Inherits the Qwen3.5-0.8B base-model license (Apache 2.0).
- Downloads last month
- 19