# Faust-1 — German-First Large Language Model (1.6B)
91
92
-Faust-1 is a **German-first large language model** with **1.6B parameters**, *trained entirely from scratch* without initialization from existing multilingual checkpoints. Model development comprises **large-scale data collection and synthetic data generation**, followed by **systematic data cleaning, normalization, and deduplication** to reduce contamination and redundancy. **Pre-training** is performed on a *predominantly German corpus* using a **decoder-only language modeling objective**, yielding a **general-purpose foundation model for the German language** that captures lexical, syntactic, and semantic regularities at scale.
93
94
-Following pre-training, the model undergoes **supervised post-training** (*instruction tuning*) using **labeled input–output pairs** to specialize the base model for *conversational and task-oriented settings*. In later stages, **preference-based optimization**—including *Direct Preference Optimization (DPO)*—is applied to improve **response quality, stability, and alignment with human expectations**, while preserving the **efficiency constraints required for small-scale and local deployment**.
95
96
---
97
89
90
# Faust-1 — German-First Large Language Model (1.6B)
91
92
+Faust-1 is a German-first large language model with 1.6B parameters, trained entirely from scratch. Model development comprises large-scale data collection and synthetic data generation, followed by data cleaning, normalization, and deduplication to reduce contamination and redundancy. Pre-training is performed on a predominantly German corpus using a decoder-only language modeling objective, resulting in a foundation model for the German language that captures lexical, syntactic, and semantic regularities at scale.
93
94
+Following pre-training, the model undergoes supervised post-training (instruction tuning) using labeled input–output pairs to adapt the base model for conversational and task-oriented use. In later stages, preference-based optimization, including Direct Preference Optimization (DPO), is applied to improve response quality, stability, and alignment with human expectations, while preserving the efficiency constraints required for small-scale and local deployment.
