compar:IA: The French Government's LLM arena to collect French-language human prompts and preference data
Large Language Models (LLMs) often show reduced performance, cultural alignment, and safety robustness in non-English languages, partly because English dominates both pre-training data and human preference alignment datasets. Training methods like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) require human preference data, which remains scarce and largely non-public for many languages beyond English. To address this gap, we introduce compar:IA, an open-source digital public service developed inside the French government and designed to collect large-scale human preference data from a predominantly French-speaking general audience. The platform uses a blind pairwise comparison interface to capture unconstrained, real-world prompts and user judgments across a diverse set of language models, while maintaining low participation friction and privacy-preserving automated filtering. As of 2026-02-07, compar:IA has collected over 600,000 free-form prompts and 250,000 preference votes, with approximately 89% of the data in French. We release three complementary datasets -- conversations, votes, and reactions -- under open licenses, and present initial analyses, including a French-language model leaderboard and user interaction patterns. Beyond the French context, compar:IA is evolving toward an international digital public good, offering reusable infrastructure for multilingual model training, evaluation, and the study of human-AI interaction.
6989d925beecc443208d28bd
Lucie Termignon
6989d925beecc443208d28be
Simonas Zilinskas
claimed_verified
2026-02-09T14:31:15.121Z
649d986a474bf415c03b772c
https://cdn-avatars.huggingface.co/v1/production/uploads/649d986a474bf415c03b772c/hcBoKXvrQHuXnaiGH1YOV.jpeg
Simonas Zilinskas
monsimas
user
monsimas
6989d925beecc443208d28bf
Hadrien Pélissier
6989d925beecc443208d28c0
Aurélien Barrot
6989d925beecc443208d28c1
Nicolas Chesnais
6989d925beecc443208d28c2
Elie Gavoty
compar:IA: The French Government's LLM arena to collect French-language human prompts and preference data
Large Language Models (LLMs) often show reduced performance, cultural alignment, and safety robustness in non-English languages, partly because English dominates both pre-training data and human preference alignment datasets. Training methods like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) require human preference data, which remains scarce and largely non-public for many languages beyond English. To address this gap, we introduce compar:IA, an open-source digital public service developed inside the French government and designed to collect large-scale human preference data from a predominantly French-speaking general audience. The platform uses a blind pairwise comparison interface to capture unconstrained, real-world prompts and user judgments across a diverse set of language models, while maintaining low participation friction and privacy-preserving automated filtering. As of 2026-02-07, compar:IA has collected over 600,000 free-form prompts and 250,000 preference votes, with approximately 89% of the data in French. We release three complementary datasets -- conversations, votes, and reactions -- under open licenses, and present initial analyses, including a French-language model leaderboard and user interaction patterns. Beyond the French context, compar:IA is evolving toward an international digital public good, offering reusable infrastructure for multilingual model training, evaluation, and the study of human-AI interaction.
Compar:IA is an open-source platform that collects large-scale human preference data for multilingual language model training and evaluation, featuring a blind pairwise comparison interface and releasing three datasets under open licenses.
Reinforcement Learning from Human Feedback Direct Preference Optimization human preference data language models multilingual model training human-AI interaction