CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production
This report presents CharacterFlywheel, an iterative flywheel process for improving large language models (LLMs) in production social chat applications across Instagram, WhatsApp, and Messenger. Starting from LLaMA 3.1, we refined models across 15 generations using data from both internal and external real-user traffic. Through continuous deployments from July 2024 to April 2025, we conducted controlled 7-day A/B tests showing consistent engagement improvements: 7 of 8 newly deployed models demonstrated positive lift over the baseline, with the strongest performers achieving up to 8.8% improvement in engagement breadth and 19.4% in engagement depth. We also observed substantial gains in steerability, with instruction following increasing from 59.2% to 84.8% and instruction violations decreasing from 26.6% to 5.8%. We detail the CharacterFlywheel process which integrates data curation, reward modeling to estimate and interpolate the landscape of engagement metrics, supervised fine-tuning (SFT), reinforcement learning (RL), and both offline and online evaluation to ensure reliable progress at each optimization step. We also discuss our methods for overfitting prevention and navigating production dynamics at scale. These contributions advance the scientific rigor and understanding of LLMs in social applications serving millions of users.
69a65f98663a122a7ce6673d
Yixin Nie
69a65f98663a122a7ce6673e
Lin Guan
69a65f98663a122a7ce6673f
Zhongyao Ma
69a65f98663a122a7ce66740
Anchit Gupta
69a65f98663a122a7ce66741
Yipin Zhou
69a65f98663a122a7ce66742
Xiao Li
69a65f98663a122a7ce66743
Zhengping Zhou
69a65f98663a122a7ce66744
Raymond Zeng
69a65f98663a122a7ce66745
Gelin Zhou
69a65f98663a122a7ce66746
Shigan Chu
69a65f98663a122a7ce66747
Ajay Thampi
claimed_verified
2026-03-05T09:59:10.036Z
6827926e0012a2e9f366481d
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zEJtseFJPg3z86b5j-THb.png
Ajay Thampi
ajaythampi
user
ajaythampi
69a65f98663a122a7ce66748
Wancen Mu
69a65f98663a122a7ce66749
Nathan Shuster
69a65f98663a122a7ce6674a
Ketong Wang
69a65f98663a122a7ce6674b
Lin Chen
69a65f98663a122a7ce6674c
Jason Brewer
69a65f98663a122a7ce6674d
Derek Hao Hu
69a65f98663a122a7ce6674e
Alexander McCauley
69a65f98663a122a7ce6674f
Jason Weston
69a65f98663a122a7ce66750
Sem Park
69a65f98663a122a7ce66751
Na Zhang
69a65f98663a122a7ce66752
Kevin Tang
CharacterFlywheel: Scaling Iterative Improvement of Engaging and Steerable LLMs in Production
This report presents CharacterFlywheel, an iterative flywheel process for improving large language models (LLMs) in production social chat applications across Instagram, WhatsApp, and Messenger. Starting from LLaMA 3.1, we refined models across 15 generations using data from both internal and external real-user traffic. Through continuous deployments from July 2024 to April 2025, we conducted controlled 7-day A/B tests showing consistent engagement improvements: 7 of 8 newly deployed models demonstrated positive lift over the baseline, with the strongest performers achieving up to 8.8% improvement in engagement breadth and 19.4% in engagement depth. We also observed substantial gains in steerability, with instruction following increasing from 59.2% to 84.8% and instruction violations decreasing from 26.6% to 5.8%. We detail the CharacterFlywheel process which integrates data curation, reward modeling to estimate and interpolate the landscape of engagement metrics, supervised fine-tuning (SFT), reinforcement learning (RL), and both offline and online evaluation to ensure reliable progress at each optimization step. We also discuss our methods for overfitting prevention and navigating production dynamics at scale. These contributions advance the scientific rigor and understanding of LLMs in social applications serving millions of users.
CharacterFlywheel is an iterative optimization process that enhances large language models for social chat applications through multiple generations of refinement, achieving significant improvements in user engagement and instruction following while maintaining production stability.
large language models supervised fine-tuning reinforcement learning reward modeling A/B testing engagement breadth engagement depth instruction following instruction violations overfitting prevention production dynamics