DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing
Reinforcement learning (RL)-based enhancement of large language models (LLMs) often leads to reduced output diversity, undermining their utility in open-ended tasks like creative writing. Current methods lack explicit mechanisms for guiding diverse exploration and instead prioritize optimization efficiency and performance over diversity. This paper proposes an RL framework structured around a semi-structured long Chain-of-Thought (CoT), in which the generation process is decomposed into explicitly planned intermediate steps. We introduce a Diverse Planning Branching method that strategically introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. Experimental results on creative writing benchmarks demonstrate that our approach significantly improves output diversity without compromising generation quality, consistently outperforming existing baselines.
696913db8854afb90e2fc9d7
Qian Cao
claimed_verified
2026-01-27T09:44:07.906Z
5fbdf878485ef14d9a960f4d
https://cdn-avatars.huggingface.co/v1/production/uploads/1634724388998-5fbdf878485ef14d9a960f4d.jpeg
Qian Cao
Aman
user
Aman
696913db8854afb90e2fc9d8
Yahui Liu
696913db8854afb90e2fc9d9
Wei Bi
696913db8854afb90e2fc9da
Yi Zhao
696913db8854afb90e2fc9db
Ruihua Song
696913db8854afb90e2fc9dc
Xiting Wang
696913db8854afb90e2fc9dd
Ruiming Tang
696913db8854afb90e2fc9de
Guorui Zhou
696913db8854afb90e2fc9df
Han Li
DPWriter: Reinforcement Learning with Diverse Planning Branching for Creative Writing
Reinforcement learning (RL)-based enhancement of large language models (LLMs) often leads to reduced output diversity, undermining their utility in open-ended tasks like creative writing. Current methods lack explicit mechanisms for guiding diverse exploration and instead prioritize optimization efficiency and performance over diversity. This paper proposes an RL framework structured around a semi-structured long Chain-of-Thought (CoT), in which the generation process is decomposed into explicitly planned intermediate steps. We introduce a Diverse Planning Branching method that strategically introduces divergence at the planning phase based on diversity variation, alongside a group-aware diversity reward to encourage distinct trajectories. Experimental results on creative writing benchmarks demonstrate that our approach significantly improves output diversity without compromising generation quality, consistently outperforming existing baselines.
A reinforcement learning framework with diverse planning branching and group-aware rewards enhances large language model output diversity in creative writing tasks.
reinforcement learning large language models Chain-of-Thought diverse planning branching group-aware diversity reward creative writing benchmarks