DramaBench: A Six-Dimensional Evaluation Framework for Drama Script Continuation
Drama script continuation requires models to maintain character consistency, advance plot coherently, and preserve dramatic structurecapabilities that existing benchmarks fail to evaluate comprehensively. We present DramaBench, the first large-scale benchmark for evaluating drama script continuation across six independent dimensions: Format Standards, Narrative Efficiency, Character Consistency, Emotional Depth, Logic Consistency, and Conflict Handling. Our framework combines rulebased analysis with LLM-based labeling and statistical metrics, ensuring objective and reproducible evaluation. We conduct comprehensive evaluation of 8 state-of-the-art language models on 1,103 scripts (8,824 evaluations total), with rigorous statistical significance testing (252 pairwise comparisons, 65.9% significant) and human validation (188 scripts, substantial agreement on 3/5 dimensions). Our ablation studies confirm all six dimensions capture independent quality aspects (mean | r | = 0.020). DramaBench provides actionable, dimensionspecific feedback for model improvement and establishes a rigorous standard for creative writing evaluation.
694a9fa4d8d7445f420c1701
Shijian Ma
claimed_verified
2025-12-25T20:46:59.489Z
6457709270b1337109951a7f
https://cdn-avatars.huggingface.co/v1/production/uploads/6457709270b1337109951a7f/sBWkRG_UCDN1FN5tyPYht.jpeg
MaShijian
FutureMa
user
FutureMa
694a9fa4d8d7445f420c1702
Yunqi Huang
694a9fa4d8d7445f420c1703
Yan Lin
claimed_verified
2025-12-25T20:47:02.236Z
651e313af20b31b94f47b1bf
/avatars/67a4a971aa198397a165c4f1af8332f5.svg
Yan LIN
jackylin2012
user
jackylin2012
DramaBench: A Six-Dimensional Evaluation Framework for Drama Script Continuation
Drama script continuation requires models to maintain character consistency, advance plot coherently, and preserve dramatic structurecapabilities that existing benchmarks fail to evaluate comprehensively. We present DramaBench, the first large-scale benchmark for evaluating drama script continuation across six independent dimensions: Format Standards, Narrative Efficiency, Character Consistency, Emotional Depth, Logic Consistency, and Conflict Handling. Our framework combines rulebased analysis with LLM-based labeling and statistical metrics, ensuring objective and reproducible evaluation. We conduct comprehensive evaluation of 8 state-of-the-art language models on 1,103 scripts (8,824 evaluations total), with rigorous statistical significance testing (252 pairwise comparisons, 65.9% significant) and human validation (188 scripts, substantial agreement on 3/5 dimensions). Our ablation studies confirm all six dimensions capture independent quality aspects (mean | r | = 0.020). DramaBench provides actionable, dimensionspecific feedback for model improvement and establishes a rigorous standard for creative writing evaluation.
DramaBench is a comprehensive benchmark for evaluating drama script continuation across six dimensions using a combination of rule-based analysis and LLM-based labeling.
LLM-based labeling rule-based analysis statistical metrics human validation ablation studies