Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training
Training on verifiable symbolic data is a promising way to expand the reasoning frontier of language models beyond what standard pre-training corpora provide. Yet existing procedural generators often rely on fixed puzzles or templates and do not deliver the distributional breadth needed at scale. We introduce Reasoning Core, a scalable suite that procedurally generates verifiable symbolic reasoning data across core formal domains: PDDL planning over randomized domains, first-order logic with equality, context-free grammar parsing and generation, causal reasoning over random Bayesian networks, and systems of equations. Each task is paired with an external solver for rigorous verification and admits continuous difficulty control for curriculum design. Examples can optionally include solver-derived reasoning traces, enabling supervised training from the earliest pre-training stages, and the same interface provides verifiable reward functions for reinforcement learning. Our experiments show that mixing Reasoning Core data into pre-training improves downstream reasoning while preserving, or slightly improving, language modeling quality. Zero-shot evaluations confirm these tasks challenge frontier models such as GPT-5. The code and data are publicly available under the MIT license.
69a69d2c663a122a7ce6695e
Valentin Lacombe
69a69d2c663a122a7ce6695f
Valentin Quesnel
69a69d2c663a122a7ce66960
Damien Sileo
claimed_verified
2026-03-03T10:11:04.137Z
5fc0bcb41160c47d1d43856b
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc0bcb41160c47d1d43856b/AHCEW4TfTdyjNBx-V_F5A.png
Damien Sileo
sileod
user
sileod
Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training
Training on verifiable symbolic data is a promising way to expand the reasoning frontier of language models beyond what standard pre-training corpora provide. Yet existing procedural generators often rely on fixed puzzles or templates and do not deliver the distributional breadth needed at scale. We introduce Reasoning Core, a scalable suite that procedurally generates verifiable symbolic reasoning data across core formal domains: PDDL planning over randomized domains, first-order logic with equality, context-free grammar parsing and generation, causal reasoning over random Bayesian networks, and systems of equations. Each task is paired with an external solver for rigorous verification and admits continuous difficulty control for curriculum design. Examples can optionally include solver-derived reasoning traces, enabling supervised training from the earliest pre-training stages, and the same interface provides verifiable reward functions for reinforcement learning. Our experiments show that mixing Reasoning Core data into pre-training improves downstream reasoning while preserving, or slightly improving, language modeling quality. Zero-shot evaluations confirm these tasks challenge frontier models such as GPT-5. The code and data are publicly available under the MIT license.
Reasoning Core is a scalable system that generates verifiable symbolic reasoning data across multiple formal domains for training language models with enhanced reasoning capabilities.
verifiable symbolic data language models procedural generation PDDL planning first-order logic context-free grammar causal reasoning Bayesian networks systems of equations external solver curriculum design reinforcement learning zero-shot evaluation