The Depth Ceiling: On the Limits of Large Language Models in Discovering Latent Planning
The viability of chain-of-thought (CoT) monitoring hinges on models being unable to reason effectively in their latent representations. Yet little is known about the limits of such latent reasoning in LLMs. We test these limits by studying whether models can discover multi-step planning strategies without supervision on intermediate steps and execute them latently, within a single forward pass. Using graph path-finding tasks that precisely control the number of required latent planning steps, we uncover a striking limitation unresolved by massive scaling: tiny transformers trained from scratch discover strategies requiring up to three latent steps, fine-tuned GPT-4o and Qwen3-32B reach five, and GPT-5.4 attains seven under few-shot prompting. Although the maximum latent planning depth models can learn during training is five, the discovered strategy generalizes up to eight latent steps at test-time. This reveals a dissociation between the ability to discover a latent strategy under final-answer supervision alone and the ability to execute it once discovered. If similar limits hold more broadly, strategies requiring multiple coordinated latent planning steps may need to be explicitly taught or externalized, lending credence to CoT monitoring.
69d71916c6ed9968ac5c69b0
Yi Xu
claimed_verified
2026-04-09T08:06:49.047Z
644d55bd53ad80c659395473
https://cdn-avatars.huggingface.co/v1/production/uploads/644d55bd53ad80c659395473/Jdff2RyL6ZGhFSY8kab1L.jpeg
Yi Xu
yixu1
user
yixu1
69d71916c6ed9968ac5c69b1
Philipp Jettkant
69d71916c6ed9968ac5c69b2
Laura Ruis
The Depth Ceiling: On the Limits of Large Language Models in Discovering Latent Planning
The viability of chain-of-thought (CoT) monitoring hinges on models being unable to reason effectively in their latent representations. Yet little is known about the limits of such latent reasoning in LLMs. We test these limits by studying whether models can discover multi-step planning strategies without supervision on intermediate steps and execute them latently, within a single forward pass. Using graph path-finding tasks that precisely control the number of required latent planning steps, we uncover a striking limitation unresolved by massive scaling: tiny transformers trained from scratch discover strategies requiring up to three latent steps, fine-tuned GPT-4o and Qwen3-32B reach five, and GPT-5.4 attains seven under few-shot prompting. Although the maximum latent planning depth models can learn during training is five, the discovered strategy generalizes up to eight latent steps at test-time. This reveals a dissociation between the ability to discover a latent strategy under final-answer supervision alone and the ability to execute it once discovered. If similar limits hold more broadly, strategies requiring multiple coordinated latent planning steps may need to be explicitly taught or externalized, lending credence to CoT monitoring.
Research reveals that large language models can perform latent reasoning with varying depths, but there's a gap between discovering and executing multi-step planning strategies, suggesting limitations for chain-of-thought monitoring.
chain-of-thought monitoring latent representations large language models multi-step planning strategies graph path-finding tasks latent planning depth few-shot prompting model generalization