Scaling Latent Reasoning via Looped Language Models
Modern LLMs are trained to "think" primarily via explicit text generation,
such as chain-of-thought (CoT), which defers reasoning to post-training and
under-leverages pre-training data. We present and open-source Ouro, named after
the recursive Ouroboros, a family of pre-trained Looped Language Models
(LoopLM) that instead build reasoning into the pre-training phase through (i)
iterative computation in latent space, (ii) an entropy-regularized objective
for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and
2.6B models enjoy superior performance that match the results of up to 12B SOTA
LLMs across a wide range of benchmarks. Through controlled experiments, we show
this advantage stems not from increased knowledge capacity, but from superior
knowledge manipulation capabilities. We also show that LoopLM yields reasoning
traces more aligned with final outputs than explicit CoT. We hope our results
show the potential of LoopLM as a novel scaling direction in the reasoning era.
Our model could be found in: http://ouro-llm.github.io.
6902c3f772739622ee92a6e8
Rui-Jie Zhu
claimed_verified
2025-10-30T14:29:11.604Z
63ff09f24852102d4871c19c
https://cdn-avatars.huggingface.co/v1/production/uploads/63ff09f24852102d4871c19c/lyE3xemtZss3qebK5sEXw.png
Rui-Jie Zhu
ridger
user
ridger
6902c3f772739622ee92a6e9
Zixuan Wang
6902c3f772739622ee92a6ea
Kai Hua
claimed_verified
2025-11-03T20:57:07.501Z
64e851825ddcace745ba15bd
/avatars/7b6612c411222974d9ea181784eef915.svg
Kai Hua
kkish
user
kkish
6902c3f772739622ee92a6eb
Tianyu Zhang
claimed_verified
2025-10-30T14:28:53.839Z
6452d79149b6b9a2383b5775
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/T28lP0kE7PZIGzJjhSpSx.jpeg
Tianyu Zhang
TianyuZhang
user
TianyuZhang
6902c3f772739622ee92a6ec
Ziniu Li
claimed_verified
2025-11-03T20:57:10.847Z
647c4c901f878439e2fd34d6
/avatars/b4399d210d7239d4662b11a4ee7b527d.svg
Ziniu Li
ziniuli
user
ziniuli
6902c3f772739622ee92a6ed
Haoran Que
6902c3f772739622ee92a6ee
Boyi Wei
claimed_verified
2025-10-31T14:27:59.712Z
65319bd7f85995389d4f019c
/avatars/657858b8435b220c9a29918c0dae9c6d.svg
Boyi Wei
boyiwei
user
boyiwei
6902c3f772739622ee92a6ef
Zixin Wen
6902c3f772739622ee92a6f0
Fan Yin
6902c3f772739622ee92a6f1
He Xing
6902c3f772739622ee92a6f2
Lu Li
claimed_verified
2025-10-30T14:28:29.408Z
658fbec068d0b76331a81e38
https://cdn-avatars.huggingface.co/v1/production/uploads/658fbec068d0b76331a81e38/E-1fyTCM4eltkWOXwWU5k.jpeg
Lu Li
luli2949
user
luli2949
6902c3f772739622ee92a6f3
Jiajun Shi
6902c3f772739622ee92a6f4
Kaijing Ma
6902c3f772739622ee92a6f5
Shanda Li
6902c3f772739622ee92a6f6
Taylor Kergan
6902c3f772739622ee92a6f7
Andrew Smith
6902c3f772739622ee92a6f8
Xingwei Qu
6902c3f772739622ee92a6f9
Mude Hui
6902c3f772739622ee92a6fa
Bohong Wu
6902c3f772739622ee92a6fb
Qiyang Min
claimed_verified
2025-11-21T09:02:43.604Z
645507b89d37c3fb33279fe3
/avatars/ec1122137c49204ab968182d1f726c35.svg
min
qiyang-attn
user
qiyang-attn
6902c3f772739622ee92a6fc
Hongzhi Huang
6902c3f772739622ee92a6fd
Xun Zhou
6902c3f772739622ee92a6fe
Wei Ye
6902c3f772739622ee92a6ff
Jiaheng Liu
6902c3f772739622ee92a700
Jian Yang
6902c3f772739622ee92a701
Yunfeng Shi
6902c3f772739622ee92a702
Chenghua Lin
6902c3f772739622ee92a703
Enduo Zhao
6902c3f772739622ee92a704
Tianle Cai
6902c3f772739622ee92a705
Ge Zhang
claimed_verified
2025-10-30T14:29:37.002Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
6902c3f772739622ee92a706
Wenhao Huang
6902c3f772739622ee92a707
Yoshua Bengio
6902c3f772739622ee92a708
Jason Eshraghian
Scaling Latent Reasoning via Looped Language Models
Modern LLMs are trained to "think" primarily via explicit text generation,
such as chain-of-thought (CoT), which defers reasoning to post-training and
under-leverages pre-training data. We present and open-source Ouro, named after
the recursive Ouroboros, a family of pre-trained Looped Language Models
(LoopLM) that instead build reasoning into the pre-training phase through (i)
iterative computation in latent space, (ii) an entropy-regularized objective
for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and
2.6B models enjoy superior performance that match the results of up to 12B SOTA
LLMs across a wide range of benchmarks. Through controlled experiments, we show
this advantage stems not from increased knowledge capacity, but from superior
knowledge manipulation capabilities. We also show that LoopLM yields reasoning
traces more aligned with final outputs than explicit CoT. We hope our results
show the potential of LoopLM as a novel scaling direction in the reasoning era.
Our model could be found in: http://ouro-llm.github.io.
LoopLM, a family of pre-trained Looped Language Models, enhances reasoning by integrating iterative computation and entropy regularization during pre-training, achieving superior performance with better knowledge manipulation.
Looped Language Models LoopLM iterative computation latent space entropy-regularized objective learned depth allocation chain-of-thought knowledge manipulation reasoning traces