Universal Reasoning Model
Universal transformers (UTs) have been widely used for complex reasoning tasks such as ARC-AGI and Sudoku, yet the specific sources of their performance gains remain underexplored. In this work, we systematically analyze UTs variants and show that improvements on ARC-AGI primarily arise from the recurrent inductive bias and strong nonlinear components of Transformer, rather than from elaborate architectural designs. Motivated by this finding, we propose the Universal Reasoning Model (URM), which enhances the UT with short convolution and truncated backpropagation. Our approach substantially improves reasoning performance, achieving state-of-the-art 53.8% pass@1 on ARC-AGI 1 and 16.0% pass@1 on ARC-AGI 2. Our code is avaliable at https://github.com/zitian-gao/URM.
6942c644fb33037a39577c15
Zitian Gao
claimed_verified
2025-12-18T08:20:55.038Z
641ddac5be3bd3a5a06ed4a4
/avatars/14969dff861d53b0a75305606495eca7.svg
Zitian Gao
zgao3186
user
zgao3186
6942c644fb33037a39577c16
Lynx Chen
6942c644fb33037a39577c17
Yihao Xiao
6942c644fb33037a39577c18
He Xing
6942c644fb33037a39577c19
Ran Tao
6942c644fb33037a39577c1a
Haoming Luo
6942c644fb33037a39577c1b
Joey Zhou
6942c644fb33037a39577c1c
Bryan Dai
Universal Reasoning Model
Universal transformers (UTs) have been widely used for complex reasoning tasks such as ARC-AGI and Sudoku, yet the specific sources of their performance gains remain underexplored. In this work, we systematically analyze UTs variants and show that improvements on ARC-AGI primarily arise from the recurrent inductive bias and strong nonlinear components of Transformer, rather than from elaborate architectural designs. Motivated by this finding, we propose the Universal Reasoning Model (URM), which enhances the UT with short convolution and truncated backpropagation. Our approach substantially improves reasoning performance, achieving state-of-the-art 53.8% pass@1 on ARC-AGI 1 and 16.0% pass@1 on ARC-AGI 2. Our code is avaliable at https://github.com/zitian-gao/URM.
The Universal Reasoning Model enhances Universal Transformers with short convolution and truncated backpropagation to improve reasoning performance on ARC-AGI tasks.
Universal Transformers ARC-AGI recurrent inductive bias nonlinear components truncated backpropagation Universal Reasoning Model