LoopViT: Scaling Visual ARC with Looped Transformers
Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
69817c5ece18b18628096138
Wen-Jie Shu
69817c5ece18b18628096139
Xuerui Qiu
claimed_verified
2026-02-03T10:06:35.669Z
66d98c2bc9758ee602cb2e22
/avatars/cea03f263d5ed292c4e558871112c61c.svg
Qiu
Xuerui123
user
Xuerui123
69817c5ece18b1862809613a
Rui-Jie Zhu
claimed_verified
2026-02-04T12:32:51.149Z
63ff09f24852102d4871c19c
https://cdn-avatars.huggingface.co/v1/production/uploads/63ff09f24852102d4871c19c/lyE3xemtZss3qebK5sEXw.png
Rui-Jie Zhu
ridger
user
ridger
69817c5ece18b1862809613b
Harold Haodong Chen
claimed_verified
2026-02-03T10:06:37.965Z
6570450a78d7aca0c361a177
https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/MX7jHhTQwLs-BvYIu5rqb.jpeg
Harold Chen
Harold328
user
Harold328
69817c5ece18b1862809613c
Yexin Liu
69817c5ece18b1862809613d
Harry Yang
LoopViT: Scaling Visual ARC with Looped Transformers
Recent advances in visual reasoning have leveraged vision transformers to tackle the ARC-AGI benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive entropy: the model halts inference when its internal state ``crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1 benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
Loop-ViT introduces a recursive vision transformer architecture that decouples reasoning depth from model capacity through weight-tied recurrence and dynamic exit mechanisms, achieving superior visual reasoning performance with fewer parameters.
vision transformers ARC-AGI benchmark feed-forward architecture weight-tied recurrence Hybrid Block local convolutions global attention latent chain of thought predictive entropy Dynamic Exit mechanism adaptive iterative computation