RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies
Testing on real machines is indispensable for robotic control algorithms. In the context of learning-based algorithms, especially VLA models, demand for large-scale evaluation, i.e. testing a large number of models on a large number of tasks, is becoming increasingly urgent. However, doing this right is highly non-trivial, especially when scalability and reproducibility is taken into account. In this report, we describe our methodology for constructing RoboChallenge, an online evaluation system to test robotic control algorithms, and our survey of recent state-of-the-art VLA models using our initial benchmark Table30.
6902d14272739622ee92a825
Adina Yakefu
admin_assigned
2025-11-05T15:56:08.834Z
63a369d98c0c89dcae3b8329
https://cdn-avatars.huggingface.co/v1/production/uploads/63a369d98c0c89dcae3b8329/AiH2zjy1cnt9OADAAZMLD.jpeg
Adina Yakefu
AdinaY
user
AdinaY
6902d14272739622ee92a826
Bin Xie
6902d14272739622ee92a827
Chongyang Xu
6902d14272739622ee92a828
Enwen Zhang
6902d14272739622ee92a829
Erjin Zhou
admin_assigned
2025-11-05T16:33:52.794Z
63f5bca33aa49d8cb97ff689
/avatars/d6649c9413f32dbf4477126d0610432e.svg
Erjin Zhou
zej
user
zej
6902d14272739622ee92a82a
Fan Jia
6902d14272739622ee92a82b
Haitao Yang
6902d14272739622ee92a82c
Haoqiang Fan
6902d14272739622ee92a82d
Haowei Zhang
6902d14272739622ee92a82e
Hongyang Peng
admin_assigned
2025-11-05T16:34:53.102Z
66b5a9bb8d9afb7a9d6dc9d3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/XsfkoY58DhcY-YxzbuKKn.png
Hongyang Peng
LoveEatCandy
user
LoveEatCandy
6902d14272739622ee92a82f
Jing Tan
6902d14272739622ee92a830
Junwen Huang
admin_assigned
2025-11-05T16:35:03.363Z
65672681e7be83d499809c98
/avatars/efcde7a86a47a7f7a2b26847f41c214d.svg
Junwen Huang
ssdqpbss
user
ssdqpbss
6902d14272739622ee92a831
Kai Liu
6902d14272739622ee92a832
Kaixin Liu
admin_assigned
2025-11-05T16:35:10.805Z
6375e0d73b13bf69cb3b0ec9
/avatars/99993f27d18448d72e4fef5e2ce80529.svg
Kaixin Liu
smrset
user
smrset
6902d14272739622ee92a833
Kefan Gu
6902d14272739622ee92a834
Qinglun Zhang
6902d14272739622ee92a835
Ruitao Zhang
admin_assigned
2025-11-05T16:35:29.397Z
6548d99db4c5d07feb83cfd8
/avatars/a34a20efda0695bdb830ea4733c2bc73.svg
Ruitao Zhang
Ruitao123
user
Ruitao123
6902d14272739622ee92a836
Saike Huang
6902d14272739622ee92a837
Shen Cheng
6902d14272739622ee92a838
Shuaicheng Liu
6902d14272739622ee92a839
Tiancai Wang
6902d14272739622ee92a83a
Tiezhen Wang
admin_assigned
2025-11-05T16:35:51.928Z
62d22496c58f969c152bcefd
/avatars/76c3b70e312f25e1e610473475553c5c.svg
Tiezhen WANG
xianbao
user
xianbao
6902d14272739622ee92a83b
Wei Sun
6902d14272739622ee92a83c
Wenbin Tang
6902d14272739622ee92a83d
Yajun Wei
6902d14272739622ee92a83e
Yang Chen
6902d14272739622ee92a83f
Youqiang Gui
6902d14272739622ee92a840
Yucheng Zhao
6902d14272739622ee92a841
Yunchao Ma
6902d14272739622ee92a842
Yunfei Wei
6902d14272739622ee92a843
Yunhuan Yang
6902d14272739622ee92a844
Yutong Guo
6902d14272739622ee92a845
Ze Chen
6902d14272739622ee92a846
Zhengyuan Du
6902d14272739622ee92a847
Ziheng Zhang
6902d14272739622ee92a848
Ziming Liu
6902d14272739622ee92a849
Ziwei Yan
RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies
Testing on real machines is indispensable for robotic control algorithms. In the context of learning-based algorithms, especially VLA models, demand for large-scale evaluation, i.e. testing a large number of models on a large number of tasks, is becoming increasingly urgent. However, doing this right is highly non-trivial, especially when scalability and reproducibility is taken into account. In this report, we describe our methodology for constructing RoboChallenge, an online evaluation system to test robotic control algorithms, and our survey of recent state-of-the-art VLA models using our initial benchmark Table30.
RoboChallenge is an online evaluation system designed for testing robotic control algorithms, particularly focusing on large-scale assessment of vision-language-action models while addressing challenges related to scalability and reproducibility.
robotic control algorithms vision-language-action models large-scale evaluation online evaluation system reproducibility scalability