Target-Bench: Can World Models Achieve Mapless Path Planning with Semantic Targets?
While recent world models generate highly realistic videos, their ability to perform robot path planning remains unclear and unquantified. We introduce Target-Bench, the first benchmark specifically designed to evaluate world models on mapless path planning toward semantic targets in real-world environments. Target-Bench provides 450 robot-collected video sequences spanning 45 semantic categories with SLAM-based ground truth trajectories. Our evaluation pipeline recovers camera motion from generated videos and measures planning performance using five complementary metrics that quantify target-reaching capability, trajectory accuracy, and directional consistency. We evaluate state-of-the-art models including Sora 2, Veo 3.1, and the Wan series. The best off-the-shelf model (Wan2.2-Flash) achieves only 0.299 overall score, revealing significant limitations in current world models for robotic planning tasks. We show that fine-tuning an open-source 5B-parameter model on only 325 scenarios from our dataset achieves 0.345 overall score -- an improvement of more than 400% over its base version (0.066) and 15% higher than the best off-the-shelf model. We will open-source the code and dataset.
692529de16eb3a9f131039be
Dingrui Wang
692529de16eb3a9f131039bf
Hongyuan Ye
692529de16eb3a9f131039c0
Zhihao Liang
692529de16eb3a9f131039c1
Zhexiao Sun
692529de16eb3a9f131039c2
Zhaowei Lu
692529de16eb3a9f131039c3
Yuchen Zhang
692529de16eb3a9f131039c4
Yuyu Zhao
692529de16eb3a9f131039c5
Yuan Gao
claimed_verified
2025-12-01T16:24:52.313Z
672288dd7055eec76d377268
https://cdn-avatars.huggingface.co/v1/production/uploads/672288dd7055eec76d377268/bBQ51VwKZJIkHkNffuEuJ.jpeg
Yuan Gao
Yuan-avs
user
Yuan-avs
692529de16eb3a9f131039c6
Marvin Seegert
692529de16eb3a9f131039c7
Finn Schäfer
692529de16eb3a9f131039c8
Haotong Qin
692529de16eb3a9f131039c9
Wei Li
692529de16eb3a9f131039ca
Luigi Palmieri
692529de16eb3a9f131039cb
Felix Jahncke
692529de16eb3a9f131039cc
Mattia Piccinini
692529de16eb3a9f131039cd
Johannes Betz
Target-Bench: Can World Models Achieve Mapless Path Planning with Semantic Targets?
While recent world models generate highly realistic videos, their ability to perform robot path planning remains unclear and unquantified. We introduce Target-Bench, the first benchmark specifically designed to evaluate world models on mapless path planning toward semantic targets in real-world environments. Target-Bench provides 450 robot-collected video sequences spanning 45 semantic categories with SLAM-based ground truth trajectories. Our evaluation pipeline recovers camera motion from generated videos and measures planning performance using five complementary metrics that quantify target-reaching capability, trajectory accuracy, and directional consistency. We evaluate state-of-the-art models including Sora 2, Veo 3.1, and the Wan series. The best off-the-shelf model (Wan2.2-Flash) achieves only 0.299 overall score, revealing significant limitations in current world models for robotic planning tasks. We show that fine-tuning an open-source 5B-parameter model on only 325 scenarios from our dataset achieves 0.345 overall score -- an improvement of more than 400% over its base version (0.066) and 15% higher than the best off-the-shelf model. We will open-source the code and dataset.
Target-Bench evaluates state-of-the-shelf world models on mapless path planning tasks, showing significant improvement through fine-tuning.
world models Target-Bench robot-collected video sequences SLAM-based ground truth trajectories camera motion recovery trajectory accuracy directional consistency Sora 2 Veo 3.1 Wan series fine-tuning 5B-parameter model