TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interaction
Accurate 3D human keypoints localization is a critical technology enabling robots to achieve natural and safe physical interaction with users. Conventional 3D human keypoints estimation methods primarily focus on the whole-body reconstruction quality relative to the root joint. However, in practical human-robot interaction (HRI) scenarios, robots are more concerned with the precise metric-scale spatial localization of task-relevant body parts under the egocentric camera 3D coordinate. We propose TAIHRI, the first Vision-Language Model (VLM) tailored for close-range HRI perception, capable of understanding users' motion commands and directing the robot's attention to the most task-relevant keypoints. By quantizing 3D keypoints into a finite interaction space, TAIHRI precisely localize the 3D spatial coordinates of critical body parts by 2D keypoint reasoning via next token prediction, and seamlessly adapt to downstream tasks such as natural language control or global space human mesh recovery. Experiments on egocentric interaction benchmarks demonstrate that TAIHRI achieves superior estimation accuracy for task-critical body parts. We believe TAIHRI opens new research avenues in the field of embodied human-robot interaction. Code is available at: https://github.com/Tencent/TAIHRI.
69dda863aea31e227c22c45d
Ao Li
69dda863aea31e227c22c45e
Yonggen Ling
69dda863aea31e227c22c45f
Yiyang Lin
69dda863aea31e227c22c460
Yuji Wang
claimed_verified
2026-04-14T08:15:38.372Z
659bb678e57c59004625c624
/avatars/32b395c3504acb1fe29cceb65508b351.svg
Voyage_Wang
VoyageWang
user
VoyageWang
69dda863aea31e227c22c461
Yong Deng
69dda863aea31e227c22c462
Yansong Tang
TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interaction
Accurate 3D human keypoints localization is a critical technology enabling robots to achieve natural and safe physical interaction with users. Conventional 3D human keypoints estimation methods primarily focus on the whole-body reconstruction quality relative to the root joint. However, in practical human-robot interaction (HRI) scenarios, robots are more concerned with the precise metric-scale spatial localization of task-relevant body parts under the egocentric camera 3D coordinate. We propose TAIHRI, the first Vision-Language Model (VLM) tailored for close-range HRI perception, capable of understanding users' motion commands and directing the robot's attention to the most task-relevant keypoints. By quantizing 3D keypoints into a finite interaction space, TAIHRI precisely localize the 3D spatial coordinates of critical body parts by 2D keypoint reasoning via next token prediction, and seamlessly adapt to downstream tasks such as natural language control or global space human mesh recovery. Experiments on egocentric interaction benchmarks demonstrate that TAIHRI achieves superior estimation accuracy for task-critical body parts. We believe TAIHRI opens new research avenues in the field of embodied human-robot interaction. Code is available at: https://github.com/Tencent/TAIHRI.
TAIHRI is a vision-language model designed for egocentric human-robot interaction that enables precise 3D keypoint localization through 2D keypoint reasoning and next token prediction.
Vision-Language Model 3D human keypoints localization egocentric camera next token prediction task-relevant keypoints human-robot interaction