Panoramic Affordance Prediction
Affordance prediction serves as a critical bridge between perception and action in embodied AI. However, existing research is confined to pinhole camera models, which suffer from narrow Fields of View (FoV) and fragmented observations, often missing critical holistic environmental context. In this paper, we present the first exploration into Panoramic Affordance Prediction, utilizing 360-degree imagery to capture global spatial relationships and holistic scene understanding. To facilitate this novel task, we first introduce PAP-12K, a large-scale benchmark dataset containing over 1,000 ultra-high-resolution (12k, 11904 x 5952) panoramic images with over 12k carefully annotated QA pairs and affordance masks. Furthermore, we propose PAP, a training-free, coarse-to-fine pipeline inspired by the human foveal visual system to tackle the ultra-high resolution and severe distortion inherent in panoramic images. PAP employs recursive visual routing via grid prompting to progressively locate targets, applies an adaptive gaze mechanism to rectify local geometric distortions, and utilizes a cascaded grounding pipeline to extract precise instance-level masks. Experimental results on PAP-12K reveal that existing affordance prediction methods designed for standard perspective images suffer severe performance degradation and fail due to the unique challenges of panoramic vision. In contrast, PAP framework effectively overcomes these obstacles, significantly outperforming state-of-the-art baselines and highlighting the immense potential of panoramic perception for robust embodied intelligence.
69b8ca536b3f36c249fbc418
Zixin Zhang
claimed_verified
2026-03-17T08:20:08.805Z
674aa9af9494dd9106006c27
/avatars/2f04bb009983ec0ade738aa7941cf6dc.svg
Zixin Zhang
zhangzixin02
user
zhangzixin02
69b8ca536b3f36c249fbc419
Chenfei Liao
admin_assigned
2026-03-17T08:43:22.150Z
6806464ed918f6d2fee2bc8b
https://cdn-avatars.huggingface.co/v1/production/uploads/6806464ed918f6d2fee2bc8b/rgpG2oO0m6PT0KltCF_Wf.jpeg
Chenfei Liao
Chenfei-Liao
user
Chenfei-Liao
69b8ca536b3f36c249fbc41a
Hongfei Zhang
claimed_verified
2026-04-07T08:48:13.479Z
66864408c1d35ab079488662
https://cdn-avatars.huggingface.co/v1/production/uploads/66864408c1d35ab079488662/U-btVsJ65j7m6vFLVIka8.jpeg
Hongfei (Faye) Zhang
FayeHongfeiZhang
user
FayeHongfeiZhang
69b8ca536b3f36c249fbc41b
Harold Haodong Chen
69b8ca536b3f36c249fbc41c
Kanghao Chen
69b8ca536b3f36c249fbc41d
Zichen Wen
69b8ca536b3f36c249fbc41e
Litao Guo
admin_assigned
2026-03-17T08:43:50.008Z
6476025bfd4c874fa84b6a28
/avatars/b5ca2039731fec9d8e95d6d15b7719f5.svg
litao Guo
Riversideli
user
Riversideli
69b8ca536b3f36c249fbc41f
Bin Ren
69b8ca536b3f36c249fbc420
Xu Zheng
69b8ca536b3f36c249fbc421
Yinchuan Li
admin_assigned
2026-03-17T08:43:40.673Z
67a47c8a58ebe2a5fbd025c9
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/hlcc130ecAitoQmLmG8Eo.png
Yinchuan Li
YinchuanLi
user
YinchuanLi
69b8ca536b3f36c249fbc422
Xuming Hu
69b8ca536b3f36c249fbc423
Nicu Sebe
69b8ca536b3f36c249fbc424
Ying-Cong Chen
Panoramic Affordance Prediction
Affordance prediction serves as a critical bridge between perception and action in embodied AI. However, existing research is confined to pinhole camera models, which suffer from narrow Fields of View (FoV) and fragmented observations, often missing critical holistic environmental context. In this paper, we present the first exploration into Panoramic Affordance Prediction, utilizing 360-degree imagery to capture global spatial relationships and holistic scene understanding. To facilitate this novel task, we first introduce PAP-12K, a large-scale benchmark dataset containing over 1,000 ultra-high-resolution (12k, 11904 x 5952) panoramic images with over 12k carefully annotated QA pairs and affordance masks. Furthermore, we propose PAP, a training-free, coarse-to-fine pipeline inspired by the human foveal visual system to tackle the ultra-high resolution and severe distortion inherent in panoramic images. PAP employs recursive visual routing via grid prompting to progressively locate targets, applies an adaptive gaze mechanism to rectify local geometric distortions, and utilizes a cascaded grounding pipeline to extract precise instance-level masks. Experimental results on PAP-12K reveal that existing affordance prediction methods designed for standard perspective images suffer severe performance degradation and fail due to the unique challenges of panoramic vision. In contrast, PAP framework effectively overcomes these obstacles, significantly outperforming state-of-the-art baselines and highlighting the immense potential of panoramic perception for robust embodied intelligence.