A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning
Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI. Prevailing end-to-end models couple high-level reasoning and low-level grounding into a single monolithic pipeline and rely on training over annotated datasets, which leads to poor generalization on novel objects and unseen environments. In this paper, we move beyond this paradigm by proposing A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline. Our framework coordinates specialized foundation models at test time: (1) a Dreamer that employs generative models to visualize how an interaction would look; (2) a Thinker that utilizes large vision-language models to decide what object part to interact with; and (3) a Spotter that orchestrates vision foundation models to precisely locate where the interaction area is. By leveraging the complementary strengths of pre-trained models without any task-specific fine-tuning, our zero-shot framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks and demonstrates robust generalization to real-world settings.
694223685d5b2dc10527487e
Zixin Zhang
claimed_verified
2025-12-17T10:08:17.208Z
674aa9af9494dd9106006c27
/avatars/2f04bb009983ec0ade738aa7941cf6dc.svg
Zixin Zhang
zhangzixin02
user
zhangzixin02
694223685d5b2dc10527487f
Kanghao Chen
694223685d5b2dc105274880
Hanqing Wang
694223685d5b2dc105274881
Hongfei Zhang
claimed_verified
2026-04-07T08:49:45.230Z
66864408c1d35ab079488662
https://cdn-avatars.huggingface.co/v1/production/uploads/66864408c1d35ab079488662/U-btVsJ65j7m6vFLVIka8.jpeg
Hongfei (Faye) Zhang
FayeHongfeiZhang
user
FayeHongfeiZhang
694223685d5b2dc105274882
Harold Haodong Chen
claimed_verified
2025-12-17T14:04:33.472Z
6570450a78d7aca0c361a177
https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/MX7jHhTQwLs-BvYIu5rqb.jpeg
Harold Chen
Harold328
user
Harold328
694223685d5b2dc105274883
Chenfei Liao
claimed_verified
2025-12-17T10:08:19.223Z
6806464ed918f6d2fee2bc8b
https://cdn-avatars.huggingface.co/v1/production/uploads/6806464ed918f6d2fee2bc8b/rgpG2oO0m6PT0KltCF_Wf.jpeg
Chenfei Liao
Chenfei-Liao
user
Chenfei-Liao
694223685d5b2dc105274884
Litao Guo
694223685d5b2dc105274885
Ying-Cong Chen
A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning
Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI. Prevailing end-to-end models couple high-level reasoning and low-level grounding into a single monolithic pipeline and rely on training over annotated datasets, which leads to poor generalization on novel objects and unseen environments. In this paper, we move beyond this paradigm by proposing A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline. Our framework coordinates specialized foundation models at test time: (1) a Dreamer that employs generative models to visualize how an interaction would look; (2) a Thinker that utilizes large vision-language models to decide what object part to interact with; and (3) a Spotter that orchestrates vision foundation models to precisely locate where the interaction area is. By leveraging the complementary strengths of pre-trained models without any task-specific fine-tuning, our zero-shot framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks and demonstrates robust generalization to real-world settings.
A4-Agent, a training-free framework, decouples affordance prediction into three stages using specialized pre-trained models to enhance generalization and performance in real-world settings.
Dreamer generative models Thinker large vision-language models Spotter vision foundation models zero-shot framework affordance prediction