Urban Socio-Semantic Segmentation with Vision-Language Reasoning
As hubs of human activity, urban surfaces consist of a wealth of semantic entities. Segmenting these various entities from satellite imagery is crucial for a range of downstream applications. Current advanced segmentation models can reliably segment entities defined by physical attributes (e.g., buildings, water bodies) but still struggle with socially defined categories (e.g., schools, parks). In this work, we achieve socio-semantic segmentation by vision-language model reasoning. To facilitate this, we introduce the Urban Socio-Semantic Segmentation dataset named SocioSeg, a new resource comprising satellite imagery, digital maps, and pixel-level labels of social semantic entities organized in a hierarchical structure. Additionally, we propose a novel vision-language reasoning framework called SocioReasoner that simulates the human process of identifying and annotating social semantic entities via cross-modal recognition and multi-stage reasoning. We employ reinforcement learning to optimize this non-differentiable process and elicit the reasoning capabilities of the vision-language model. Experiments demonstrate our approach's gains over state-of-the-art models and strong zero-shot generalization. Our dataset and code are available in https://github.com/AMAP-ML/SocioReasoner.
69699e5e32f0333869ff9378
Yu Wang
claimed_verified
2026-01-19T09:24:12.861Z
676e9f1ec66778afc1b227c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/x89RSnfYdhSPU3Tkak7Lc.png
Wang Yu
vvangfaye
user
vvangfaye
69699e5e32f0333869ff9379
Yi Wang
claimed_verified
2026-01-20T09:41:19.318Z
6773bcaa675a971ddf1e81dd
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/a8VUwZYXd7O_mq_zFvXMh.png
CokeWang
CokeWang
user
CokeWang
69699e5e32f0333869ff937a
Rui Dai
claimed_verified
2026-01-16T14:43:46.050Z
661de9defdbc9c247f159d15
/avatars/38e21e78327cc908201122405c48f41b.svg
Rui Dai
DerryD
user
DerryD
69699e5e32f0333869ff937b
Yujie Wang
69699e5e32f0333869ff937c
Kaikui Liu
69699e5e32f0333869ff937d
Xiangxiang Chu
69699e5e32f0333869ff937e
Yansheng Li
admin_assigned
2026-01-16T15:32:19.008Z
63ec91dec8827dd0f0f3b489
/avatars/3d0d9479a26673f859c226efaf1e4a43.svg
shengli
yanshengli
user
yanshengli
Urban Socio-Semantic Segmentation with Vision-Language Reasoning
As hubs of human activity, urban surfaces consist of a wealth of semantic entities. Segmenting these various entities from satellite imagery is crucial for a range of downstream applications. Current advanced segmentation models can reliably segment entities defined by physical attributes (e.g., buildings, water bodies) but still struggle with socially defined categories (e.g., schools, parks). In this work, we achieve socio-semantic segmentation by vision-language model reasoning. To facilitate this, we introduce the Urban Socio-Semantic Segmentation dataset named SocioSeg, a new resource comprising satellite imagery, digital maps, and pixel-level labels of social semantic entities organized in a hierarchical structure. Additionally, we propose a novel vision-language reasoning framework called SocioReasoner that simulates the human process of identifying and annotating social semantic entities via cross-modal recognition and multi-stage reasoning. We employ reinforcement learning to optimize this non-differentiable process and elicit the reasoning capabilities of the vision-language model. Experiments demonstrate our approach's gains over state-of-the-art models and strong zero-shot generalization. Our dataset and code are available in https://github.com/AMAP-ML/SocioReasoner.
Urban socio-semantic segmentation is achieved through a vision-language model framework that combines cross-modal recognition and multi-stage reasoning with reinforcement learning optimization.
vision-language model cross-modal recognition multi-stage reasoning reinforcement learning socio-semantic segmentation Urban Socio-Semantic Segmentation dataset SocioReasoner