STMI: Segmentation-Guided Token Modulation with Cross-Modal Hypergraph Interaction for Multi-Modal Object Re-Identification
Multi-modal object Re-Identification (ReID) aims to exploit complementary information from different modalities to retrieve specific objects. However, existing methods often rely on hard token filtering or simple fusion strategies, which can lead to the loss of discriminative cues and increased background interference. To address these challenges, we propose STMI, a novel multi-modal learning framework consisting of three key components: (1) Segmentation-Guided Feature Modulation (SFM) module leverages SAM-generated masks to enhance foreground representations and suppress background noise through learnable attention modulation; (2) Semantic Token Reallocation (STR) module employs learnable query tokens and an adaptive reallocation mechanism to extract compact and informative representations without discarding any tokens; (3) Cross-Modal Hypergraph Interaction (CHI) module constructs a unified hypergraph across modalities to capture high-order semantic relationships. Extensive experiments on public benchmarks (i.e., RGBNT201, RGBNT100, and MSVR310) demonstrate the effectiveness and robustness of our proposed STMI framework in multi-modal ReID scenarios.
69aab844cedd2750d3d9aaf3
Xingguo Xu
claimed_verified
2026-03-06T13:54:22.891Z
65d574611e07c6a11c95f5d9
/avatars/e97d2062caa72a499e8a55b0fbf4a746.svg
leo Xu
leo12138
user
leo12138
69aab844cedd2750d3d9aaf4
Zhanyu Liu
69aab844cedd2750d3d9aaf5
Weixiang Zhou
69aab844cedd2750d3d9aaf6
Yuansheng Gao
claimed_verified
2026-03-06T13:54:25.395Z
69847708201195baa13aefce
/avatars/b13a29aff31191dafa48fd4b642e5287.svg
Yuansheng Gao
YSGao
user
YSGao
69aab844cedd2750d3d9aaf7
Junjie Cao
69aab844cedd2750d3d9aaf8
Yuhao Wang
69aab844cedd2750d3d9aaf9
Jixiang Luo
69aab844cedd2750d3d9aafa
Dell Zhang
STMI: Segmentation-Guided Token Modulation with Cross-Modal Hypergraph Interaction for Multi-Modal Object Re-Identification
Multi-modal object Re-Identification (ReID) aims to exploit complementary information from different modalities to retrieve specific objects. However, existing methods often rely on hard token filtering or simple fusion strategies, which can lead to the loss of discriminative cues and increased background interference. To address these challenges, we propose STMI, a novel multi-modal learning framework consisting of three key components: (1) Segmentation-Guided Feature Modulation (SFM) module leverages SAM-generated masks to enhance foreground representations and suppress background noise through learnable attention modulation; (2) Semantic Token Reallocation (STR) module employs learnable query tokens and an adaptive reallocation mechanism to extract compact and informative representations without discarding any tokens; (3) Cross-Modal Hypergraph Interaction (CHI) module constructs a unified hypergraph across modalities to capture high-order semantic relationships. Extensive experiments on public benchmarks (i.e., RGBNT201, RGBNT100, and MSVR310) demonstrate the effectiveness and robustness of our proposed STMI framework in multi-modal ReID scenarios.
A novel multi-modal learning framework for object ReID that enhances foreground representations, extracts compact features through token reallocation, and captures high-order semantic relationships via cross-modal hypergraph interaction.
multi-modal learning segmentation-guided feature modulation semantic token reallocation cross-modal hypergraph interaction SAM-generated masks learnable attention modulation adaptive reallocation mechanism hypergraph construction object ReID