Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction
We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP.
699d12104e37ec6dfa1bc4dd
Shannan Yan
claimed_verified
2026-02-24T09:50:55.457Z
662a1628b3f55ae101e20ca4
/avatars/9a56305a4b0909a429431c3b73e45efb.svg
Shannan Yan
ysner
user
ysner
699d12104e37ec6dfa1bc4de
Leqi Zheng
699d12104e37ec6dfa1bc4df
Keyu Lv
699d12104e37ec6dfa1bc4e0
Jingchen Ni
699d12104e37ec6dfa1bc4e1
Hongyang Wei
699d12104e37ec6dfa1bc4e2
Jiajun Zhang
claimed_verified
2026-02-26T10:03:22.412Z
660297295c467e8c06c684b0
/avatars/b721b512148ae118cd99464ae909bb3a.svg
zjj
zjj1233
user
zjj1233
699d12104e37ec6dfa1bc4e3
Guangting Wang
699d12104e37ec6dfa1bc4e4
Jing Lyu
699d12104e37ec6dfa1bc4e5
Chun Yuan
699d12104e37ec6dfa1bc4e6
Fengyun Rao
Learning Cross-View Object Correspondence via Cycle-Consistent Mask Prediction
We study the task of establishing object-level visual correspondence across different viewpoints in videos, focusing on the challenging egocentric-to-exocentric and exocentric-to-egocentric scenarios. We propose a simple yet effective framework based on conditional binary segmentation, where an object query mask is encoded into a latent representation to guide the localization of the corresponding object in a target video. To encourage robust, view-invariant representations, we introduce a cycle-consistency training objective: the predicted mask in the target view is projected back to the source view to reconstruct the original query mask. This bidirectional constraint provides a strong self-supervisory signal without requiring ground-truth annotations and enables test-time training (TTT) at inference. Experiments on the Ego-Exo4D and HANDAL-X benchmarks demonstrate the effectiveness of our optimization objective and TTT strategy, achieving state-of-the-art performance. The code is available at https://github.com/shannany0606/CCMP.
A conditional binary segmentation framework with cycle-consistency training enables robust object correspondence across egocentric and exocentric viewpoints without ground-truth annotations.
conditional binary segmentation view-invariant representations cycle-consistency training test-time training object correspondence visual correspondence