Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models
While Late Interaction models exhibit strong retrieval performance, many of their underlying dynamics remain understudied, potentially hiding performance bottlenecks. In this work, we focus on two topics in Late Interaction retrieval: a length bias that arises when using multi-vector scoring, and the similarity distribution beyond the best scores pooled by the MaxSim operator. We analyze these behaviors for state-of-the-art models on the NanoBEIR benchmark. Results show that while the theoretical length bias of causal Late Interaction models holds in practice, bi-directional models can also suffer from it in extreme cases. We also note that no significant similarity trend lies beyond the top-1 document token, validating that the MaxSim operator efficiently exploits the token-level similarity scores.
69cfc059e02fdd68b91b7cc6
Antoine Edy
claimed_verified
2026-04-07T08:44:40.156Z
66e16a677c2eb2da5109fb5c
https://cdn-avatars.huggingface.co/v1/production/uploads/66e16a677c2eb2da5109fb5c/jPkg3g4mtCYbm8ci-ymtX.png
Antoine EDY
antoineedy
user
antoineedy
69cfc059e02fdd68b91b7cc7
Max Conti
69cfc059e02fdd68b91b7cc8
Quentin Macé
claimed_verified
2026-04-07T08:44:37.317Z
661e945eebe3616a1b09e279
https://cdn-avatars.huggingface.co/v1/production/uploads/661e945eebe3616a1b09e279/U3DL1BNouUpcusCKAPZm0.jpeg
Quentin Macé
QuentinJG
user
QuentinJG
Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models
While Late Interaction models exhibit strong retrieval performance, many of their underlying dynamics remain understudied, potentially hiding performance bottlenecks. In this work, we focus on two topics in Late Interaction retrieval: a length bias that arises when using multi-vector scoring, and the similarity distribution beyond the best scores pooled by the MaxSim operator. We analyze these behaviors for state-of-the-art models on the NanoBEIR benchmark. Results show that while the theoretical length bias of causal Late Interaction models holds in practice, bi-directional models can also suffer from it in extreme cases. We also note that no significant similarity trend lies beyond the top-1 document token, validating that the MaxSim operator efficiently exploits the token-level similarity scores.
Late Interaction retrieval models exhibit length bias in multi-vector scoring and efficient similarity exploitation through MaxSim operator, as demonstrated on NanoBEIR benchmark.
Late Interaction models multi-vector scoring MaxSim operator NanoBEIR benchmark length bias token-level similarity scores