BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives
Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.
69144007ac231a57265720af
Aarush Sinha
claimed_verified
2025-11-12T12:18:44.066Z
64dca00a5f21fb7f85989628
https://cdn-avatars.huggingface.co/v1/production/uploads/64dca00a5f21fb7f85989628/rJx4lxhTr4vkmdSkDuSYi.jpeg
Aarush
chungimungi
user
chungimungi
69144007ac231a57265720b0
Pavan Kumar S
claimed_verified
2025-11-12T12:18:42.146Z
625c3facf14e262dbb440f53
/avatars/e59e3c28d88dee86ccded3a145672a0e.svg
Pavan Kumar
rxpavan
user
rxpavan
69144007ac231a57265720b1
Roshan Balaji
claimed_verified
2025-11-13T13:06:56.655Z
63a448550cf4daf6166d608c
https://cdn-avatars.huggingface.co/v1/production/uploads/63a448550cf4daf6166d608c/G2RsLQ9yAwjdanX_Y6knJ.jpeg
Roshan Balaji
msb-roshan
user
msb-roshan
69144007ac231a57265720b2
Nirav Pravinbhai Bhatt
BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives
Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.
BiCA uses citation links to improve biomedical retrieval models by providing effective hard negatives, enhancing zero-shot and long-tailed performance with minimal fine-tuning.
hard negatives dense retrieval citation-aware PubMed GTE_small GTE_Base zero-shot retrieval nDCG@10 BEIR long-tailed topics LoTTE Success@5 domain adaptation