RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution
Retrieval algorithms like BM25 and query likelihood with Dirichlet smoothing remain strong and efficient first-stage rankers, yet improvements have mostly relied on parameter tuning and human intuition. We investigate whether a large language model, guided by an evaluator and evolutionary search, can automatically discover improved lexical retrieval algorithms. We introduce RankEvolve, a program evolution setup based on AlphaEvolve, in which candidate ranking algorithms are represented as executable code and iteratively mutated, recombined, and selected based on retrieval performance across 12 IR datasets from BEIR and BRIGHT. RankEvolve starts from two seed programs: BM25 and query likelihood with Dirichlet smoothing. The evolved algorithms are novel, effective, and show promising transfer to the full BEIR and BRIGHT benchmarks as well as TREC DL 19 and 20. Our results suggest that evaluator-guided LLM program evolution is a practical path towards automatic discovery of novel ranking algorithms.
699b8dbaf723198bd53a62d6
Jinming Nian
claimed_verified
2026-02-26T10:03:46.508Z
64aebfa77b3b338666286b5b
/avatars/75d23cab7e6912dcf6a6602c994853a3.svg
Jinming Nian
jnian
user
jnian
699b8dbaf723198bd53a62d7
Fangchen Li
699b8dbaf723198bd53a62d8
Dae Hoon Park
699b8dbaf723198bd53a62d9
Yi Fang
RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution
Retrieval algorithms like BM25 and query likelihood with Dirichlet smoothing remain strong and efficient first-stage rankers, yet improvements have mostly relied on parameter tuning and human intuition. We investigate whether a large language model, guided by an evaluator and evolutionary search, can automatically discover improved lexical retrieval algorithms. We introduce RankEvolve, a program evolution setup based on AlphaEvolve, in which candidate ranking algorithms are represented as executable code and iteratively mutated, recombined, and selected based on retrieval performance across 12 IR datasets from BEIR and BRIGHT. RankEvolve starts from two seed programs: BM25 and query likelihood with Dirichlet smoothing. The evolved algorithms are novel, effective, and show promising transfer to the full BEIR and BRIGHT benchmarks as well as TREC DL 19 and 20. Our results suggest that evaluator-guided LLM program evolution is a practical path towards automatic discovery of novel ranking algorithms.
Large language models guided by evaluators and evolutionary search can automatically discover improved lexical retrieval algorithms through program evolution techniques.
retrieval algorithms BM25 query likelihood Dirichlet smoothing large language models evaluator-guided evolutionary search program evolution AlphaEvolve ranking algorithms IR datasets BEIR BRIGHT TREC DL