AVO: Agentic Variation Operators for Autonomous Evolutionary Search
Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.
69c683d1878ef8af32384e36
Terry Chen
69c683d1878ef8af32384e37
Zhifan Ye
claimed_verified
2026-04-01T08:36:19.007Z
6716a8157782bceafb6edb34
/avatars/1d44df601e915fa3eb3aeb5bd2da1dfc.svg
Zhifan Ye
lemonrbit
user
lemonrbit
69c683d1878ef8af32384e38
Bing Xu
69c683d1878ef8af32384e39
Zihao Ye
69c683d1878ef8af32384e3a
Timmy Liu
69c683d1878ef8af32384e3b
Ali Hassani
69c683d1878ef8af32384e3c
Tianqi Chen
69c683d1878ef8af32384e3d
Andrew Kerr
69c683d1878ef8af32384e3e
Haicheng Wu
69c683d1878ef8af32384e3f
Yang Xu
69c683d1878ef8af32384e40
Yu-Jung Chen
69c683d1878ef8af32384e41
Hanfeng Chen
69c683d1878ef8af32384e42
Aditya Kane
69c683d1878ef8af32384e43
Ronny Krashinsky
69c683d1878ef8af32384e44
Ming-Yu Liu
69c683d1878ef8af32384e45
Vinod Grover
69c683d1878ef8af32384e46
Luis Ceze
69c683d1878ef8af32384e47
Roger Bringmann
69c683d1878ef8af32384e48
John Tran
69c683d1878ef8af32384e49
Wei Liu
69c683d1878ef8af32384e4a
Fung Xie
69c683d1878ef8af32384e4b
Michael Lightstone
69c683d1878ef8af32384e4c
Humphrey Shi
AVO: Agentic Variation Operators for Autonomous Evolutionary Search
Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.
Agentic variation operators enable autonomous discovery of performance-critical micro-architectural optimizations for attention kernels, outperforming state-of-the-art implementations on advanced GPU hardware.
evolutionary variation operators autonomous coding agents language model candidate generation domain-specific knowledge base execution feedback attention kernels NVIDIA Blackwell cuDNN FlashAttention-4 grouped-query attention micro-architectural optimizations