KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Making deep learning recommendation model (DLRM) training and inference fast and efficient is important. However, this presents three key system challenges - model architecture diversity, kernel primitive diversity, and hardware generation and architecture heterogeneity. This paper presents KernelEvolve-an agentic kernel coding framework-to tackle heterogeneity at-scale for DLRM. KernelEvolve is designed to take kernel specifications as input and automate the process of kernel generation and optimization for recommendation model across heterogeneous hardware architectures. KernelEvolve does so by operating at multiple programming abstractions, from Triton and CuTe DSL to low-level hardware agnostic languages, spanning the full hardware-software optimization stack. The kernel optimization process is described as graph-based search with selection policy, universal operator, fitness function, and termination rule, dynamically adapts to runtime execution context through retrieval-augmented prompt synthesis. We designed, implemented, and deployed KernelEvolve to optimize a wide variety of production recommendation models across generations of NVIDIA and AMD GPUs, as well as Meta's AI accelerators. We validate KernelEvolve on the publicly-available KernelBench suite, achieving 100% pass rate on all 250 problems across three difficulty levels, and 160 PyTorch ATen operators across three heterogeneous hardware platforms, demonstrating 100% correctness. KernelEvolve reduces development time from weeks to hours and achieves substantial performance improvements over PyTorch baselines across diverse production use cases and for heterogeneous AI systems at-scale. Beyond performance efficiency improvements, KernelEvolve significantly mitigates the programmability barrier for new AI hardware by enabling automated kernel generation for in-house developed AI hardware.
6953f16b09c8c0a53818152e
Gang Liao
claimed_verified
2025-12-31T20:54:44.892Z
654d2d7a86f53e80527dd32d
https://cdn-avatars.huggingface.co/v1/production/uploads/654d2d7a86f53e80527dd32d/9XIl3O9iFh_0q9Ly_1LdI.jpeg
Gang Liao
gangliao
user
gangliao
6953f16b09c8c0a53818152f
Hongsen Qin
6953f16b09c8c0a538181530
Ying Wang
6953f16b09c8c0a538181531
Alicia Golden
6953f16b09c8c0a538181532
Michael Kuchnik
6953f16b09c8c0a538181533
Yavuz Yetim
6953f16b09c8c0a538181534
Jia Jiunn Ang
6953f16b09c8c0a538181535
Chunli Fu
6953f16b09c8c0a538181536
Yihan He
6953f16b09c8c0a538181537
Samuel Hsia
6953f16b09c8c0a538181538
Zewei Jiang
6953f16b09c8c0a538181539
Dianshi Li
6953f16b09c8c0a53818153a
Uladzimir Pashkevich
6953f16b09c8c0a53818153b
Varna Puvvada
6953f16b09c8c0a53818153c
Feng Shi
6953f16b09c8c0a53818153d
Matt Steiner
6953f16b09c8c0a53818153e
Ruichao Xiao
6953f16b09c8c0a53818153f
Nathan Yan
6953f16b09c8c0a538181540
Xiayu Yu
6953f16b09c8c0a538181541
Zhou Fang
6953f16b09c8c0a538181542
Abdul Zainul-Abedin
6953f16b09c8c0a538181543
Ketan Singh
6953f16b09c8c0a538181544
Hongtao Yu
6953f16b09c8c0a538181545
Wenyuan Chi
6953f16b09c8c0a538181546
Barney Huang
6953f16b09c8c0a538181547
Sean Zhang
6953f16b09c8c0a538181548
Noah Weller
6953f16b09c8c0a538181549
Zach Marine
6953f16b09c8c0a53818154a
Wyatt Cook
6953f16b09c8c0a53818154b
Carole-Jean Wu
6953f16b09c8c0a53818154c
Gaoxiang Liu
KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta
Making deep learning recommendation model (DLRM) training and inference fast and efficient is important. However, this presents three key system challenges - model architecture diversity, kernel primitive diversity, and hardware generation and architecture heterogeneity. This paper presents KernelEvolve-an agentic kernel coding framework-to tackle heterogeneity at-scale for DLRM. KernelEvolve is designed to take kernel specifications as input and automate the process of kernel generation and optimization for recommendation model across heterogeneous hardware architectures. KernelEvolve does so by operating at multiple programming abstractions, from Triton and CuTe DSL to low-level hardware agnostic languages, spanning the full hardware-software optimization stack. The kernel optimization process is described as graph-based search with selection policy, universal operator, fitness function, and termination rule, dynamically adapts to runtime execution context through retrieval-augmented prompt synthesis. We designed, implemented, and deployed KernelEvolve to optimize a wide variety of production recommendation models across generations of NVIDIA and AMD GPUs, as well as Meta's AI accelerators. We validate KernelEvolve on the publicly-available KernelBench suite, achieving 100% pass rate on all 250 problems across three difficulty levels, and 160 PyTorch ATen operators across three heterogeneous hardware platforms, demonstrating 100% correctness. KernelEvolve reduces development time from weeks to hours and achieves substantial performance improvements over PyTorch baselines across diverse production use cases and for heterogeneous AI systems at-scale. Beyond performance efficiency improvements, KernelEvolve significantly mitigates the programmability barrier for new AI hardware by enabling automated kernel generation for in-house developed AI hardware.