Flash-KMeans: Fast and Memory-Efficient Exact K-Means
k-means has historically been positioned primarily as an offline processing primitive, typically used for dataset organization or embedding preprocessing rather than as a first-class component in online systems. In this work, we revisit this classical algorithm under the lens of modern AI system design and enable k-means as an online primitive. We point out that existing GPU implementations of k-means remain fundamentally bottlenecked by low-level system constraints rather than theoretical algorithmic complexity. Specifically, the assignment stage suffers from a severe IO bottleneck due to the massive explicit materialization of the N times K distance matrix in High Bandwidth Memory (HBM). Simultaneously, the centroid update stage is heavily penalized by hardware-level atomic write contention caused by irregular, scatter-style token aggregations. To bridge this performance gap, we propose flash-kmeans, an IO-aware and contention-free k-means implementation for modern GPU workloads. Flash-kmeans introduces two core kernel-level innovations: (1) FlashAssign, which fuses distance computation with an online argmin to completely bypass intermediate memory materialization; (2) sort-inverse update, which explicitly constructs an inverse mapping to transform high-contention atomic scatters into high-bandwidth, segment-level localized reductions. Furthermore, we integrate algorithm-system co-designs, including chunked-stream overlap and cache-aware compile heuristics, to ensure practical deployability. Extensive evaluations on NVIDIA H200 GPUs demonstrate that flash-kmeans achieves up to 17.9times end-to-end speedup over best baselines, while outperforming industry-standard libraries like cuML and FAISS by 33times and over 200times, respectively.
69b0e8231f7902d47f6dfa62
Shuo Yang
claimed_verified
2026-03-11T08:28:33.275Z
642b970ceb31218a5f204a29
/avatars/582287f477bbb1a0842787145e375fd3.svg
andy-yang
andy-yang
user
andy-yang
69b0e8231f7902d47f6dfa63
Haocheng Xi
admin_assigned
2026-03-12T09:25:28.371Z
66ce751a8ec9fda2cf5a9e85
/avatars/c17093ca81dad007b3e50bae503955a7.svg
Haocheng Xi
xihc-ucb
user
xihc-ucb
69b0e8231f7902d47f6dfa64
Yilong Zhao
69b0e8231f7902d47f6dfa65
Muyang Li
69b0e8231f7902d47f6dfa66
Xiaoze Fan
69b0e8231f7902d47f6dfa67
Jintao Zhang
claimed_verified
2026-03-11T08:28:31.155Z
66c0a08bac74db25de8427ec
https://cdn-avatars.huggingface.co/v1/production/uploads/66c0a08bac74db25de8427ec/9D-piDBZqSt6KNkHImmkv.jpeg
Jintao Zhang
jt-zhang
user
jt-zhang
69b0e8231f7902d47f6dfa68
Han Cai
69b0e8231f7902d47f6dfa69
Yujun Lin
69b0e8231f7902d47f6dfa6a
Xiuyu Li
69b0e8231f7902d47f6dfa6b
Kurt Keutzer
admin_assigned
2026-03-12T09:26:15.068Z
6251bf4b183aa4266924ad91
https://cdn-avatars.huggingface.co/v1/production/uploads/1678041834400-6251bf4b183aa4266924ad91.jpeg
Kurt Keutzer
kurtkeutzer
user
kurtkeutzer
69b0e8231f7902d47f6dfa6c
Song Han
69b0e8231f7902d47f6dfa6d
Chenfeng Xu
69b0e8231f7902d47f6dfa6e
Ion Stoica
Flash-KMeans: Fast and Memory-Efficient Exact K-Means
k-means has historically been positioned primarily as an offline processing primitive, typically used for dataset organization or embedding preprocessing rather than as a first-class component in online systems. In this work, we revisit this classical algorithm under the lens of modern AI system design and enable k-means as an online primitive. We point out that existing GPU implementations of k-means remain fundamentally bottlenecked by low-level system constraints rather than theoretical algorithmic complexity. Specifically, the assignment stage suffers from a severe IO bottleneck due to the massive explicit materialization of the N times K distance matrix in High Bandwidth Memory (HBM). Simultaneously, the centroid update stage is heavily penalized by hardware-level atomic write contention caused by irregular, scatter-style token aggregations. To bridge this performance gap, we propose flash-kmeans, an IO-aware and contention-free k-means implementation for modern GPU workloads. Flash-kmeans introduces two core kernel-level innovations: (1) FlashAssign, which fuses distance computation with an online argmin to completely bypass intermediate memory materialization; (2) sort-inverse update, which explicitly constructs an inverse mapping to transform high-contention atomic scatters into high-bandwidth, segment-level localized reductions. Furthermore, we integrate algorithm-system co-designs, including chunked-stream overlap and cache-aware compile heuristics, to ensure practical deployability. Extensive evaluations on NVIDIA H200 GPUs demonstrate that flash-kmeans achieves up to 17.9times end-to-end speedup over best baselines, while outperforming industry-standard libraries like cuML and FAISS by 33times and over 200times, respectively.
Flash-kmeans enables efficient online k-means clustering on GPUs through novel kernel-level optimizations that eliminate I/O bottlenecks and reduce atomic write contention.
k-means GPU implementations High Bandwidth Memory distance matrix atomic write contention FlashAssign sort-inverse update chunked-stream overlap cache-aware compile heuristics