Statistical Learning Theory in Lean 4: Empirical Processes from Scratch
We present the first comprehensive Lean 4 formalization of statistical learning theory (SLT) grounded in empirical process theory. Our end-to-end formal infrastructure implement the missing contents in latest Lean 4 Mathlib library, including a complete development of Gaussian Lipschitz concentration, the first formalization of Dudley's entropy integral theorem for sub-Gaussian processes, and an application to least-squares (sparse) regression with a sharp rate. The project was carried out using a human-AI collaborative workflow, in which humans design proof strategies and AI agents execute tactical proof construction, leading to the human-verified Lean 4 toolbox for SLT. Beyond implementation, the formalization process exposes and resolves implicit assumptions and missing details in standard SLT textbooks, enforcing a granular, line-by-line understanding of the theory. This work establishes a reusable formal foundation and opens the door for future developments in machine learning theory. The code is available at https://github.com/YuanheZ/lean-stat-learning-theory
69878404beecc443208d23af
Yuanhe Zhang
claimed_verified
2026-02-09T08:30:52.457Z
671925c6ebe45c60d885ebcf
/avatars/9f3a049409ca8ae1365bce17dbb4af85.svg
Yuanhe Zhang
liminho123
user
liminho123
69878404beecc443208d23b0
Jason D. Lee
69878404beecc443208d23b1
Fanghui Liu
Statistical Learning Theory in Lean 4: Empirical Processes from Scratch
We present the first comprehensive Lean 4 formalization of statistical learning theory (SLT) grounded in empirical process theory. Our end-to-end formal infrastructure implement the missing contents in latest Lean 4 Mathlib library, including a complete development of Gaussian Lipschitz concentration, the first formalization of Dudley's entropy integral theorem for sub-Gaussian processes, and an application to least-squares (sparse) regression with a sharp rate. The project was carried out using a human-AI collaborative workflow, in which humans design proof strategies and AI agents execute tactical proof construction, leading to the human-verified Lean 4 toolbox for SLT. Beyond implementation, the formalization process exposes and resolves implicit assumptions and missing details in standard SLT textbooks, enforcing a granular, line-by-line understanding of the theory. This work establishes a reusable formal foundation and opens the door for future developments in machine learning theory. The code is available at https://github.com/YuanheZ/lean-stat-learning-theory
A comprehensive formalization of statistical learning theory in Lean 4 addresses gaps in mathematical libraries and demonstrates human-AI collaboration for verified machine learning theory foundations.
statistical learning theory empirical process theory Gaussian Lipschitz concentration Dudley's entropy integral theorem sub-Gaussian processes least-squares regression sparse regression human-AI collaborative workflow Lean 4 Mathlib library formal verification proof strategy tactical proof construction