CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.
699749637a658569d5a1009e
Zarif Ikram
claimed_verified
2026-02-23T09:46:11.876Z
640840f7829c2aedb104802e
/avatars/7b4e161a97d7e3740353f2238ca30a58.svg
Zarif Ikram
zarifikram
user
zarifikram
699749637a658569d5a1009f
Arad Firouzkouhi
699749637a658569d5a100a0
Stephen Tu
699749637a658569d5a100a1
Mahdi Soltanolkotabi
699749637a658569d5a100a2
Paria Rashidinejad
claimed_verified
2026-02-20T08:37:04.735Z
6488c5b833ff40e585bdcf64
/avatars/3bfbc2ecccd798a67afe1555ae7bb919.svg
Paria Rashidinejad
pariard
user
pariard
CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing
A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.
CrispEdit is a second-order editing algorithm for large language models that preserves capabilities by constraining updates to low-curvature subspaces of the capability-loss landscape using Bregman divergence and efficient Kronecker-factored approximations.
large language model capability preservation constrained optimization Bregman divergence Gauss-Newton Hessian Kronecker-factored approximate curvature matrix-free projector low-curvature subspace edit success capability degradation