Easy to Learn, Yet Hard to Forget: Towards Robust Unlearning Under Bias
Machine unlearning, which enables a model to forget specific data, is crucial for ensuring data privacy and model reliability. However, its effectiveness can be severely undermined in real-world scenarios where models learn unintended biases from spurious correlations within the data. This paper investigates the unique challenges of unlearning from such biased models. We identify a novel phenomenon we term ``shortcut unlearning," where models exhibit an ``easy to learn, yet hard to forget" tendency. Specifically, models struggle to forget easily-learned, bias-aligned samples; instead of forgetting the class attribute, they unlearn the bias attribute, which can paradoxically improve accuracy on the class intended to be forgotten. To address this, we propose CUPID, a new unlearning framework inspired by the observation that samples with different biases exhibit distinct loss landscape sharpness. Our method first partitions the forget set into causal- and bias-approximated subsets based on sample sharpness, then disentangles model parameters into causal and bias pathways, and finally performs a targeted update by routing refined causal and bias gradients to their respective pathways. Extensive experiments on biased datasets including Waterbirds, BAR, and Biased NICO++ demonstrate that our method achieves state-of-the-art forgetting performance and effectively mitigates the shortcut unlearning problem.
69a851d98b5b27c7ea9fa4b3
JuneHyoung Kwon
claimed_verified
2026-03-16T10:44:52.463Z
6756f15abba07064c254d8d3
/avatars/0b7067c2475d5cb1950579cbb98b54f3.svg
June Hyoung Kwon
herbwood27
user
herbwood27
69a851d98b5b27c7ea9fa4b4
MiHyeon Kim
69a851d98b5b27c7ea9fa4b5
Eunju Lee
69a851d98b5b27c7ea9fa4b6
Yoonji Lee
69a851d98b5b27c7ea9fa4b7
Seunghoon Lee
69a851d98b5b27c7ea9fa4b8
YoungBin Kim
Easy to Learn, Yet Hard to Forget: Towards Robust Unlearning Under Bias
Machine unlearning, which enables a model to forget specific data, is crucial for ensuring data privacy and model reliability. However, its effectiveness can be severely undermined in real-world scenarios where models learn unintended biases from spurious correlations within the data. This paper investigates the unique challenges of unlearning from such biased models. We identify a novel phenomenon we term ``shortcut unlearning," where models exhibit an ``easy to learn, yet hard to forget" tendency. Specifically, models struggle to forget easily-learned, bias-aligned samples; instead of forgetting the class attribute, they unlearn the bias attribute, which can paradoxically improve accuracy on the class intended to be forgotten. To address this, we propose CUPID, a new unlearning framework inspired by the observation that samples with different biases exhibit distinct loss landscape sharpness. Our method first partitions the forget set into causal- and bias-approximated subsets based on sample sharpness, then disentangles model parameters into causal and bias pathways, and finally performs a targeted update by routing refined causal and bias gradients to their respective pathways. Extensive experiments on biased datasets including Waterbirds, BAR, and Biased NICO++ demonstrate that our method achieves state-of-the-art forgetting performance and effectively mitigates the shortcut unlearning problem.
Machine unlearning faces challenges when models have learned spurious correlations, leading to shortcut unlearning where bias attributes are forgotten instead of class attributes; a new framework called CUPID addresses this by partitioning data based on loss landscape sharpness and disentangling causal and bias pathways in model parameters.
machine unlearning spurious correlations shortcut unlearning loss landscape sharpness causal pathways bias pathways parameter disentanglement