Understanding Machine Unlearning Through the Lens of Mode Connectivity

This paper introduces mode connectivity in unlearning (MCU), a framework for studying machine unlearning through smooth, low-loss paths in parameter space. Across curriculum learning, second-order optimization, and multiple unlearning methods, the authors report that many unlearned models occupy connected basins with smooth retain/forget behavior, while privacy metrics can still differ substantially within the same basin. They also find nonlinear progress between the original and unlearned models, and that linear connectivity indicates approximate unlearning is usually mechanistically distinct from retraining. MCU-based ensembling reportedly improves generalization and robustness to relearning attacks.