LACE-SVD: Loss-Aware SVD with Cumulative Error Correction for LLM Compression

LACE-SVD targets two weaknesses in SVD-based LLM compression: layer ranks are often assigned without modeling loss sensitivity, and local approximation errors can accumulate through residual streams. The method estimates calibration negative-log-likelihood increases for candidate layer compression ratios, solves a constrained rank-allocation problem, and applies closed-form local updates. It also adds propagation-aware correction to residual-stream output modules to reduce cumulative layer-output discrepancies. At a reported compression ratio of 0.6, LLaMA-7B achieves WikiText-2 perplexity of 32.57, compared with 46.18 for Dobi-SVD.