RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
AI Summary
RestoreKV augments query-agnostic KV cache eviction with a learned, context-conditioned restoration cache. After context prefill, a small set of restore tokens attends to the full KV cache in one LoRA-adapted pass, while the original scorer and eviction rule remain unchanged. The adapters are disabled during later queries and decoding. Training updates only 0.4% of parameters through parameter-efficient self-distillation from a frozen full-cache model. Across four backbones and four long-context benchmarks, RestoreKV improved 59 of 60 paired budget-matched settings on Qwen3-4B. At a 5% budget, KVzip improved from 38.2 to 73.2 RULER-4K accuracy.
Why it's worth reading
KV-cache compression is becoming a deployment bottleneck for long-context inference. RestoreKV is timely because it adds restoration without changing the later query path, while reporting budget-matched results and construction overhead.
Deep Read
What Happened
Original facts: The paper introduces RestoreKV to reduce the quality degradation caused by query-agnostic KV-cache eviction under tight budgets. After context prefill, it creates a compact, context-conditioned restoration cache from the full KV cache.
Core Tech
Original facts: A small set of restore tokens attends to the full KV cache in a single LoRA-adapted pass. The base importance scorer and eviction rule are unchanged, and the adapters are disabled for later queries and decoding. Training uses parameter-efficient self-distillation from a frozen full-cache model and updates only 0.4% of parameters.
Key Evidence & Numbers
Original facts: The evaluation covers four backbones and four long-context benchmarks. On Qwen3-4B, RestoreKV improves 59 of 60 paired, budget-matched settings across five base eviction methods. At a 5% budget, KVzip rises from 38.2 to 73.2 on RULER-4K. With KVzip+, it reaches 86.4 accuracy at 16x compression on the KVPress Benchmark. A 32K-context evaluation reports less than 0.5% one-time cache-construction overhead.
Why It Matters
Analysis: Existing eviction methods mainly optimize which original KV entries survive. RestoreKV adds a complementary objective: generate a compact representation of information lost during eviction. Because that representation is conditioned on the current context, it may preserve relationships that a purely selection-based cache cannot retain.
Practical Impact
Analysis: The method keeps the later query and decoding path unchanged, which is useful when one prefetched context serves many subsequent queries. If the reported overhead and quality gains generalize across hardware, batch sizes, and workloads, RestoreKV could reduce persistent KV memory while recovering much of the accuracy lost to compression.
Limitations & Uncertainty
Original facts: The abstract does not identify all four backbones and benchmarks, nor does it provide per-budget and per-task results. Analysis: RestoreKV still requires access to the full KV cache during context construction, so its main savings concern subsequent storage and query-time use rather than eliminating prefill cost. Unverified inference: Robustness for one-shot queries, dynamic contexts, out-of-distribution tasks, and extremely small budgets cannot be established from the abstract alone.
Original Sources
- Paper and abstract: arXiv:2608.01247
- Project page: RestoreKV Project Page