Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
AI Summary
ONCE is a plug-in video-token compression framework that moves expensive compression work from per-video inference to an offline stage. It learns a frequency-aware global codebook in visual feature space once, then compresses incoming tokens through lightweight codebook lookup and aggregation. According to the abstract, experiments span multiple video-understanding benchmarks and diverse compression baselines, with ONCE retaining competitive task performance while recording the lowest inference latency among the compared methods. The supplied abstract does not report model names, compression ratios, accuracy values, latency measurements, codebook size, or implementation availability.
Why it's worth reading
Video context length is becoming a practical Video-LLM bottleneck, and ONCE directly targets the repeated overhead of online compression, although its claimed efficiency advantage still requires verification from the full experimental results.
Deep Read
1. What happened
Original facts: The paper introduces ONCE, a plug-in video-token compression framework for Video-LLMs. It shifts the expensive learning component of compression offline and reuses a global codebook across incoming videos.
2. Core technology
Original facts: ONCE learns a frequency-aware global codebook in visual feature space. At inference time, it performs lightweight compression through codebook lookup and token aggregation. The authors describe this as an offline-to-online paradigm intended to avoid repeatedly running pruning or merging computations for every video and to reduce model-specific design requirements.
Analysis: Conceptually, the method appears to quantize and aggregate recurring visual features around previously learned prototypes. Its effectiveness should depend on codebook coverage, feature-distribution shift, and the aggregation procedure.
3. Key evidence and numbers
Original facts: The abstract reports experiments on multiple video-understanding benchmarks against diverse compression baselines. It claims competitive task performance and the lowest inference latency among the compared methods.
Missing information: The supplied abstract provides no benchmark names, Video-LLM names, accuracy results, token-reduction ratios, latency values, throughput figures, codebook size, offline training cost, or hardware configuration. The size of the claimed advantage and the amortization point for offline training therefore cannot be assessed.
4. Why it matters
Analysis: Visual-token sequences grow with video duration and spatial resolution, increasing downstream language-model computation. If an input-independent codebook transfers reliably across videos or models, compression could become an amortized offline asset instead of a recurring inference-time expense.
5. Practical impact
Analysis: The approach could reduce end-to-end latency for video question answering, long-video retrieval, and streaming analysis while making compression easier to attach to existing systems. A practical evaluation should jointly measure task quality, compression ratio, lookup overhead, memory use, offline training cost, and transfer across domains.
6. Limitations and uncertainty
Original facts: The abstract asserts a strong accuracy-efficiency trade-off but does not disclose quantitative results.
Unverified inference: It is unclear whether one global codebook remains effective across visual encoders, video domains, and resolutions. Frequency-aware compression may also suppress rare but task-critical events. In addition, arXiv identifier 2608.01271 and the stated publication date of 2026-08-02 are future-dated, so the paper text, authorship, version history, and code cannot currently be independently verified from the supplied material.
7. Original sources
- arXiv abstract page: https://arxiv.org/abs/2608.01271
- This analysis uses only the title, abstract, URL, and publication date supplied by the user; no unverified measurements or additional citations were added.