OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

OmniPack is a training-free token-compression framework for omni-modal LLMs. It first removes structural redundancy before the LLM using modality-specific importance, global coverage, and similarity-aware merging. After sufficient multimodal interaction, it applies text-guided semantic refinement and audio-visual collaboration inside the LLM. The authors report experiments across five benchmarks and three Omni-LLM backbones, claiming the strongest performance-efficiency trade-off among evaluated methods. On Qwen2.5-Omni-7B, OmniPack reportedly retains 98.0% of baseline performance at 16.7% of the original FLOPs, and 92.9% at 6.8% FLOPs.