Beyond Retrieval: Analytic Memory for Multimodal Agents

This paper introduces analytic memory as a complement to retrieval memory for long-term multimodal agents. Instead of only returning relevant past records, analytic memory organizes recurring observations into queryable structures that support filtering, aggregation, ranking, and temporal comparison. AdaMM extracts provenance-linked attribute-value observations from dialogue, images, and contextual metadata, discovers recurring field structures without application-defined schemas, and materializes them for analysis. A memory-aware planner routes query components to retrieval or analytic tools. On the MemEye and MemGallery benchmarks, AdaMM reports improvements of up to 11.3% and 6.9%, respectively.