GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience
AI Summary
GROVE is a training-free framework for assistants that continuously observe video and need to answer questions about the past while also recognizing when past experience is relevant to the present. It grows one causal memory from streaming input, preserving fine-grained perceptual evidence and progressively consolidating it into timestamped moments, coherent episodes, and recurring patterns across days. Each temporal stratum has a scale-specific retrieval skill. The authors report that GROVE achieves the best results among compared methods on benchmarks including MM-lifelong and EgoServe, while ablations indicate that the strata and their access skills are complementary.
Why it's worth reading
Wearable assistants are moving beyond query-triggered recall, and GROVE offers a concrete architecture for unifying continual memory construction, multi-timescale retrieval, and proactive assistance without additional training.
Deep Read
What happened
The paper introduces GROVE, or Growing and Reasoning over Temporally Stratified Memory, for wearable assistants that continuously receive video. The supplied abstract says existing video-memory systems are mainly built for question-conditioned recall, while proactive assistants often separate memory from control. GROVE uses one causally growing memory to support both reactive question answering and proactive assistance.
Core tech
GROVE builds a temporal hierarchy online from a continuous video stream. Fine-grained perceptual evidence is progressively consolidated into timestamped moments, coherent episodes, and recurring patterns across days. Each stratum has a scale-native retrieval skill: locating an observation, replaying an activity, or traversing long-range regularities. A user query initiates reactive retrieval; the current situation initiates proactive retrieval. Both share the same memory and access interface. “Training-free” is a claim in the supplied abstract; the provided material does not specify the foundation models, prompting, indexing, or runtime implementation behind it.
Key evidence & numbers
The abstract reports that GROVE achieves the best results among compared methods on multiple benchmarks, including MM-lifelong and EgoServe. It also reports controlled ablations showing that temporal strata and their access skills are complementary, with patterns providing the largest benefit in the reported setting. The supplied abstract is truncated at “when evi...”, so the full condition, task breakdown, improvement margins, baselines, and statistical treatment cannot be confirmed. No exact scores, video volume, latency, memory footprint, or operating cost are provided.
Why it matters
This work frames video memory as an evolving temporal structure rather than a store searched only after a question arrives. Short-lived evidence, complete activities, and cross-day regularities have different storage costs, retrieval targets, and failure modes. Explicit strata could make retrieval more targeted and provide a clearer basis for proactive assistance. Analytically, the architecture exposes separate questions for evaluation: how memory grows, how each timescale is accessed, and when the present context should trigger recall.
Practical impact
Potential applications include personal activity review, object or task tracking, repeated-work assistance, and context-aware reminders. Hierarchical memory may reduce the need to scan the entire video history for every request and keep long-term patterns separate from one-off evidence. Deployment would still require clear policies for online processing, video deletion and compression, timestamp and event-boundary errors, and user confirmation or revocation of proactive suggestions. On-device processing and fine-grained access control are particularly important for first-person video.
Limitations & uncertainty
Only the paper page and a truncated abstract are available here, so the full experimental claims cannot be independently checked from the supplied material. The abstract does not define the benchmark tasks, dataset sizes, baselines, metrics, or exact margins. A training-free system may still depend on external vision-language models, embedding services, or prompt engineering; their cost and reliability are unspecified. Cross-day patterns could amplify false associations, privacy exposure, or unwanted reminders. Performance under new users, changing environments, occlusion, and long-term distribution shift is also unknown. The statement that patterns provide the largest benefit should be treated as a reported result in the abstract, not as a general law.
Original sources
- arXiv paper page
- Source label: hf-papers
- Supplied publication timestamp: 2026-08-02T20:00:00.000Z
- The method and evaluation statements above are based on the user-provided title and abstract; missing experimental numbers and citations have not been reconstructed.