Metronome studies a failure mode in real-time interaction model serving, where streaming sessions continuously grow their KV cache and eventually exhaust the pool. On a full-duplex stack, the system can abruptly jump from millisecond-scale frame latency to a stall, while conventional latency and deadline-miss metrics remain healthy. The proposed in-engine KV window bounds each session’s resident state. Across four interaction models on one GPU with real audio, the window eliminated collapse in the reported tests, with 0/20 versus 14/20 failed runs across two batches. An online admission controller then uses monotone latency as a load signal to estimate schedulable concurrency.
No heat snapshots are available in the last 24 hours.