HuggingFace Daily PapersJujia Zhao
UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents
Papers78
Evaluating shopping agents often suffers from a gap between synthetic user behavior and messy real-world sessions. UserArena introduces an interactive e-commerce benchmark built on 64,729 multi-turn trajectories from 11,010 users across 566,102 items, explicitly validated against controlled production A/B test outcomes. Paired with On-Policy Trajectory Distillation to preserve behavioral intent over long horizons, it offers an offline sandbox calibrated directly to live platform metrics.
Why it's worth reading
It bridges agentic recommendation simulation with real production A/B outcomes, setting a rigorous benchmark for multi-turn offline policy validation.
Tags
Agentic RecommendationUser SimulationE-commerceBenchmarkA/B TestingOffline EvaluationOPTD