XYEval: Benchmark Evaluates Agents Blindly Following Bad Advice

AI agents frequently exhibit sycophantic tendencies, deferring to flawed suggestions or misleading premises introduced in user prompts. The XYEval benchmark evaluates how agentic systems navigate bad advice during multi-step tasks, examining whether models can detect problematic guidance rather than obediently executing it. As autonomous workflows integrate deeper into production environments, discerning invalid inputs and steering tasks back on track marks an essential reliability frontier.