The Disruptive Impact of Large Language Models on Capture the Flag Competitions and the Path Toward Fair Play

This mixed-methods study examines how large language models are changing cybersecurity Capture the Flag competitions. It synthesizes published benchmarks, including a recent government evaluation, case studies from cryptography, web exploitation, and binary exploitation, observation of public community discussions, and interviews with experienced players and organizers. The paper reports that easy and intermediate challenges across these categories can now be reliably automated, while narrower subcategories remain resistant. It proposes four safeguards: tiered competition divisions, LLM-resistant challenge design, investigative telemetry, and a community code of conduct linked to each event's declared purpose.