When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills
AI Summary
The paper introduces AntiSkillBench, an end-to-end benchmark for privacy leakage, attribute disclosure, behavioral impersonation, and defenses in persona-skill pipelines. It contains 7,500 persona-grounded dialogue traces built from 50 behaviorally rich profiles, evaluates three skill-distillation strategies, and tests four online or post-hoc defense configurations. Experiments across three frontier agents suggest that risks persist across backbones and distillation protocols, extending beyond explicit attributes to communication styles and personality traits. Existing defenses show limited and distillation-dependent effectiveness.
Why it's worth reading
As persona skills become reusable personalization assets, this benchmark provides timely evidence that record-level privacy defenses may not cover skill distillation, behavioral disclosure, or agent impersonation.
Deep Read
What happened
Original facts: The paper introduces AntiSkillBench, an end-to-end benchmark for the security of persona-skill pipelines that transform interaction histories into portable, executable artifacts. It evaluates skill-level privacy leakage, agent-level attribute disclosure, behavioral impersonation, and defenses.
Core tech
Original facts: The benchmark contains 7,500 persona-grounded dialogue traces built from 50 behaviorally rich profiles. It covers three skill-distillation strategies and four online or post-hoc defense configurations, including active risk suppression and passive provenance protection. Analysis: This shifts the evaluation target from individual records or retrieved memories to compressed, reusable, executable skill artifacts.
Key evidence & numbers
Original facts: Experiments involve three frontier agents. The abstract reports that risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits. Existing defenses are described as limited and distillation-dependent. Unverified inference: The abstract does not provide metric values, confidence intervals, model names, or per-defense results, so relative performance cannot be established from the supplied material.
Why it matters
Analysis: A persona skill concentrates fragmented personal signals into a reusable behavioral program. When such a skill is shared, copied, or deployed across agents, privacy loss may involve reconstruction of communication habits, preferences, and decision patterns rather than disclosure of one isolated fact. Behavioral impersonation is therefore a distinct risk from conventional attribute leakage.
Practical impact
Analysis: Developers should treat skill export, sharing, and invocation as separate privacy boundaries, testing explicit information, inferred attributes, stylistic features, and behavioral consistency. The benchmark's multiple distillation and defense settings also suggest that prompt filtering, post-hoc auditing, or provenance labeling alone may not cover the full lifecycle.
Limitations & uncertainty
Original facts: The abstract does not specify how the 50 profiles were constructed, whether the traces use real user data, the exact definitions of the three distillation strategies and four defenses, or the complete metrics and baselines. Analysis: Synthetic profiles may differ from real long-term interaction histories, and three agents may not represent all agent architectures. The full paper, code, and data licensing should be checked.
Original sources
- arXiv abstract page
- Paper identifier: arXiv:2608.03700
- Source: hf-papers; published: 2026-08-03T20:00:00.000Z