Co-design of LLM-based preference agents: participation may drive overtrust

This primarily qualitative study examines whether participatory design makes LLM-based preference agents more trustworthy or merely appear so. Twelve participants created personal agents for household-energy decisions through a background survey, co-design interview, and validation survey. Most participants ultimately felt well represented, yet independent validation found mixed human-agent alignment. Agent responses were notably more homogeneous, decisive, and abstract than those of the human sample. The author describes participation and process transparency as a possible “overtrust engine” that can increase confidence while concealing systematic misalignment, especially when such agents are deployed at scale.