Sample-Efficient Learning from Agent Experience

The paper introduces Experience Distillation, a method for internalizing an agent’s interaction history into model weights without collecting additional environment interactions. Across 749 curated software-engineering tasks and six text-adventure games, it retained at least 64.8% of the gains achieved by in-context learning, while direct supervised fine-tuning recovered only 3.8%. Against classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matched performance using at least 9.6 times fewer environment samples.