A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain

This paper presents an auditable controller for frozen LLM agents. Instead of allowing costly code search or unconstrained self-modification, it defines a small, human-legible action space over prompt templates, tools, memory and retrieval, planning, and verification policies. The controller learns online with an ε-greedy contextual bandit and REINFORCE, using a multi-objective reward covering task success, verifier score, policy compliance, cost, latency, and unsupported-claim penalties. The authors instantiate it with DSPy and evaluate it on tool-use workflows, HumanEval code generation, and HotpotQA multi-hop QA using a local Ollama model and AWS Bedrock. Code, datasets, logs, and a deployment recipe are released.