This paper formalizes AI-agent-based experiment simulation as a Simulated Randomized Controlled Trial (S-RCT), with an error decomposition separating agent approximation error from subsampling error. Evaluated on 67 historical marketing A/B tests, an off-the-shelf foundation-model baseline achieved a sign overlap of 0.70 but systematically overstated effect sizes. A two-phase pre-period calibration protocol reduced squared prediction error, after removing irreducible measurement noise, by approximately 77x. A within-subject design reduced standard errors by approximately 2.4x. The authors present the framework as agent-agnostic and discuss where simulated signals may assist experiment planning.
No heat snapshots are available in the last 24 hours.