Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators

This paper proposes a single preference-free intrinsic reward for unsupervised reinforcement learning in environments containing both reducible and irreducible uncertainty. Its central signal is parameter information gain: it encourages exploration where dynamics remain unresolved, then vanishes as the model explains those dynamics. The method combines a pseudocount for epistemic value, a probe-based penalty for aleatoric variance, and a short-horizon gate for informative successors, without fitting an explicit next-state predictor. Freezing reward-defining objects within windows is used to obtain a stationary Bellman operator, bounded learning targets, and conditional uniform-concentration results under stated assumptions.