Generalised Bellman Recurrence and Three Dualities in Sequential Decision-Making

This paper derives the Bellman equation from three structural conditions: dynamics must factor through sufficient statistics, returns must decompose recursively, and uncertainty aggregation must be compatible with both. When the conditions hold on a common state, their mutual consistency produces the Bellman recurrence. When one condition fails, tractability may be recovered by augmenting the state or deforming the return or dynamics. The paper also presents three dualities linking probability and return, return and aggregation, and aggregation and probability, framing methods from reinforcement learning, control, and decision theory as instances of one construction.