Property-driven Causal Abstractions for Markov Decision Processes

This paper introduces a property-driven causal abstraction method for factored Markov Decision Processes. It defines causal relations over predicates of state variables and groups states that share the same reasons for satisfying or violating a target abstraction property. The authors theoretically and empirically compare causal abstractions across MDPs, interval MDPs, and stochastic games. According to the abstract, experiments on several standard benchmarks produce compact abstractions that support near-optimal policies for the original MDP, and the abstractions often generalize to related large-scale models.