Trace refinement in labelled Markov decision processes

Journal article

Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced labelled Markov chains are trace-equivalent. We show that this problem is decidable in polynomial time if the second MDP is a Markov chain. The algorithm is based on new results on a particular notion of bisimulation between distributions over the states. However, we show that the general trace-refinement problem is undecidable, even if the first MDP is a Markov chain. Decidability of those problems has been open since 2008. We further study the decidability and complexity of the trace-refinement problem provided that the strategies are restricted to be memoryless.

Authors: Fijalkow, N; Kiefer, S; Shirmohammadi, M

• Publication status: Accepted
• Peer review status: Peer reviewed
• Publisher: European Joint Conferences on Theory and Practice of Software (ETAPS)
• Journal: 19th International Conference on Foundations of Software Science and Computation Structures (FoSSaCS)
• Publication date: 2015-01-01
• Keywords: cs.LO