Decisive Markov Chains

Abstract

We consider qualitative and quantitative verification problems forinfinite-state Markov chains. We call a Markov chain decisive w.r.t. a givenset of target states F if it almost certainly eventually reaches either F or astate from which F can no longer be reached. While all finite Markov chains aretrivially decisive (for every set F), this also holds for many classes ofinfinite Markov chains. Infinite Markov chains which contain a finite attractorare decisive w.r.t. every set F. In particular, this holds for probabilisticlossy channel systems (PLCS). Furthermore, all globally coarse Markov chainsare decisive. This class includes probabilistic vector addition systems (PVASS)and probabilistic noisy Turing machines (PNTM). We consider both safety andliveness problems for decisive Markov chains, i.e., the probabilities that agiven set of states F is eventually reached or reached infinitely often,respectively. 1. We express the qualitative problems in abstract terms fordecisive Markov chains, and show an almost complete picture of its decidabilityfor PLCS, PVASS and PNTM. 2. We also show that the path enumeration algorithmof Iyer and Narasimha terminates for decisive Markov chains and can thus beused to solve the approximate quantitative safety problem. A modified variantof this algorithm solves the approximate quantitative liveness problem. 3.Finally, we show that the exact probability of (repeatedly) reaching F cannotbe effectively expressed (in a uniform way) in Tarski-algebra for either PLCS,PVASS or (P)NTM.Comment: 32 pages, 0 figure