Exact quantitative probabilistic model checking through rational search

Abstract

Model checking systems formalized using probabilistic models such as discrete time Markov chains (DTMCs) and Markov decision processes (MDPs) can be reduced to computing constrained reachability properties. Linear programming methods to compute reachability probabilities for DTMCs and MDPs do not scale to large models. Thus, model checking tools often employ iterative methods to approximate reachability probabilities. These approximations can be far from the actual probabilities, leading to inaccurate model checking results. On the other hand, specialized techniques employed in existing state-of-the-art exact quantitative model checkers, don’t scale as well as their iterative counterparts. In this work, we present a new model checking algorithm that improves the approximate results obtained by scalable iterative techniques to compute exact reachability probabilities. Our techniques are implemented as an extension of the PRISM model checker and are evaluated against other exact quantitative model checking engines.

Proof of the claim in Theorem 2

It can be shown easily that f is non-expanding, i.e, for any ${{{\bar{x}}}}_1,{{\bar{x}}_2}\in {\mathcal {U}}$,

$$ ||f({\bar{x}}_2) - f({\bar{x}}_1) || \le || {\bar{x}}_1 -{\bar{x}}_2||. $$

We will assume without loss of generality that ${\mathsf {Prob}}^{{\mathsf {min}}}_{1}[\xi ]$ consists of exactly one element $z_0.$ Further, we assume that ${\mathsf {Prob}}^{{\mathsf {min}}}_{0}[\xi ]$ consists of at least 1 element as otherwise the claim is trivially true.

Let $Z^?= Z{\setminus } ( {\mathsf {Prob}}^{{\mathsf {min}}}_{0}[\xi ] \,\cup \,{\mathsf {Prob}}^{{\mathsf {min}}}_{1}[\xi ]).$ For ${{\bar{x}}} \in {\mathcal {U}}, z\in Z^?$ and $\alpha \in {\mathsf {enabled}}(z),$ we denote the sum $\displaystyle \sum _{z'\in Z} {\varDelta }(z,\alpha ,z') \cdot {\bar{x}}(z')$ by $h_{{{\bar{x}}},z,\alpha }.$ By definition

$$ f({\bar{x}})(z) = \displaystyle \min _{\alpha \in {\mathsf {enabled}}(z)} h_{{{\bar{x}}},z,\alpha }. $$

Fix ${\bar{x}},{\bar{y}}\in {\mathcal {U}}.$ The definition of $Z^?$ implies that for any scheduler ${\mathfrak {S}},$ the probability of reaching $z_0$ from a state $z\in Z^?$ is not zero. From this, there it follows that there is an enumeration $z_1,z_2,\ldots z_r$ of $Z^?$ such that for any $1\le i \le r$ and any action $\alpha \in {\mathsf {enabled}}(z_i),$ ${\varDelta }(z_i,\alpha ,z_j) >0$ for some $0\le j<i.$

We will show by induction on $0 \le i\le r,$

$$ |f^{i+1}({\bar{x}}) (z_i)- f^{i+1}({\bar{y}}) (z_i)| \le (1-p^i_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}||. $$

Observe that this suffices to conclude the claim since this implies for any $z_i\in Z^?,$

$$ \begin{aligned} |f^n({{\bar{x}}}) (z_i) - f^n({\bar{y}}) (z_i)|&\le || f^{i+1 } ({\bar{x}}) (z_i) - f^{i+1 } ({\bar{y}}) (z_i)|| \\&\le (1-p^i_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}|| \le (1-p^n_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}||. \end{aligned} $$

Now we show, by induction, that for each $0 \le i\le r,$ $|f^{i+1}({{\bar{x}}}) (z_i)- f^{i+1}({\bar{y}}) (z_i)| \le (1-p^i_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}||.$

Base case: The base case is trivial since $f({\bar{x}}) (z_0) = 1 = f({\bar{y}}) (z_0).$

Induction hypothesis: Let $|f^{i+1}({\bar{x}}) (z_i)- f^{i+1}({\bar{y}}) (z_i)| \le (1-p^i_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}||$ for each $0\le i \le \ell .$ Fix $\beta \in {\mathsf {enabled}}(z_{\ell +1}).$ Denote the set $\{ z_0,z_1,\ldots ,z_\ell \}$ by $Z_\ell .$ We have that

$$ \begin{aligned} h_{f^{\ell +2}({\bar{x}}),z_{\ell +1},\beta }&= \displaystyle \sum _{z'\in Z} {\varDelta }(z_{\ell +1},\beta ,z') \cdot f^{\ell +1}({\bar{x}}) (z') \\&= h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta } + \displaystyle \sum _{z'\in Z} {\varDelta }(z_{\ell +1},\beta ,z') \cdot (f^{\ell +1}(\bar{x})(z')-f^{\ell +1}({\bar{y}})(z'))\\&= h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta } + \displaystyle \sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot (f^{\ell +1}({\bar{x}})(z')-f^{\ell +1}({\bar{y}})(z')) \\&\quad + \displaystyle \sum _{z'\in Z{\setminus } Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot (f^{\ell +1}({\bar{x}})(z')-f^{\ell +1}({\bar{y}})(z')). \end{aligned} $$

Now, note that $(1-p^i_{\mathsf {min}})\le (1-p^\ell _{\mathsf {min}})$ for each $i\le \ell .$ Thus, we get by induction hypothesis,

$$ \begin{aligned} h_{f^{\ell +2}({{\bar{x}}}),z_{\ell +1},\beta }&\le h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta } + (1-p^\ell _{\mathsf {min}}) \displaystyle \sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot || {\bar{x}} -{\bar{y}}|| \\&\quad \displaystyle \sum _{z'\in Z{\setminus } Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot (f^{\ell +1}({\bar{x}})(z')-f^{\ell +1}({\bar{y}})(z')). \end{aligned} $$

As f is non-expanding, we get that

$$ \begin{aligned} h_{f^{\ell +2}({\bar{x}}),z_{\ell +1},\beta }&\le h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta } + (1-p^\ell _{\mathsf {min}}) \displaystyle \sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot || {\bar{x}} -{\bar{y}}|| \\&\quad + \displaystyle \sum _{z'\in Z{\setminus } Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z') \cdot || {\bar{x}} -{\bar{y}}|| \\&\le h_{f^{\ell +2}({{\bar{y}}}),z_{\ell +1},\beta } + || {\bar{x}} -{\bar{y}}|| \cdot \displaystyle \sum _{z'\in Z} {\varDelta }(z_{\ell +1},\beta ,z') \\&\quad - p^\ell _{\mathsf {min}}|| {\bar{x}} -{\bar{y}}|| \cdot \sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z')\\&\le h_{f^{\ell +2}({{\bar{y}}}),z_{\ell +1},\beta } + || {\bar{x}} -{\bar{y}}|| (1- p^\ell _{\mathsf {min}}\sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z')). \end{aligned} $$

By construction, $\sum _{z'\in Z_\ell } {\varDelta }(z_{\ell +1},\beta ,z')) \ge p_{\mathsf {min}}$ and hence

$$ \begin{aligned} h_{f^{\ell +2}({{\bar{x}}}),z_{\ell +1},\beta }&\le h_{f^{\ell +2}({{\bar{y}}}),z_{\ell +1},\beta } + || {\bar{x}} -{\bar{y}}|| (1- p^{\ell +1}_{\mathsf {min}})||. \end{aligned} $$

Now, we have that

$$ f^{\ell +2}({\bar{x}})(z_{\ell +1}) \le h_{f^{\ell +2}({\bar{x}}),z_{\ell +1},\beta } \le h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta } + || {\bar{x}} -{\bar{y}}|| (1- p^{\ell +1}_{\mathsf {min}})||. $$

As $\beta$ is arbitrary, the above inequality also holds for the $\beta$ that minimizes $h_{f^{\ell +2}({\bar{y}}),z_{\ell +1},\beta }.$ Hence,

$$ f^{\ell +2}({\bar{x}})(z_{\ell +1}) \le f^{\ell +2}({\bar{y}})(z_{\ell +1}) + || {\bar{x}} -{\bar{y}}|| (1- p^{\ell +1}_{\mathsf {min}})||. $$

Similarly, we can show that

$$ f^{\ell +2}({{\bar{y}}})(z_{\ell +1}) \le f^{\ell +2}({\bar{x}})(z_{\ell +1}) + || {\bar{x}} -{\bar{y}}|| (1- p^{\ell +1}_{\mathsf {min}})||. $$

Thus, we get

$$ |f^{\ell +2}({\bar{x}}) (z_{\ell +1})- f^{\ell +2}({\bar{y}}) (z_{\ell +1})| \le (1-p^{\ell +1}_{\mathsf {min}}) || {\bar{x}} -{\bar{y}}|| $$

as required.