Property-Driven Partitioning for Abstraction Refinement Roberto Sebastiani1,⋆ , Stefano Tonetta2 , and Moshe Y. Vardi3,⋆⋆ 1 3 DIT, Università di Trento, Italy rseba@dit.unitn.it 2 University of Lugano, Switzerland tonettas@lu.unisi.ch Dept. of Computer Science, Rice University, USA vardi@cs.rice.edu Abstract. Partitioning and abstraction have been studied extensively both in hardware and in software verification. The abstraction is typically partitioned according to the system design in the case of hardware or the control graph in the case of software. In this work we build on previous work on Property-Driven Partitioning (PDP), a hybrid Symbolic Model-Checking (SMC) technique for ωregular properties in which the state space is partitioned according to the states of the property automaton. We investigate a new paradigm for abstraction refinement in SMC, which combines abstraction and PDP: each PDP partition may contain a different abstraction, so that it can be refined independently from the others; in case of a spurious counterexample π, the system is refined only in those partitions that are necessary to rule out π. We performed a preliminary experimental evaluation comparing standard Counterexample-Guided Abstraction Refinement (CEGAR) with its partitioned counterpart, which confirmed that the partitioned technique always allows for using coarser abstractions. While earlier work has shown that PDP almost always improves the performance of SMC, our experiments here show that this is not always the case for partitioned abstraction refinement, as in some cases the overhead due to the localization of the abstraction is too high. 1 Introduction Verifying properties of finite-state machines is a fundamental problem in formal design verification. Symbolic Model Checking (SMC) [2] has been successful at verifying temporal specifications. In particular, LTL model checking is solved by building an automaton A with the complementary language of the property, computing the product of A with the system M and checking for emptiness [17]. Typically, the product is represented either explicitly, by enumerating its elements, or symbolically, by means of propositional formulas. The main obstacle to model checking is the state-space explosion, that is, the state-space is often too large to be handled. ⋆ ⋆⋆ Supported in part by ORCHID, a project sponsored by Provincia Autonoma di Trento, and by a grant from Intel Corporation. Supported in part by NSF grants CCR-9988322, CCR-0124077, CCR-0311326, and ANI0216467, by BSF grant 9800096, and by Texas ATP grant 003604-0058-2003. O. Grumberg and M. Huth (Eds.): TACAS 2007, LNCS 4424, pp. 389–404, 2007. c Springer-Verlag Berlin Heidelberg 2007  390 R. Sebastiani, S. Tonetta, and M.Y. Vardi One of the most efficient technique to reduce the state-space in SMC is abstraction [6]. An abstract system is simply an approximation of the original one. A conservative abstraction preserves all the behaviors of the concrete system. This way, if a universal property is satisfied by an abstract model, then the property is correct also in the concrete model. On the contrary, false negatives may be produced. An abstract counterexample that does not correspond to any concrete behavior is called spurious. When such a counterexample is found, the abstraction is refined, i.e., we add enough information to the system in order to rule out the spurious counterexample. This process is repeated until the property is proved correct or a concrete counterexample is found. Such a loop is known as Counterexample-Guided Abstraction Refinement (CEGAR) [5]. Partitioning and abstraction have been extensively studied both in hardware and in software verification. For hardware, the abstraction is typically partitioned according to the design that usually is composed of submodules (see, e.g., [3,14]). For software, the partitioning is guided by the control-flow graph of the program. In particular, Lazy Abstraction (LA) [11] produces different levels of precision in different parts of the program. The abstraction can be refined locally to some control paths [11] or to some control locations [10]. In [15,16], motivated by previous work on Generalized Symbolic Trajectory Evaluation (GSTE) [20], Property-Driven Partitioning (PDP) has been proposed as a new hybrid approach to LTL model checking. In this approach, the property automaton A is constructed explicitly, but its product with the system is represented in a partitioned fashion. If the state space of the system is S and that of the property automaton is B , then PDP maintains a subset Q ⊆ S × B of the product space as a collection {Qb : b ∈ B } of sets, where each Qb = {s ∈ S | (s, b) ∈ Q} is represented symbolically. Thus, PDP maintains an array of BDDs instead of a single BDD to represent a subset of the product space. Based on extensive experimentation, it has been shown that the hybrid approach of PDP is superior to pure symbolic model checking. In GSTE, this partitioning technique is embedded into a quaternary-abstraction framework. Each partition stores a quaternary assignment (or a set of quaternary assignments, if we use symbolic indexing [19]). Quaternary abstraction, also known as Cartesian abstraction, is a powerful mechanism that merges a set of states into an abstract state: the value “unknown” is used for variables that have different assignments in the set. The use of this “unknown” value gives different levels of approximation. By combining PDP with quaternary abstraction, GSTE has a different abstraction precision for each partition. Building on the ideas of LA and GSTE, we explore in this paper the combination of PDP with predicate abstraction. We investigate a localized-abstraction framework, called partitioned abstraction, where each partition of PDP may contain a different abstraction. This way, we allow a flexible abstraction where every partition can be refined independently from the others. Intuitively, the state of the automaton corresponds to a particular phase of the operation of the circuit, and we expect that different partitions contain the appropriate abstractions for different phases. We adapted the CEGAR loop to this framework. In PDP, a counterexample is a sequence of state-partition pairs. When the counterexample is spurious, the failure of the concretization must happen in some partition. Thus, we tuned the refinement in order Property-Driven Partitioning for Abstraction Refinement 391 to refine the system only in those partitions that are necessary to rule out the spurious counterexample. Although partitioned abstraction is independent of any specific abstraction technique, we implemented it in N U SMV [4] in combination with localization reduction and the score-based refinement heuristic of [18]. We performed an experimental evaluation providing a comparison between the standard non-partitioned refinement loop with its partitioned counterpart. We evaluated the trade-off between the disadvantages of dealing with several abstractions at the same moment and the advantages of using coarser abstractions when possible. Our testing shows that this technique can be very efficient and can solve model-checking problems where the non-partitioned version fails. Unlike SMC, however, where partitioning always improved the performance of verification, here, partitioning the abstraction may involve high overhead. These experiments, for the first time1 , compare an abstraction-localization technique with a uniform one. The results show that abstraction partitioning is successful at allowing verification with coarser abstractions. In some cases this may lead to an order-of-magnitude improvement in running time. Nevertheless, in many cases the improvement in the coarseness of the abstractions does not compensate for the overhead of having to compute many abstractions. Overall, abstraction partitioning, in spite of its initial promise, does not lead to a consistent improvement in running time. 2 Background 2.1 Symbolic Model Checking Given a system M and a property ϕ, the model checking problem consists of checking if M is a model of ϕ. The semantics of M and ϕ are based on a shared set PROP of atomic propositions. Both the system and the property define languages over finite or infinite words. The alphabet of such languages is Σ = 2PROP. The system M is described with a set V of state variables. We use V ′ to denote the set of next state variables {v′ }v∈V , where v′ represents the next value of v. We represent the system M as a symbolic finite-state machine (FSM), i.e., a tuple V , I, T, L, where V is a set of state variables, I(V ) is a formula that represents the initial condition, T (V , V ′ ) is a formula that represents the transition relation, L = {La (V )}a∈PROP is a set of formulas that associate the atomic propositions with states. The set SV of states is given by all truth assignments to the variables V . Given a state s, we use s′ to denote the corresponding truth assignment to the next state variables, i.e. s′ = s[V ′ /V ]. A state s is labeled with the atomic proposition a iff s |= La (V ). A state s is initial iff s |= I(V ). Given two states s1 and s2 , there exists a transition between s1 and s2 iff s1 , s′2 |= T (V , V ′ ). A finite (resp., infinite) path of the FSM is a finite (resp. infinite) sequence of states π = s1 , ..., sk (resp. π = s1 , s2 , ... ) such that s1 is an initial state and, for 1 ≤ i < k (resp. 1 Henzinger et al. [10] compared BLAST with and without the technique of predicate localization and showed that it led to a running-time improvement in one case out of six. Since BLAST itself uses lazy abstraction, which can be viewed as a weak form of abstraction localization, it is hard to draw conclusions from that work about the advantage of abstraction localization. 392 R. Sebastiani, S. Tonetta, and M.Y. Vardi i ≥ 1), there exists a transition between si and si+1 . A lasso-shape path is a finite path π with a point l, 1 ≤ l < |π| (called loop-back point), such that sl = s|π| . Thus, a lassoshape path represents an infinite path (though, not all infinite paths are lasso shaped). An infinite word w ∈ Σω is accepted by the FSM M if there exists a path π of M such that π(i) |= Lw(i) (V ), for all i ≥ 1. The language LM of M is defined as the set of all infinite words accepted by M. Given the language Lϕ of the property ϕ, we say that the system M satisfies the property ϕ (M |= ϕ) iff LM ⊆ Lϕ . In other words, all paths of the system must correspond to a word accepted by the property. LTL model checking. A Büchi automaton (BA) is a tuple B , b0 , Σ, δ, F, where B is a set of states, b0 ∈ B is the initial state, δ ∈ B × Σ × B is the transition relation, and F ⊆ B is the fairness condition. A fair run of a BA A over the infinite word w = l1 , l2 , ... ∈ Σω is an infinite sequence of states π = b1 , b2 , ... such that b1 = b0 and, for i ≥ 1, (bi , li , bi+1 ) ∈ δ, and π(i) ∈ F for infinitely many i. The word w belongs to the language LA of A iff there exists a fair run over w. If ϕ is an LTL property, the model checking problem is usually solved by building a BA A¬ϕ with the complementary language of the property, computing its product with the system, and then checking for emptiness [17]. Extant model checkers use either a pure explicit-state approach, e.g., in SPIN [12], or a pure symbolic approach, e.g., in NuSMV [4]. To check language containment, a symbolic model checker implements a fixpoint algorithm [2]. Sets of states are manipulated by using basic set operations such as intersection, union, complementation, and the preimage and postimage operations. Since sets are represented by predicates on Boolean variables, intersection, union and complementation are translated into ∧, ∨ and ¬ respectively. The preimage and postimage operations are translated into the following formulas: preimage(Q) = ∃V ′ ((Q[V ′ /V ])(V ′ ) ∧ T (V , V ′ )), postimage(Q) = (∃V (Q(V ) ∧ T (V , V ′ )))[V /V ′ ]. 2.2 Abstraction ˆ T̂ , {L̂a }a∈PROP, a relaGiven two systems M = V , I, T, {La }a∈PROP and M̂ = Vˆ , I, ˆ tion H(V , V ) is a simulation relation iff the following conditions hold: – every initial state of M corresponds to an initial state of M̂; namely, if s |= I(V ), ˆ Vˆ ) and s, ŝ |= H(V , Vˆ ); then there exists a state ŝ of M̂ such that ŝ |= I( – every transition of M corresponds to a transition of M̂; namely, if s1 , ŝ1 |= H(V , Vˆ ), and s1 , s′2 |= T (V , V ′ ), then there exists a state ŝ2 of M̂ such that s2 , ŝ2 |= H(V , Vˆ ) and ŝ1 , ŝ′2 |= T̂ (V , V ′ ) ; – if two states are related by H, they must have the same labels; namely, if s, ŝ |= H(V , Vˆ ) then s |= La (V ) iff ŝ |= L̂a (Vˆ ), for all a ∈ PROP. If such relation exists, we say that M̂ is an abstraction of M, or M refines M̂ (M H M̂). Intuitively, every path π of M is “simulated” by some path π̂ of M̂ that accepts the same infinite word. In terms of languages, if M H M̂, then LM ⊆ LM̂ . Thus, simulation preserves properties, so that if M H M̂ and M̂ |= ϕ then M |= ϕ (though, in general, the reverse does not hold). Property-Driven Partitioning for Abstraction Refinement 393 Given the relation H, we define the abstraction function αH : SV → 2SVˆ and the concretization function γH : SVˆ → 2SV as follows: – αH (s) = {ŝ ∈ SVˆ | s, ŝ |= H(s, ŝ)}, – γH (ŝ) = {s ∈ SV | s, ŝ |= H(s, ŝ)}. We extend these functions to paths: if π̂ = sˆ1 , sˆ2 , ... is a path of M̂ then γH (π̂) is a set of paths of M such that π ∈ γH (π̂) iff π = s1 , s2 , ... is a path of M and si ∈ γH (sˆi ) for 1 ≤ i ≤ |π̂| (similarly for αH ). In predicate abstraction [9], the abstract state-space is described with a set of predicates; each predicate is represented by an abstract variable. Given an FSM M, we select a set P of predicates, such that each predicate P is a formula over the variables V that characterizes relevant facts of the system. For every P ∈ P, we introduce a new abstract Î variable vP . H is defined as follows: H(V , VP ) = P∈P vP ↔ P(V ), i.e., for every concrete state s, the corresponding abstract state ŝ assigns vP to true iff s |= P(V ). In the minimal (or most accurate) abstraction (wrt. P), every abstract transition corresponds to at least one concrete transition. This involves a quantification of the concrete variables that may be computationally expensive. The usual technique to get an approximation of the minimal abstraction is early quantification [6]: the existential quantifications are pushed inside conjunctions; in this way, we simplify the expressions by losing precision. Localization reduction. A particular case of abstraction uses the localization reduction technique [13]: suppose the system is defined functionally, i.e., M = V , I, T, L, where Î – I(V ) = ∃W ( v∈VÎv = fvI (W )) for some functions fvI and input variables W , – T (V , V ′ ) = ∃W ( v∈V v′ = fvT (V , W )) for some functions fvT and input variables W . Localization reduction consists in choosing a set of state variables, removing the corresponding constraints, and considering them as input variables. Let Vˆ ⊆ V , and ˘ = V \Vˆ . We can build the abstraction M̂ = Vˆ , I, ˆ T̂ , L̂ where: W ˆ Vˆ ) = ∃W (Î ˆ v = fvI (W )), – I( v∈V ˘ , W )), ˘ ∃W (Î v′ = f T (Vˆ , W – T̂ (Vˆ , Vˆ ′ ) = ∃W v∈Vˆ v ˘ (La (Vˆ , W ˘ )). – L̂ = {L̂a }a∈PROP, where L̂a (Vˆ ) = ∃W ˘ invisible variables (also known as abstracted or freed variables). We call W Remark 1. Note that localization reduction is a particular case of predicate abstraction with early quantification, where the set of predicates is given by the visible variables themselves, namely P = {v}v∈Vˆ . 2.3 Counterexample-Guided Abstraction Refinement Counterexample-guided Abstraction Refinement (CEGAR) is an iterative abstraction refinement methodology that starts from a very coarse abstraction and incrementally refine it until a result is yielded [5]. Every iteration corresponds to a false negative (spurious counterexample), which is removed from the abstract system by refining the abstraction. Given an FSM M and a property ϕ, the CEGAR method is based on the following steps: 394 R. Sebastiani, S. Tonetta, and M.Y. Vardi ¬v1 ¬vi−1 ¬vN−1 ¬vi ¬vN vN−1 vi−1 ... ... vN−1 vi−1 v1 b0 vi vN ¬vi−1 ¬vN−1 Fig. 1. N-bit counter ¬vN b1 ¬vN Fig. 2. BA for !GFvN 1. [Initial Abstraction] build an initial abstraction M̂ (M H M̂); 2. [Model Checking] check if M̂ |= ϕ: if it does, conclude that M |= ϕ; otherwise, produce a counterexample π; 3. [Concretization] check if the counterexample is concretizable (namely, γH (π) = / if it does, conclude that M |= ϕ; otherwise, call it spurious; 0): 4. [Refinement] build a new abstraction M̃ that refines M̂ (namely, M H̃ M̃ H̆ M̂) / turn M̂ into M̃ and go to step 2 and rules out π (γH̆ (π) = 0); Numerous techniques have been conceived for the refinement of the abstract FSM. Typically, they analyze the reason why a spurious counterexample cannot be concretized, and they find a set of variables that may be used to eliminate it. The purpose of the following example is twofold: on the one hand, we show how standard refinement works; on the other hand, we present a major limitation of this approach. Example 1. Consider a system with N symbolic variables, which represents a N-bit counter (Fig. 1). The system can be thought of as the composition of N modules, one for every variable. (The labels on the arcs represent values triggering the transition.) A property of the system is that the most significant bit vN becomes true infinitely often. The BA A, corresponding to the negation of the property, is shown in Fig. 2. Since only the most significant bit vN occurs in the property, the initial abstract system contains only one module (the rightmost module of Fig. 1); all the other variables are considered as inputs. The model checking procedure will produce a counterexample in which the abstract system eventually loops on the abstract state ¬vN . This counterexample is spurious because it does not correspond to any concrete path. In order to refine the abstraction and kill the spurious counterexample, we need to add vN−1 , because vN becomes true only when vN−1 is true. Similarly we have to add vN−2 and so on. This way, standard refinement techniques end up into the concrete system. More generally, notice that, in the context of localization reduction, it is not possible to prove the property, if even one of the variable is made invisible. Property-Driven Partitioning for Abstraction Refinement Product states b0 Product states b0 s0 s1 b1 395 s0 b1 Fig. 3. Preimage in PDP 3 Property-Driven Partitioning for Abstraction Refinement Property-driven partitioning. In property-driven partitioning (PDP), the property automaton, whose state space is often quite manageable, is represented explicitly, while the system, whose state space is typically exceedingly large, is represented symbolically. PDP partitions the symbolic state space according to an explicitly compiled property automaton. Let M = V , I, T, {La }a∈PROP be the FSM to be verified and A = B , b0 , Σ, δ, F the BA for the complemented property. The product between M and a A is the BA P , P0 , Σ, δP , F P , where – P = SV × B , – p ∈ P0 iff p = (s, b0 ) and s |= I(V ), – (p1 , l, p2 ) ∈ δP iff p1 = (s1 , b1 ), p2 = (s2 , b2 ), s1 , s′2 |= T (V , V ′ ), (b1 , a, b2 ) ∈ δ, and s1 |= La (V ) for every a ∈ l, – F P = {SV × F}. We consider the partitioning of the product state space P : {Pb }b∈B , where Pb = {p ∈ P : p = (s, b)}. Thus, a subset Q of P can be represented by the following set of states of M: {Qb }b∈B , where Qb = {s : (s, b) ∈ Q}. If Q1 = {Q1b }b∈B and Q2 = {Q2b }b∈B , we translate the set operations used in symbolic algorithms into: Q1 ∧ Q2 := {Q1b ∧ Q2b }b∈B , Q1 ∨ Q2 := {Q1b ∨ Q2b }b∈B , ¬Q := {¬Qb }b∈B , Ï preimage(Q) := { (b,a,b′ )∈δ preimage(Qb′ ) ∧ a}b∈B , Ï postimage(Q) := { (b′ ,a,b)∈δ postimage(Qb′ ∧ a)}b∈B . Example 2. Consider the model checking problem of Example 1. The property automaton A has two states, which yield two partitions in PDP. We use si to denote the state of M where the counter has value i. Consider the product of state s1 with the state b1 of A. Suppose we want to compute the preimage of such a state. First, we compute the preimage of s1 in M, which yields {s0 }. Then, we propagate this state to the other partitions. The result is shown in Fig. 3. 396 R. Sebastiani, S. Tonetta, and M.Y. Vardi s0 s1 s2 s3 b0 !v2 !v2 b1 Fig. 4. Example of partitioned abstraction All symbolic model-checking algorithms that operate on the product can be partitioned according to the property automaton, operating on a BDD array rather than on a single BDD (see [15,16]). 3.1 Combining PDP with Abstraction Given an FSM M = V , I, T, L and a BA A = B , b0 , Σ, δ, F, we define the abstract partitioned product P between M and A as a collection of FSMs {Mb }b∈B , one for every state b of the automaton A, with M Hb Mb . We use Vb , Ib , Tb , {Lab }a∈PROP to denote the abstraction Mb . A state of P is a pair (s, b) where the state s of M is in the partition corresponding to state b of A (namely, s ∈ SVb ). A state (s, b) is initial if b = b0 and s |= Ib (Vb ). There exists a transition between two states (s1 , b1 ) and (s2 , b2 ) iff 1) there exists l ∈ Σ such that (b1 , l, b2 ) ∈ δ, and s |= Lab1 (Vb1 ) for all a ∈ l, and 2) there exists s3 ∈ SVb such that s1 , s′3 |= Tb1 (Vb1 , Vb′1 ) and s2 ∈ αHb2 (γHb1 (s3 )). 1 Intuitively, given two partitions b1 and b2 , there is a mapping (αHb ◦ γHb ) between the 2 1 abstract states of the partition b1 and the abstract states of the partition b2 : the transitions from states of the partition b1 to states of the partition b2 are given by the composition of Tb1 with the mapping αHb2 ◦ γHb1 . Finally, a state (s, b) is fair iff b is fair. Note that the combination of partitioning with abstraction is independent of the particular abstraction technique used. In fact, in principle, you may use different abstraction techniques in different partitions. Example 3. Consider the model checking problem of Example 1 with N = 2. We drew a possible abstract partitioned product in Fig. 4: we used the abstract system described in the Example 1 in the partition corresponding to b0 , while the partition b1 contains the concrete system. / then L (M) ∩ L (A) = 0. / Theorem 1. If L (P ) = 0, Proof. Suppose L (M) ∩ L (A) = 0/ and w ∈ L (M) ∩ L (A). Then, there exists a path πM = s1 , s2 , ... of M and a fair path πA = b1 , b2 , ... of A over some word w = l1 , l2 , .... For every state b of A, M Mb so that there exists a path πbM ∈ αHb (πM ) and πbM accepts Property-Driven Partitioning for Abstraction Refinement ¬vN−1 ¬vN ¬v1 vN−1 vN−1 ... ¬vi−1 ¬vN−1 ¬vi ¬vN vi−1 v . . . N−1 vi−1 vN−1 Product states Product states vN b0 ¬vN−1 ⊤ v1 vi vN ¬vi−1 ¬vN−1 b0 s1 s0 b1 397 b1 Fig. 5. Preimage in partitioned abstraction w. Let πbM = sb1 , sb2 , ... for every state b of A. Consider the sequence πP of states of P where πP (i) = (sbi i , bi ). It is sufficient to prove that πP is a fair path of P . First, πP (1) is initial since b1 = b0 and sb11 |= Ib1 (Vb1 ). Second, for all i ≥ 1, (πP (i), πP (i + 1)) is a ′ i |= transition. In fact, 1) (bi , li , bi+1 ) ∈ δ, and sbi i |= Labi (Vbi ), for all a ∈ li , and 2) sbi i , sbi+1 b i+1 i )). Finally, πP is fair since πA is fair. ∈ αHbi+1 (γHbi (sbi+1 Tbi (Vbi , Vb′i ), and si+1 Notice that, if Mb = M for every b ∈ B , P is the hybrid structure we use in PDP. In this case, indeed, the opposite direction of Theorem 1 holds too. We generalize the preimage and postimage computation defined for PDP as follows: Ï preimage(Q) := { (b,a,b′ )∈δ preimageTb (αHb (γHb′ (Qb′ ))) ∧ a}b∈B Ï postimage(Q) := { (b′ ,a,b)∈δ postimageTb (αHb (γHb′ (Qb′ ∧ a)))}b∈B . Example 4. Consider the partitioned preimage computation of Example 2. Now, we recomputed the same preimage with a partitioned abstraction product: we will use the abstract system described in the Example 1 in the partition corresponding to b0 , while the partition b1 will contain the concrete system. In this case, before computing the component b0 of the preimage, we have to map the state s1 of partition b1 to the abstraction of partition b0 . s1 corresponds to the abstract state ¬vN , whose preimage (in the abstract system) yields the whole abstract state-space (denoted with ⊤). The result is shown in Fig. 5. 3.2 Property-Driven Partitioned Refinement We investigated a new technique that combines refinement with PDP. We call such technique Property-driven Partitioned Refinement (PDPR). The idea is that, instead of refining the abstraction globally, we localize the refinement to some partition. This way, at every iteration of the CEGAR loop, we refine the abstraction trying to keep it coarser than standard refinement. First, we adapted the standard refinement loop to LTL model checking (see Fig. 6). The LTL formula ¬ϕ (the negation of the property) is translated into a BA A (line 1). 398 R. Sebastiani, S. Tonetta, and M.Y. Vardi 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: Check(M, ϕ) A := LT L2BA(¬ϕ); M̂ := InitialAbsctraction(M, ϕ) P := Product(M̂, A); loop result := EmptyLanguage(P ); if result = T RUE then return M |= ϕ end if π := GenerateCounterexample(P ); result := Concretizable(π, M); if result = T RUE then return M |= ϕ end if Re f ineAbstraction(P , π); end loop Fig. 6. The procedure for checking if M |= ϕ The system is abstracted so that M̂ contains all the information related to the variables that occur in the property. The product P between M̂ and A is built (line 3). The emptiness of P is checked with the Emerson-Lei algorithm [8] (line 5). If a counterexample is found, we check if it corresponds to a concrete path (line 9) with Bounded Model Checking (BMC) [1], as proposed by [7]. Since the counterexample may be infinite, we consider lasso-shape paths. This way, if the counterexample is given by the abstract states ŝ1 , ..., ŝk with ŝl = ŝk , we then generate the BMC formula I(V0 ) ∧  1≤i<k T (Vi , Vi+1 ) ∧  1≤i≤k Si (Vi ) ∧  v =v l k v∈V where the predicates Si identify the abstract states ŝi . If the formula is not satisfiable, we refine the abstraction (line 12). Note that this concretization step is sound but not complete: we are fixing a specific lasso shape, so it may happen that the above formula is not satisfiable, but the counterexample is concretizable. The completeness of the whole algorithm is guaranteed by the fact that the refinement procedure is monotone, since at every iteration we add some information; eventually, the concrete counterexample will be found. PDPR follows the same schema, though the product is handled differently. The main difference between the standard framework and PDPR consists of the refinement procedure, which refines the abstract system only in some partitions. The refinement depends on the type of abstraction we are considering. In the following, we adopt the framework of localization reduction. The refinement (Fig. 7) consists in computing the score of each invisible variable (line 5). We pick the best scored variable (line 7) and we add it to the system (line 8). We repeat the procedure until the refined abstraction rules out the spurious counterexample. This is checked with a symbolic fix-point simulation (line 12). In the case of PDPR, the refinement is similar (Fig. 8). The only difference is that we score pairs of variables and partitions (line 8): if (v, b) is the best scored pair, we add v to the system of partition b (line 13). Property-Driven Partitioning for Abstraction Refinement 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: Re f ineAbstraction(P , π) // P is the product of M̂ and A repeat / V˜ := 0; while |V˜ | < threshold do for all v ∈ V˘ do ComputeScore(v, M̂, π); end for v := BestVar; V˜ := V˜ ∪ {v}; end while M̂ := Abstract(M̂, V˜ ); P := Product(M̂, A); result := Simulate(P , π); until result = FALSE Fig. 7. Refinement function 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 399 Re f ineAbstraction(P , π) // P is the partitioned product of {M̂b }b∈B and A repeat for all b ∈ B do / V˜ b := 0; end for while | ∪b∈B V˜ b | < threshold do for all b ∈ B do for all v ∈ V˘ b do ComputeScore(v, b, M̂, π); end for end for v := BestVar; b := BestPartition; V˜ b := V˜ b ∪ {v}; end while for all b ∈ B do M̂b := Abstract(M̂b , V˜ b ); end for P := PartitionedProduct({M̂b }b∈B , A); result := Simulate(P , π); until result = FALSE Fig. 8. Partitioned version of the refinement function Example 5. Consider the model checking problem of Example 1. PDPR starts with two identical systems, which is the system containing only the constraints of vN . Thus, PDPR will produce the same spurious counterexample described in the Example 1, which was obtained with the standard abstraction. The refinement heuristic finds that we have to add vN−1 in the partition b1 , in order to force vN to become true. Similarly, we have to add vN−2 and all the other variables to partition b1 . PDPR ends the refinement loop with the concrete system in partition b1 . Though, the abstract system of partition b0 is never touched. In the subsequent emptiness checking, we use the partitioned abstraction described in Example 4. This abstraction is coarser than the concrete system (necessary for a standard refinement), but it is sufficiently fine to prove the property. 4 Experimental Results We now evaluate the efficiency of PDPR in practice by testing it over some scalable verification models. The purpose of this experimental evaluation is to provide a comparison between the standard non-partitioned refinement approach and PDPR. Thus, we fix an abstraction refinement setting, in which we use PDP as representation of the product, and we compare the partitioned versus the non-partitioned refinement loop. Our tests do not provide a direct comparison with the numerous state-of-the-art refinement techniques because the partitioned abstraction is orthogonal to them. Rather, we try to 400 R. Sebastiani, S. Tonetta, and M.Y. Vardi 20 100000 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 18 16 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 10000 14 1000 12 10 100 8 10 6 4 1 2 0 0.1 10 15 20 25 30 35 40 10 15 20 25 30 35 40 Fig. 9. Experiments on the N-bit-counter example. X axis: N. Y axis: Left plot: average number of invisible variables; Right plot: CPU time (secs). 50 100000 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 45 40 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 10000 35 1000 30 25 100 20 10 15 10 1 5 0 0.1 2 2.5 3 3.5 4 4.5 5 5.5 6 2 2.5 3 3.5 4 4.5 5 5.5 6 Fig. 10. Experiments on the memory example. X axis: number of bits for memory addresses. Y axis: Left plot: average number of invisible variables; Right plot: CPU time (secs). 30 10000 non partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 28 26 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 1000 24 22 100 20 18 10 16 14 1 10 10.5 11 11.5 12 12.5 13 13.5 14 10 10.5 11 11.5 12 12.5 13 13.5 14 Fig. 11. Experiments on the stack example. X axis: stack’s capacity. Y axis: Left plot: average number of invisible variables; Right plot: CPU time (secs). evaluate the trade-off between the disadvantages of dealing with several abstractions at the same moment and the advantages of using coarser abstractions when possible. N U SMV PDP is an extension of N U SMV [4] that uses a PDP representation to perform the emptiness checking of the product. We have enhanced N U SMV PDP with an abstraction refinement technique that implements localization abstraction with the Property-Driven Partitioning for Abstraction Refinement 80 100000 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 70 60 401 10000 50 40 1000 non partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order inc-vars 30 20 10 10.5 11 11.5 12 12.5 100 13 13.5 14 10 10.5 11 11.5 12 12.5 13 13.5 14 Fig. 12. Experiments on the gas-station example. X axis: number of customers. Y axis: Left plot: average number of invisible variables; Right plot: CPU time (secs). 350 1e+06 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 300 non-partitioned partitioned with order dec-trans partitioned with order inc-vars partitioned with order dec-vars 100000 250 10000 200 1000 150 100 100 10 50 0 1 10 15 20 25 30 35 40 45 50 55 60 10 15 20 25 30 35 40 45 50 55 60 Fig. 13. Experiments on the mutual-exclusion example. X axis: number of processes. Y axis: Left plot: average number of invisible variables; Right plot: CPU time (secs). refinement heuristic of [18]2 . In these settings, at every refinement iteration, a new abstract system is computed and composed with the property automaton; then, a PDPbased search checks the emptiness of the product. We tested this standard abstraction refinement technique against its partitioned counterpart. In the partitioned refinement, we fixed an ordering of the partitions. This gives a priority to the partition to be refined when two invisible variables of different partitions get the same score. We selected three relevant orderings: – [inc-vars] an increasing ordering according to the number of invisible variables in each partitions; this way, we prefer refining the abstraction in those partitions that had already been refined (thus, previously guilty of other spurious counterexample); – [dec-vars] a decreasing ordering according to the number of invisible variables in each partitions; this way, we prefer refining the coarsest abstraction; – [dec-trans] a decreasing ordering according to the number of outgoing transitions in the property automaton; this tries to alleviate the cost of the mapping between abstractions: every time we have to traverse backwardly a transition that starts from b, we have to quantify over the invisible variables of the partition b. 2 Unlike [18], we use one counterexample per refinement iteration. 402 R. Sebastiani, S. Tonetta, and M.Y. Vardi We performed our tests on the Rice Terascale Cluster (RTC)3 , a TeraFLOP Linux cluster based on Intel Itanium 2 Processors. We run N U SMV with a static variable ordering so that, when the abstraction is the same, the non-partitioned and the partitioned version perform equally and the efficiency is not biased by a dynamic variable reordering. We first run both the non-partitioned and the partitioned version of abstraction refinement on the N-bit-counter system of Example 1. Then, we tested the algorithm over other scalable systems: namely, memory, stack, mutual-exclusion example, and gas station. In each system we scale up a parameter N so that the number of symbolic variables used to describe the system is linear in N (apart from the memory example where the number of variables is exponential in N). – The memory example implements standard read and write functionalities. In order to be able to scale up the verification, we use just one bit for the data so that we can write and read only two values. We scale up the number N of bits used for the memory addresses. We verify that if we write a value on a particular address, next time we read from it we obtain the same value, unless another write operation was performed in the meanwhile. Inspired by the work on GSTE of [20], we express the property by enumerating each address case so that the property automaton contains different subgraph for each case. The partitioned abstraction should take benefit from this, by keeping in each subgraph only the information regarding that particular memory address. – In the stack example, we have the standard pop and push functions. In this case, scalability is given by the maximum size N of the stack. The property says that whenever we push an element on the stack, and later the stack results empty, there must be a pop call in the middle – In the mutual-exclusion example, N processes non-deterministically try to access the critical session. The access is controlled by the main module, which guarantees that a process does not wait forever. The property says that, if a process is the only one that is waiting, then it accesses the critical session in one step. – In the gas-station example, there are N customers who want to use one pump. They have to prepay an operator who then activates the pump. When the pump has charged, the operator give the change to the customer. The property says the if a customer pays before another one, then she must be server first. We verified many other properties, but the initial abstraction was fine enough to prove the property; in one case the final abstraction coincided (with both techniques) with the concrete system. We reported only the other cases, where the refinement had been effectively useful to prove the property.4 The results are shown in Figures 9, 11, 12, and 13. In every plot, we scale up the size of the system. The plots on the left of each figure present the number of invisible variables (the average in the case of partitioned abstraction) of the final abstraction. The plots on the right show the time in seconds plotted in log scale. These results show that partitioned abstraction is successful at allowing verification with significantly coarser abstractions. The coarsest abstraction is obtained when the 3 4 http://rcsg.rice.edu/rtc/ All data available at www.inf.unisi.ch/postdoc/tonetta/TACAS07-PDP Property-Driven Partitioning for Abstraction Refinement 403 states of the property automaton correspond to particular phases of the circuit, like in the case of the memory example (Fig. 10). In some cases (Figs. 9 and 12), this leads to an order-of-magnitude improvement in running time. Nevertheless, in some cases (Figs. 10, 11 and 13), the coarser abstractions do not compensate for the overhead of having to refine several abstractions. It is interesting to note that no single refinement order dominates in our experiments; the best refinement order seems model and property specific. For lack of space we did not show the plots with space performances but they present the same pattern of time plots. The only exception is the memory example, which is particularly interesting because the number of invisible variables grows exponentially (hiding an exponential amount of design with an order of magnitude improvement in space), but the verification time degrades by an order of magnitude. 5 Conclusions We investigated a new approach for abstraction refinement, in which the abstraction is not uniform; rather, like in [10], the abstraction is localized and presents different precisions in different parts of the state space. We combined this idea with property-driven partitioning, which partitions the product state space for efficient LTL model checking. We presented a new refinement algorithm that exploits this approach by refining the abstraction only in those partitions where it is necessary. This way, as our tests confirmed, we can prove the correctness of a system with a higher level of abstraction, leading, in some cases, to dramatic performance improvement. Unlike, however, in SMC, where the partitioning almost always speeds up the verification time, here the use of partitioning is less compelling, as the overhead of having to refine multiple abstractions is in some cases too high, leading to performance degradation. This conclusion is consistent with the results in [10], where predicate localization improves performance only in a small number of test cases. We note that most of time is spent in the refinment step, meaning that, the coarser abstractions do not compensate for the overhead of having to refine several abstractions. Since the partitioned abstractions are much coarser, we conjecture that we could afford to use less accurate but faster refinement techniques in order to speed up the overall verification time. This is left for future research. References 1. A. Biere, A. Cimatti, E. M. Clarke, and Y. Zhu. Symbolic Model Checking without BDDs. In Proceedings of TACAS’99, pages 193–207, 1999. 2. J.R. Burch, E.M. Clarke, K.L. McMillan, D.L. Dill, and L.J. Hwang. Symbolic Model Checking: 1020 States and Beyond. Information and Computation, 98(2):142–170, 1992. 3. H. Cho, G.D. Hachtel, E. Macii, M. Poncino, and F. Somenzi. Automatic state space decomposition for approximate FSM traversal based on circuit analysis. IEEE Trans. on CAD of Integrated Circuits and Systems, 15(12):1451–1464, 1996. 4. A. Cimatti, E.M. Clarke, F. Giunchiglia, and M. Roveri. N U SMV: a new Symbolic Model Verifier. In Proceedings of CAV’99, pages 495 - 499, 1999. 5. E.M. Clarke, O. Grumberg, S. Jha, Y. Lu, and H. Veith. Counterexample-Guided Abstraction Refinement. In Proceedings of CAV’00, pages 154 - 169, 2000. 404 R. Sebastiani, S. Tonetta, and M.Y. Vardi 6. E.M. Clarke, O. Grumberg, and D.E. Long. Model Checking and Abstraction. ACM Trans. Program. Lang. Syst., 16(5):1512–1542, 1994. 7. E.M. Clarke, A. Gupta, J.H. Kukula, and O. Strichman. SAT Based Abstraction-Refinement Using ILP and Machine Learning Techniques. In Proceedings of CAV’02, pages 265–279, 2002. 8. E.A. Emerson and C.L. Lei. Efficient Model Checking in Fragments of the Propositional µ-Calculus. In Proceedings of the LICS’86, pages 267 - 278, 1986. 9. S. Graf and H. Saı̈di. Construction of Abstract State Graphs with PVS. In Proceedings of CAV’97, pages 72 - 83, 1997. 10. T.A. Henzinger, R. Jhala, R. Majumdar, and K.L. McMillan. Abstractions from proofs. In Proceedings of POPL’04, pages 232 - 244, 2004. 11. T.A. Henzinger, R. Jhala, R. Majumdar, and G. Sutre. Lazy abstraction. In Proceedings of POPL’02, pages 58 - 70, 2002. 12. G.J. Holzmann. The SPIN model checker: Primer and reference manual. Addison Wesley, 2003. 13. R.P. Kurshan. Computer Aided Verification of Coordinating Processes. Princeton University Press, 1994. 14. M.D. Nguyen, D. Stoffel, M. Wedler, and W. Kunz. Transition-by-Transition FSM Traversal for Reachability Analysis in Bounded Model Checking. In Proceedings of ICCAD’05, 2005. 15. R. Sebastiani, E. Singerman, S. Tonetta, and M. Y. Vardi. GSTE is Partitioned Model Checking. In Proceedings of CAV’04, pages 229 - 241, 2004. 16. R. Sebastiani, S. Tonetta, and M.Y. Vardi. Symbolic Systems, Explicit Properties: On Hybrid Approaches for LTL Symbolic Model Checking. In Proceedings of CAV’05, pages 350 - 363, 2005. 17. M.Y. Vardi and P. Wolper. An Automata-Theoretic Approach to Automatic Program Verification. In Proceedings of LICS’86, pages 332 - 344. IEEE Computer Society, 1986. 18. C. Wang, B. Li, H.S. Jin, G.D. Hachtel, and F. Somenzi. Improving Ariadne’s Bundle by Following Multiple Threads in Abstraction Refinement. In Proceedings of ICCAD’03, pages 408 - 415, 2003. 19. J. Yang and C.-J. H. Seger. Introduction to Generalized Symbolic Trajectory Evaluation. IEEE Transactions on Very Large Scale Integration Systems, 11(3), 2003. 20. J. Yang and C.-J.H. Seger. Generalized Symbolic Trajectory Evaluation - Abstraction in Action. In Proceedings of FMCAD’02, pages 70 - 87, 2002.