Software Model Checking via Large-Block Encoding
Dirk Beyer ∗
Alessandro Cimatti †
Alberto Griggio ∗‡
Simon Fraser University
FBK-irst, Trento
University of Trento &
Simon Fraser University
Simon Fraser University
University of Trento
A fundamental source of inefficiency of this approach is the
fact that the control-flow of the program can induce a huge
number of paths (and nodes) in the ART, which are explored
independently of each other.
We propose a novel, broader view on ART-based software
model checking, where a much more compact abstract space
is used, resulting thus in a much smaller number of paths
to be enumerated by the ART. Instead of using edges that
represent single program operations, we encode entire parts
of the program in one edge. In contrast to SBE, we call our
new approach large-block encoding (LBE). In general, the new
encoding may result in an exponential reduction of the number
of ART nodes.
The generalization from SBE to LBE has two main consequences. First, LBE requires a more general representation
of abstract states than SBE. SBE is typically based on mere
conjunctions of predicates. Because the LBE approach summarizes large portions of the control flow, conjunctions are not
sufficient, and we need to use arbitrary Boolean combinations
of predicates to represent the abstract states. Second, LBE
requires a more accurate abstraction in the abstract-successor
computations. Intuitively, an abstract edge represents many
different paths of the program, and therefore it is necessary
that the abstract-successor computations take the relationships
between the predicates into account.
In order to make this generalization practical, we rely on
efficient solvers for satisfiability modulo theories (SMT). In
particular, enabling factors are the capability of performing
Boolean reasoning efficiently (e.g., [21]), the availability of
effective algorithms for abstraction computation (e.g., [11],
[18]), and interpolation procedures to extract new predicates [9], [12].
Considering Boolean abstraction and large-block encoding in addition to the traditional techniques, we obtain the
following interesting observations: (i) whilst the SBE approach requires a large number of successor computations, the
LBE approach reduces the number of successor computations
dramatically (possibly exponentially); (ii) whilst Cartesian
abstraction can be efficiently computed with a linear number
of SMT solver queries, Boolean abstraction is expensive to
compute because it requires an enumeration of all satisfiable
assignments for the predicates. Therefore, two combinations of
the above strategies provide an interesting tradeoff: The combination of SBE with Cartesian abstraction was successfully
implemented by tools like B LAST and SLAM. We investigate
the combination of LBE with Boolean abstraction, by first
Abstract—Several successful approaches to software verification are based on the construction and analysis of an abstract
reachability tree (ART). The ART represents unwindings of the
control-flow graph of the program. Traditionally, a transition
of the ART represents a single block of the program, and
therefore, we call this approach single-block encoding (SBE).
SBE may result in a huge number of program paths to be
explored, which constitutes a fundamental source of inefficiency.
We propose a generalization of the approach, in which transitions
of the ART represent larger portions of the program; we call
this approach large-block encoding (LBE). LBE may reduce the
number of paths to be explored up to exponentially. Within
this framework, we also investigate symbolic representations: for
representing abstract states, in addition to conjunctions as used
in SBE, we investigate the use of arbitrary Boolean formulas;
for computing abstract-successor states, in addition to Cartesian
predicate abstraction as used in SBE, we investigate the use of
Boolean predicate abstraction. The new encoding leverages the
efficiency of state-of-the-art SMT solvers, which can symbolically
compute abstract large-block successors. Our experiments on
benchmark C programs show that the large-block encoding
outperforms the single-block encoding.
I. I NTRODUCTION
Software model checking is an effective technique for
software verification. Several advances in the field have lead
to tools that are able to verify programs of considerable size,
and show significant advantages over traditional techniques
in terms of precision of the analysis (e.g., SLAM [3] and
B LAST [6]). However, efficiency and scalability remain major
concerns in software model checking and hamper the adaptation of the techniques in industrial practice. Several successful
tools for software model checking are based on the construction and analysis of an abstract reachability tree (ART), and
predicate abstraction is one of the favorite abstract domains.
The ART represents unwindings of the control-flow graph
of the program. The search is usually guided by the control
flow of the program. Nodes of the ART typically consist of
the control-flow location, the call stack, and formulas that
represent the data states. During the refinement process, the
ART nodes are incrementally refined.
In the traditional ART approach, each program operation
(assignment operation, assume operation, function call, function return) is represented by a single edge in the ART.
Therefore, we call this approach single-block encoding (SBE).
∗ Supported in part by the Canadian NSERC grant RGPIN 341819-07 and
by the SFU grant PRG 06-3. † Supported in part by the European Commission
under project FP7-2007-IST-1-217069 COCONUT. ‡ Supported in part by
SRC/GRC under Custom Research Project 2009-TJ-1880 WOLFLING and
by MIUR under PRIN project 20079E5KM8 002.
978-1-4244-4966-8/09/$25.00 ©2009 IEEE
M. Erkan Keremoglu ∗ Roberto Sebastiani ‡
25
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
with the generation of verification conditions as performed by
static program verifiers like S PEC # [4] or C ALYSTO [1].
formally defining LBE in terms of a summarization of the
control-flow automaton for the program, and then implementing this LBE approach together with a Boolean predicate
abstraction. We evaluate the performance and precision by
comparing it with the model checker B LAST and with an
own implementation of the traditional approach. Our own
implementation of the SBE and LBE approach is integrated
as a new component into CPACHECKER [8]1 . The experiments
show that, despite the simplicity of the idea underlying LBE,
our new approach outperforms the previous approach.
Structure. Section II provides the necessary background. Section III explains our contribution in detail. We experimentally
evaluate our novel approach in Sect. IV. In Sect. V, we draw
some conclusions and outline directions for future research.
II. BACKGROUND
A. Programs and Control-Flow Automata
We restrict the presentation to a simple imperative programming language, where all operations are either assignments or
assume operations, and all variables range over integers.2 We
represent a program by a control-flow automaton (CFA). A
CFA A = (L, G) consists of a set L of program locations,
which model the program counter l , and a set G ⊆ L×Ops×L
of control-flow edges, which model the operations that are
executed when control flows from one program location to
another. The set of variables that occur in operations from Ops
is denoted by X. A program P = (A, l0 , lE ) consists of a CFA
A = (L, G) (which models the control flow of the program),
an initial program location l0 ∈ L (which models the program
entry) such that G does not contain any edge (·, ·, l0 ), and
a target program location lE ∈ L (which models the error
location).
A concrete data state of a program is a variable assignment
c : X → Z that assigns to each variable an integer value. The
set of all concrete data states of a program is denoted by C. A
set r ⊆ C of concrete data states is called region. We represent
regions using first-order formulas (with free variables from
X): a formula ϕ represents the set S of all data states c that
imply ϕ (i.e., S = {c | c |= ϕ}). A concrete state of a program
is a pair (l, c), where l ∈ L is a program location and c is a
concrete data state. A pair (l, ϕ) represents the following set of
all concrete states: {(l, c) | c |= ϕ}. The concrete semantics of
an operation op ∈ Ops is defined by the strongest postcondition operator SPop : for a formula ϕ, SPop (ϕ) represents the
set of data states that are reachable from any of the states
in the region represented by ϕ after the execution of op.
Given a formula ϕ that represents a set of concrete data
states, for an assignment operation s := e, we have
s : ϕ[s→
∧ (s = e[s→
); and for an assume
SPs:=e (ϕ) = ∃
s]
s]
operation assume(p), we have SPassume(p) (ϕ) = ϕ ∧ p.
A path σ is a sequence (op 1 , l1 ), ..., (op n , ln ) of pairs
of operations and locations. The path σ is called program
path if for every i with 1 ≤ i ≤ n there exists a
CFA edge g = (li−1 , op i , li ), i.e., σ represents a syntactical walk through the CFA. The concrete semantics for a
program path σ = (op 1 , l1 ), ..., (op n , ln ) is defined as
the successive application of the strongest postoperator for
each operation: SPσ (ϕ) = SPop n (...SPop 1 (ϕ)...). The set
of concrete states that result from running σ is represented
by the pair (ln , SPσ (true)). A program path σ is feasible if
Example. We illustrate the advantage of LBE over SBE on the
example program in Fig. 1 (a). In SBE, each program location
is modeled explicitly, and an abstract-successor computation is
performed for each program operation. Figure 1 (b) shows the
structure of the resulting ART. In the figure, abstract states are
drawn as ellipses, and labeled with the location of the abstract
state; the arrows indicate that there exists an edge from the
source location to the target location in the control-flow. The
ART represents all feasible program paths. For example, the
leftmost program path is taking the ‘then’ branch of every ‘if’
statement. For every edge in the ART, an abstract-successor
computation is performed, which potentially includes several
SMT solver queries. The problems given to the SMT solver
are usually very small, and the runtime sums up over a large
amount of simple queries. Therefore, model checkers that are
based on SBE (like B LAST) experience serious performance
problems on programs with such an exploding structure (cf. the
test_locks examples in Table I). In LBE, the control-flow
graph is summarized, such that control-flow edges represent
entire subgraphs of the original control-flow. In our example,
most of the program is summarized into one control-flow edge.
Figure 1 (c) shows the structure of the resulting ART, in which
all feasible paths of the program are represented by one single
edge. The exponential growth of the ART does not occur.
Related Work. The model checkers SLAM and B LAST are
typical examples for the SBE approach [3], [6], both based on
counterexample-guided abstraction refinement (CEGAR) [13].
The tool SATABS is also based on CEGAR, but it performs
a fully symbolic search in the abstract space [15]. In contrast,
our approach still follows the lazy-abstraction paradigm [17],
i.e., it abstracts and refines chunks of the program “on-thefly”. The work of McMillan is also based on lazy abstraction,
but instead of using predicate abstraction as abstract domain,
he directly uses Craig interpolants from infeasible error paths,
thus avoiding abstract-successor computations [19]. A different approach to software model checking is bounded model
checking (BMC), with the most prominent example CBMC
[14]. Programs are unrolled up to a given depth, and a formula
is constructed which is satisfiable iff one of the considered
program executions reaches a certain error location. The BMC
approaches are targeted towards discovering bugs, and can not
be used to prove program safety. Finally, the summarizations
performed in our large-block encoding bear some similarities
1 Available
at http://www.sosy-lab.org/∼dbeyer/CPAchecker
2 Our implementation is based on CPACHECKER , which operates on
C programs that are given in the C IL intermediate language [20]; function
calls are supported.
26
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
L1
ERROR
L2
L1:
L2:
L3:
L4:
L5:
L6:
L7:
L8:
L9:
L10:
L11:
L12:
L13:
ERR:
if(p1) {
x1 = 1;
}
if(p2) {
x2 = 2;
}
if(p3) {
x3 = 3;
}
if(p1) {
if (x1 != 1) goto ERR;
}
if (p2) {
if (x2 != 2) goto ERR;
}
if (p3) {
if (x3 != 3) goto ERR;
}
return EXIT_SUCCESS;
return EXIT_FAILURE;
(a) Example C program
Fig. 1.
L4
L5
L6
L8
ERROR
L7
L8
L9
L8
L9
ERROR
L9
ERROR
L9
L11
L10
L11
ERROR
L12
L4
L5
L5
L6
L7
L7
L8
L9
L9
L8
L9
L10
L11
L10
L11
L10
L11
L12
L7
L7
L8
L9
ERROR
L11
L12
L13
L10
L11
L12
L13
L13
L9
L10
L11
L13
L9
L9
ERROR
L10
L11
L3
L5
L6
L7
L3
ERROR
L6
L7
L7
L8
L8
L10
L11
L12
L13
L9
L11
L11
ERROR
L12
ERROR
L13
L12
L13
L13
L13
L13
ERROR
L1
L12
L13
L13
L13
ERROR
(b) ART for SBE
(c) ART for LBE
Example program and corresponding ARTs for SBE and LBE; this example was considered as verification challenge for ART-based approaches
SPσ (true) is satisfiable. A concrete state (ln , cn ) is called
reachable if there exists a feasible program path σ whose final
location is ln and such that cn |= SPσ (true). A location l is
reachable if there exists a concrete state c such that (l, c) is
reachable. A program is safe if lE is not reachable.
whose corresponding propositional variable occurs positive in
the assignment. The disjunction of all such conjunctions is
the Boolean predicate abstraction for ϕ. The abstract strongest
postoperator SPπ for a predicate abstraction with precision π
π
transforms the abstract state ϕπB into its successor ϕ′ B for
π
a program operation op, written as ϕ′ B = SPπop (ϕπB ), if
π
ϕ′ B is the Boolean predicate abstraction of SPop (ϕπB ), i.e.,
ϕ′ πB = (SPop (ϕπB ))πB . For more details, we refer the reader to
the work of Lahiri et al. [18].
B. Predicate Abstraction
Let P be a set of predicates over program variables in a
quantifier-free theory T . A formula ϕ is a Boolean combination of predicates from P. A precision for a formula is a finite
subset π ⊂ P of predicates.
C. ART-based Software Model Checking with SBE
Cartesian Predicate Abstraction. Let π be a precision. The
Cartesian predicate abstraction ϕπC of a formula ϕ is the
strongest conjunction
of predicates from π that is entailed by
{p ∈ π | ϕ ⇒ p}. Such a predicate abstraction of
ϕ: ϕπC :=
a formula ϕ, which represents a region of concrete program
states, is used as an abstract state (i.e., an abstract representation of the region) in program verification. For a formula ϕ and
a precision π, the Cartesian predicate abstraction ϕπC of ϕ can
be computed by |π| SMT-solver queries. The abstract strongest
postoperator SPπ for a predicate abstraction with precision π
π
transforms the abstract state ϕπC into its successor ϕ′ C for
π
′π
π
a program operation op, written as ϕ C = SPop (ϕC ), if
π
ϕ′ C is the Cartesian predicate abstraction of SPop (ϕπC ), i.e.,
′π
ϕ C = (SPop (ϕπC ))πC . For more details, we refer the reader to
the work of Ball et al. [2].
Boolean Predicate Abstraction. Let π be a precision. The
Boolean predicate abstraction ϕπB of a formula ϕ is the
strongest Boolean combination of predicates from π that is
entailed by ϕ. For a formula ϕ and a precision π, the Boolean
predicate abstraction ϕπB of ϕ can be computed by querying an
SMT solver in the following way: For each predicate pi ∈ π,
we introduce a propositional variable vi . Now we ask an SMT
solver to enumerateall satisfying assignments of v1 , ..., v|π|
in the formula ϕ ∧ pi ∈π (pi ⇔ vi ). For each satisfying assignment, we construct a conjunction of all predicates from π
The precision for a program is a function Π : L → 2P ,
which assigns to each program location a precision for a formula. An ART-based algorithm for software model checking
takes an initial precision Π (which is typically very coarse) for
the predicate abstraction, and constructs an ART for the input
program and Π. An ART is a tree whose nodes are labeled
with program locations and abstract states [6] (i.e., n = (l, ϕ)).
For a given ART node, all children nodes are labeled with
successor locations and abstract successor states, according
to the strongest postoperator and the predicate abstraction. A
node n = (l, ϕ) is called covered if there exists another ART
node n′ = (l, ϕ′ ) that entails n (i.e., s.t. ϕ′ |= ϕ). An ART is
called complete if every node is either covered or all possible
abstract successor states are present in the ART as children of
the node. If a complete ART is constructed and the ART does
not contain any error node, then the program is considered
correct [6]. If the algorithm adds an error node to the ART,
then the corresponding path σ is checked to determine if σ
is feasible (i.e., if the corresponding concrete program path
is executable) or infeasible (i.e., if there is no corresponding
program execution). In the former case the path represents
a witness for a program bug. In the latter case the path is
analyzed, and a refinement Π′ of Π is generated, such that the
same path cannot occur again during the ART exploration. The
concept of using an infeasible error path for abstraction refine-
27
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
1: while
1: while
ment is called counterexample-guided abstraction refinement
(CEGAR) [13]. The concept of iteratively constructing an ART
and refining only the precisions along the considered path is
called lazy abstraction [17]. Craig interpolation is a successful
approach to predicate extraction during refinement [16]. After
refining the precision, the algorithm continues with the next
iteration, using Π′ instead of Π to construct the ART, until
either a complete error-free ART is obtained, or an error is
found (note that the procedure might not terminate). For more
details and a more in-depth illustration of the overall ART
algorithm, we refer the reader to the B LAST article [6].
In order to make the algorithm scale on practical examples,
implementations such as B LAST or SLAM use the simple
but coarse Cartesian abstraction, instead of the expensive
but precise Boolean abstraction. Despite its potential imprecision, Cartesian abstraction has been proved successful for
the verification of many real-world programs. In the SBE
approach, given the large number of successor computations,
the computation of the Boolean predicate abstraction is in fact
too expensive, as it may require an SMT solver to enumerate
an exponential number of assignments on the predicates in
the precision, for each single successor computation. The
reason for the success of Cartesian abstraction if used together
with SBE, is that for a given program path, state overapproximations that are expressible as conjunctions of atomic
predicates —for which Boolean and Cartesian abstractions are
equivalent— are often good enough to prove that the error
location is not reachable in the abstract space.
L1: while (i>0) {
[i > 0]
[i ≤ 0]
L2:
if (x==1) {
2:
if
L3:
z = 0;
7
[x == 1]
[x = 1]
} else {
L4:
z = 1;
3: then
4: else
}
z=0
z=1
L5:
i = i-1;
5
i=i−1
L6: }
[i > 0]
[i ≤ 0]
2: if
[x == 1]
[x = 1]
3: then
7
4: else
z=0
z=1
5
i=i−1
6
1: while
[i > 0]
[i ≤ 0]
2: if
[x = 1]
z=1
[i ≤ 0]
[i ≤ 0]
2: if
7
7
[x == 1]
z=0
1: while
1: while
[i > 0]
[x == 1] [x = 1]
z=0 z=1
[i > 0]
7
[x == 1] [x = 1]
z=0 z=1
i=i−1
5
i=i−1
5
i=i−1
Fig. 2. CFA Transformation: a) Program, b) CFA, c)–e) Intermediate CFAs,
f) CFA-Summary. In the CFAs, assume(p) is represented as [p], op 1 ; op 2
is represented by drawing op 2 below op 1 , and op 1 op 2 by drawing op 2
beside op 1
Rule 2 (Choice). If L2 = {l1 , l2 } and A|L2 = (L2 , G2 )
is the subgraph of A with nodes
l1
l1
from L2 and the set G2 of
edges contains the two edges op
op 2
op 1 op 2
1
(l1 , op 1 , l2 ) and (l1 , op 2 , l2 ),
l2
l2
then we change the CFA A in
the following way: (1) we remove the two edges (l1 , op 1 , l2 )
and (l1 , op 2 , l2 ) from G and add the edge (l1 , op 1 op 2 , l2 )
to G, where SPop 1 op 2 (ϕ) = SPop 1 (ϕ) ∨ SPop 2 (ϕ). (Note
that there might be a backwards edge (l2 , ·, l1 ).)
Let P = (A, l0 , lE ) be a program and let A′ be a CFA. The
CFA A′ is a CFA-summary of A if A′ is obtained from A via
application of Rule 1 and then stepwise application of Rules 1
and 2, and no rule can be further applied.
III. L ARGE -B LOCK E NCODING
A. Summarization of Control-Flow Automata
The large-block encoding is achieved by a summarization
of the program CFA, in which each loop-free subgraph of the
CFA is replaced by a single control-flow edge with a large
formula that represents the removed subgraph. This process,
which we call CFA-summarization, consists of the fixpoint
application of the three rewriting rules that we describe below:
first we apply Rule 0 once, and then we repeatedly apply Rules
1 and 2, until no rule is applicable anymore.
Let P = (A, l0 , lE ) be a program with CFA A = (L, G).
Example. Figure 2 shows a program (a) and its corresponding
CFA (b). The control-flow automaton (CFA) is iteratively
transformed to a CFA-summary (f) as follows: Rule 1 eliminates location 6 to (c), Rule 1 eliminates location 3 and then
location 4 to (d), Rule 2 replaces the two edges 2–5 to (e),
Rule 1 eliminates location 2 and then location 5 to (f).
Rule 0 (Error Sink). We remove all edges (lE , ·, ·) from G, s.t.,
the target location lE is a sink node with no outgoing edges.
Rule 1 (Sequence). If G contains an edge (l1 , op 1 , l2 )
with l1 = l2 and no other incoming edges for l2
(i.e. edges (·, ·, l2 )), and G→
l2 is
l1
l1
the subset of G of outgoing edges
op 1
for l2 , then we change the CFA A
op 1 ; op 2
in the following way: (1) we rel2
move location l2 from L, and (2)
op 1 ; op 3
op 3
we remove the edge (l1 , op 1 , l2 ) op 2
→
and all edges in Gl2 from G, and
l3
l4
l3
l4
for each edge (l2 , op i , li ) ∈ G→
l2 ,
we add the edge (l1 , op 1 ; op i , li )
to G, where SPop 1 ; op i (ϕ) = SPop i (SPop 1 (ϕ)). (Note that
G→
l2 might contain an edge (l2 , ·, l1 ).)
In the context of this article, we use the CFA-summary for
program analysis, i.e., we want to verify if the error location
of the program is reachable. The following theorem states that
our summarization of a CFA is correct in this sense. (The proof
can be found in our extended technical report [5].)
Theorem 3.1 (Correctness of Summarization): Let P =
(A, l0 , lE ) be a program and let A′ = (L′ , G′ ) be a CFAsummary of A. Then: (i) {l0 , lE } ⊆ L′ , and (ii) lE is reachable
in (A′ , l0 , lE ) if and only if lE is reachable in P .
The summarization can be performed in polynomial time.
The time taken by Rule 0 is proportional to the number
of outgoing edges for lE . Since each application of Rule 1
or Rule 2 removes at least one edge, there can be at most
|G| − 1 such applications. A naive way to determine the set
of locations and edges to which to apply each rule requires
28
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
O(|V | · k) time, where k is the maximum out-degree of
locations. Finally, each application of Rule 2 requires O(1)
time, and each application of Rule 1 O(k) time. Therefore, a
naive summarization algorithm requires O(|G| · |V | · k) time,
which reduces to O(|G|·|V |) if k is bounded (i.e., if we rewrite
a priori all switches into nested ifs).3
involving formulas with a complex Boolean structure. With
LBE, the cost of each symbolic operation, rather than their
number, becomes a critical performance factor.
To this extent, LBE makes it possible to fully exploit the
power and functionality of modern SMT solvers: First, the
capability of modern SMT solvers to perform large amounts of
Boolean reasoning allows for handling large Boolean combinations of atomic expressions, instead of simple conjunctions.
Second, the capability of some SMT solvers (e.g., [10]) to
perform All-SMT and interpolation allows for efficient computation of Boolean abstractions and interpolants, respectively.
SMT-based Boolean abstraction and interpolation were shown
to outperform previous approaches [11], [12], [18], especially
when dealing with complex formulas. With SBE, instead, the
use of modern SMT technology does not lead to significant
improvements of the overall ART-based algorithm, because
each SMT query involves only simple conjunctions. 4
B. LBE versus SBE for Software Model Checking
The use of LBE instead of the standard SBE requires
no modification to the general model-checking algorithm,
which is still based on ART construction with CEGAR-based
refinement. The main difference is that in LBE, there is no
one-to-one correspondence between ART paths and syntactical
program paths. A single CFA edge corresponds to a set of
paths between its source and target location, and a single
ART path corresponds to a set of program paths. An ART
node represents an overapproximation of the data region that
is reachable by following any of the program paths represented
by the ART path that leads to it. This difference leads to two
observations.
First, LBE can lead to exponentially-smaller ARTs than
SBE, and thus it can drastically reduce the number of successor computations (cf. example in Sect. I) and the number
of abstraction-refinement steps for infeasible error paths. Each
of these operations, however, is typically more expensive than
with SBE, because more complex formulas are involved.
Second, LBE requires a more general representation of abstract states. When using SBE, abstract states are typically represented as sets/conjunctions of predicates. This is sufficient
for practical examples because each abstract state represents
a data region reachable by a single program path, which can
be encoded essentially as a conjunction of atomic formulas.
With LBE, such representation would be too coarse, since
each abstract state represents a data region that is reachable on
several different program paths. Therefore, we need to use a
representation for arbitrary (and larger) Boolean combinations
of predicates. This generalization of the representation of
abstract states requires a generalization of the representation
of the transfers, i.e., replacing the Cartesian abstraction with
a more precise form of abstraction. In this paper, we evaluate
the use of the Boolean abstraction, which allows for a precise
representation of arbitrary Boolean combinations of predicates.
With respect to the traditional SBE approach, LBE allows
us to trade part of the cost of the explicit enumeration of program paths with that of the symbolic computation of abstract
successor states: rather than having to build large ARTs via
SBE by performing a substantial amount of relatively cheap
operations (Cartesian abstract postoperator applications along
single-block edges and counterexample analysis of individual
program paths), we build smaller ARTs via LBE by performing more expensive symbolic operations (Boolean abstract
postoperator applications along large portions of the control
flow and counterexample analysis of multiple program paths),
IV. P ERFORMANCE E VALUATION
Implementation. In order to evaluate the proposed verification
method, we integrate our algorithm as a new component into
the configurable software verification toolkit CPACHECKER [8].
This implementation is written in JAVA. All example programs
are preprocessed and transformed into the simple intermediate
language C IL [20]. For parsing C programs, CPACHECKER
uses a library from the Eclipse C/C++ Development Kit. For
efficient querying of formulas in the quantifier-free theory
of rational linear arithmetic and equality with uninterpreted
function symbols, we use the SMT solver M ATH SAT [10],
which is integrated as a library (written in C++). We use BDDs
for representing abstract-state formulas.
Our benchmark programs, the source code, and an executable of our LBE implementation are available on
the supplementary web site on Large-Block Encoding
(http://www.sosy-lab.org/∼dbeyer/cpa-lbe). We ran all
experiments on a 1.8 GHz Intel Core2 machine with 2 GB of
RAM and 2 MB of cache, running GNU/Linux. We used a
timeout of 1 800 s and a memory limit of 1.8 GB.
Example Programs. We use three categories of benchmark
programs. First, we experiment with programs that are specifically designed to cause an exponential blowup of the ART
when using SBE (test_locks*, in the style of the example in
Sect. I). Second, we use the device-driver programs that were
previously used as benchmarks in the B LAST project. 5 Third,
we solve various verification problems for the SSH client
4 For example, B LAST uses S IMPLIFY , version 1.5.4, as of October 2001, for
computing abstract successor states. Experiments have shown that replacing
this old S IMPLIFY version by a highly-tuned modern SMT solver does
not significantly improve the performance, because B LAST does not use
much power of the SMT solver. Moreover, it was shown that although the
M ATH SAT SMT solver outperformed other tools in the computation of Craig
interpolants for general formulas, the difference in performance is negligible
on formulas generated by a standard SBE ART-based algorithm [12].
5 The B LAST distribution contains 8 Windows driver benchmarks. However, we could not run three of them (parclass.i, mouclass.i, and
serial.i), because C IL fails to parse them, making both CPACHECKER
and B LAST fail.
3 In our implementation, we use a more efficient algorithm, which we do
not describe here for lack of space.
29
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
TABLE II
P ERFORMANCE RESULTS , PROGRAMS WITH ARTIFICIAL BUGS
TABLE I
P ERFORMANCE RESULTS
B LAST
Program
(best result)
test locks 5.c
4.50
7.81
test locks 6.c
13.91
test locks 7.c
25.00
test locks 8.c
46.84
test locks 9.c
94.57
test locks 10.c
204.55
test locks 11.c
529.16
test locks 12.c
1229.27
test locks 13.c
>1800.00
test locks 14.c
>1800.00
test locks 15.c
cdaudio.i.cil.c
175.76
diskperf.i.cil.c
>1800.00
floppy.i.cil.c
218.26
kbfiltr.i.cil.c
23.55
parport.i.cil.c
738.82
s3 clnt.blast.01.i.cil.c
33.01
62.65
s3 clnt.blast.02.i.cil.c
60.62
s3 clnt.blast.03.i.cil.c
63.96
s3 clnt.blast.04.i.cil.c
811.27
s3 srvr.blast.01.i.cil.c
360.47
s3 srvr.blast.02.i.cil.c
276.19
s3 srvr.blast.03.i.cil.c
175.64
s3 srvr.blast.04.i.cil.c
304.63
s3 srvr.blast.06.i.cil.c
478.05
s3 srvr.blast.07.i.cil.c
115.76
s3 srvr.blast.08.i.cil.c
s3 srvr.blast.09.i.cil.c
445.21
115.10
s3 srvr.blast.10.i.cil.c
367.98
s3 srvr.blast.11.i.cil.c
304.05
s3 srvr.blast.12.i.cil.c
580.33
s3 srvr.blast.13.i.cil.c
s3 srvr.blast.14.i.cil.c
303.21
115.88
s3 srvr.blast.15.i.cil.c
305.11
s3 srvr.blast.16.i.cil.c
TOTAL (solved/time)
32 / 8591.12
TOTAL w/o test_locks* 23 / 6435.51
CPACHECKER
B LAST
Program
(best result)
cdaudio.BUG.i.cil.c
18.79
diskperf.BUG.i.cil.c
889.79
floppy.BUG.i.cil.c
119.60
kbfiltr.BUG.i.cil.c
46.80
parport.BUG.i.cil.c
1.67
s3 clnt.blast.01.BUG.i.cil.c
8.84
9.02
s3 clnt.blast.02.BUG.i.cil.c
s3 clnt.blast.03.BUG.i.cil.c
6.64
9.78
s3 clnt.blast.04.BUG.i.cil.c
s3 srvr.blast.01.BUG.i.cil.c
7.59
7.16
s3 srvr.blast.02.BUG.i.cil.c
7.42
s3 srvr.blast.03.BUG.i.cil.c
7.33
s3 srvr.blast.04.BUG.i.cil.c
39.81
s3 srvr.blast.06.BUG.i.cil.c
310.84
s3 srvr.blast.07.BUG.i.cil.c
40.51
s3 srvr.blast.08.BUG.i.cil.c
265.48
s3 srvr.blast.09.BUG.i.cil.c
40.24
s3 srvr.blast.10.BUG.i.cil.c
49.05
s3 srvr.blast.11.BUG.i.cil.c
38.66
s3 srvr.blast.12.BUG.i.cil.c
251.56
s3 srvr.blast.13.BUG.i.cil.c
39.94
s3 srvr.blast.14.BUG.i.cil.c
s3 srvr.blast.15.BUG.i.cil.c
40.19
39.54
s3 srvr.blast.16.BUG.i.cil.c
TOTAL (solved/time)
24 / 2296.25
SBE
LBE
4.01
0.29
7.22
0.32
12.63
0.34
23.93
0.57
52.04
0.38
131.39
0.40
MO
0.70
MO
0.46
MO
0.49
MO
0.50
MO
0.56
MO
53.55
MO
232.00
MO
56.36
41.12
7.82
MO
378.04
755.81
19.51
1075.45
16.00
746.31
49.50
730.80
25.45
>1800.00
125.33
>1800.00
122.83
>1800.00
98.47
>1800.00
71.77
>1800.00
59.70
>1800.00
85.82
>1800.00
61.29
>1800.00
126.47
>1800.00
63.36
>1800.00
162.76
>1800.00
170.33
>1800.00
74.49
>1800.00
50.38
>1800.00
21.01
>1800.00
127.82
11 / 3580.71 35 / 2265.07
5 / 3349.48 24 / 2260.07
CPACHECKER
SBE
74.39
26.53
36.49
75.45
14.62
1514.90
843.42
780.72
724.04
MO
>1800.00
>1800.00
>1800.00
MO
>1800.00
>1800.00
>1800.00
>1800.00
>1800.00
>1800.00
>1800.00
1656.54
>1800.00
>1800.00
10 / 5747.10
LBE
9.85
6.78
4.30
11.52
2.64
3.33
3.27
2.61
3.18
2.09
2.10
2.08
1.93
5.08
28.35
36.47
4.94
12.01
4.80
6.11
15.20
4.63
10.19
5.21
24 / 188.67
The configuration using -dfs -predH 7 is the winner (in terms
of solved problems and total runtime) for the programs without
defects, but is not able to verify four example programs
(timeout) [5]. For the unsafe programs, -bfs -predH 7 performs best. All four configurations use the command-line options -craig 2 -nosimplemem -alias "", which specify that
B LAST runs with lazy, Craig-interpolation-based refinement,
no C IL preprocessing for memory access, and without pointer
analysis. In all experiments with B LAST, we use the same
interpolation procedure (M ATH SAT) as in our CPACHECKERbased implementation. 6 In the performance tables, we show
the best result among the four configurations for every single
instance (column best result). (The results of all four configurations are provided in our extended technical report [5].)
Second, in order to separate the optimization efforts in
B LAST from the conceptual essence of the traditional lazyabstraction algorithm, we developed a re-implementation of
the traditional algorithms (column ’SBE’), as described in the
B LAST tool article [6]. This re-implementation is integrated as
component into CPACHECKER, so that the difference between
SBE and LBE is only in the algorithms, not in the environment
(same parser, same BDD package, same query optimization,
etc.). Our SBE implementation uses a DFS algorithm.
Third, we ran the experiments using our new LBE algorithm, which is also implemented within CPACHECKER (column LBE). Our LBE implementation uses a DFS algorithm.
Note that the purpose of our experiments is to give evidence
of the performance difference between SBE and LBE, because
these two settings are closest to each other, since SBE and LBE
and server software (s3_clnt* and s3_srvr*), which share
the same program logic, but check different safety properties.
The safety property is encoded as conditional call of a failure
location and therefore reduces to the reachability of a certain
error location. All benchmark programs from the B LAST web
page are preprocessed with C IL. For the second and third
groups of programs, we also performed experiments with
artificial defects introduced.
Experimental Configurations. For a careful and fair performance comparison, we performed experiments using three
different configurations. First, we use B LAST, version 2.5,
which is a highly optimized state-of-the-art software model
checker. B LAST is implemented in the programming language
OC AML. We ran B LAST using all four combinations of
breadth-first search (-bfs) versus depth-first search (-dfs),
both with and without heuristics for improving the predicate
discovery. B LAST provides five different levels of heuristics
for predicate discovery, and we use only the lowest (-predH
0) and the highest option (-predH 7). Interestingly, every combination is best for some particular example programs, with
considerable differences in runtime and memory consumption.
6 We tried also to use M ATH SAT instead of S IMPLIFY for computing
abstract successor states, but this did not improve the performance of B LAST.
30
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
TABLE III
D ETAILED COMPARISON BETWEEN SBE AND LBE; ENTRIES MARKED WITH (*) DENOTE PARTIAL STATISTICS FOR ANALYSES THAT TERMINATED
UNSUCCESSFULLY ( IF AVAILABLE )
Program
test locks 5.c
test locks 6.c
test locks 7.c
test locks 8.c
test locks 9.c
test locks 10.c
test locks 11.c
test locks 12.c
test locks 13.c
test locks 14.c
test locks 15.c
cdaudio.i.cil.c
diskperf.i.cil.c
floppy.i.cil.c
kbfiltr.i.cil.c
parport.i.cil.c
s3 clnt.blast.01.i.cil.c
s3 clnt.blast.02.i.cil.c
s3 clnt.blast.03.i.cil.c
s3 clnt.blast.04.i.cil.c
s3 srvr.blast.01.i.cil.c
s3 srvr.blast.02.i.cil.c
s3 srvr.blast.03.i.cil.c
s3 srvr.blast.04.i.cil.c
s3 srvr.blast.06.i.cil.c
s3 srvr.blast.07.i.cil.c
s3 srvr.blast.08.i.cil.c
s3 srvr.blast.09.i.cil.c
s3 srvr.blast.10.i.cil.c
s3 srvr.blast.11.i.cil.c
s3 srvr.blast.12.i.cil.c
s3 srvr.blast.13.i.cil.c
s3 srvr.blast.14.i.cil.c
s3 srvr.blast.15.i.cil.c
s3 srvr.blast.16.i.cil.c
ART
size
1344
2301
3845
6426
10926
19091
24779(*)
28119(*)
31739(*)
35178(*)
38777(*)
53323(*)
–
31079(*)
19640
26188(*)
122678
354132
196599
172444
232195(*)
254667(*)
–
–
295698(*)
–
279991(*)
189541(*)
307671(*)
–
258546(*)
167333(*)
318982(*)
279319(*)
346185(*)
SBE
# ref
#
steps
Tot
50
10
72
12
98
14
128
16
162
18
200
20
242(*) 22(*)
288(*) 24(*)
338(*) 26(*)
392(*) 28(*)
450(*) 30(*)
445(*) 147(*)
–
–
301(*) 79(*)
153
53
360(*) 143(*)
557
59
532
55
534
55
538
55
774(*) 70(*)
745(*) 79(*)
–
–
–
–
576(*) 63(*)
–
–
549(*) 57(*)
720(*) 72(*)
597(*) 55(*)
–
–
563(*) 57(*)
682(*) 70(*)
643(*) 65(*)
579(*) 58(*)
596(*) 59(*)
predicates
Avg
Max
3
10
4
12
5
14
6
16
7
18
8
20
9(*)
22(*)
10(*)
24(*)
10(*)
26(*)
11(*)
28(*)
12(*)
30(*)
9(*)
78(*)
–
–
7(*)
35(*)
5
27
4(*)
41(*)
20
59
19
55
19
55
19
55
20(*)
70(*)
19(*)
78(*)
–
–
–
–
14(*)
63(*)
–
–
15(*)
57(*)
16(*)
71(*)
16(*)
55(*)
–
–
15(*)
57(*)
18(*)
69(*)
13(*)
64(*)
15(*)
58(*)
12(*)
58(*)
ART
size
4
4
4
4
4
4
4
4
4
4
4
6909
4890
9668
1577
38488
36
36
39
36
101
109
91
103
94
92
89
193
91
48
99
90
92
71
98
LBE
# ref
# predicates
steps Tot
Avg
Max
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
140 79
5
16
145 56
6
21
176 58
4
13
47 18
2
6
474 168
4
17
5 47
11
47
5 51
12
51
5 75
18
75
5 47
11
47
6 88
22
88
7 75
18
75
6 85
21
85
7 82
20
82
6 84
21
84
5 85
21
85
5 88
22
88
4 72
18
72
5 79
19
79
6 69
17
69
6 94
23
94
5 81
20
81
6 83
20
83
4 71
17
71
6 86
21
86
The first group of experiments in Table I shows that the time
complexity of SBE (and B LAST) can grow exponentially in
the number of nested conditional statements, as expected. Table III explains why the SBE approach exhausts the memory:
the number of abstract nodes in the reachability tree grows
exponentially in the number of nested conditional statements.
Therefore, SBE does not scale. The LBE approach reduces
the loop-free part of the branching control-flow structure to a
few edges (cf. example in the introduction), and the size of
the ART is constant for this example program, because only
the structure inside the body of the loop changes. There are
no refinement steps necessary in the LBE approach, because
the edges to the error location are infeasible. Therefore, no
predicates are used. The runtime of the LBE approach slightly
increases with the size of the program, because the size of the
formulas that are sent to the SMT solver is slightly increasing.
Although in principle the complexity of the SMT problem
grows exponentially in the size of the formulas, the heuristics
used by SMT solvers avoid the exponential enumeration that
we observe in the case of SBE.
differ only in the CFA summarization and Boolean abstraction.
The first column is provided in Tables I and II to give evidence
that the new approach beats the highly-optimized traditional
implementation B LAST.
We actually configured and ran experiments with all four
combinations: SBE versus LBE, and Cartesian versus Boolean
abstraction. The experimentation clearly showed that SBE
does not benefit from Boolean abstraction in terms of precision, with substantial degrade in performance: the only programs for which it terminated successfully were the first five
instances of the test_locks group. Similarly, the combination
of LBE with Cartesian abstraction fails to solve any of the
experiments, due to loss of precision. Thus, we report only
on the two successful configurations, i.e., SBE in combination
with Cartesian abstraction, and LBE with Boolean abstraction.
Discussion of Evaluation Results. Tables I and II present
performance results of our experiments, for the safe and unsafe
programs respectively. All runtimes are given in seconds of
processor time, ‘>1800.00’ indicates a timeout, ‘MO’ indicates an out-of-memory. Table III shows statistics about the
algorithms for SBE and LBE only.
31
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.
Furthermore, we plan to investigate the use of adjustable
precision-based techniques for the construction of the large
blocks on-the-fly (instead of the current preprocessing step).
This would enable a dynamic adjustment of the size of the
large blocks, and thus we could fine-tune the amount of work
that is delegated to the SMT solver. Also, we plan to explore
other techniques for computing abstract successors which are
more precise than Cartesian abstraction but less expensive than
Boolean abstraction.
For the two other classes of experiments, we see that LBE
is able to successfully complete all benchmarks, and shows
significant performance gains over SBE. SBE is able to solve
only about one third of all benchmarks, and for the ones that
complete, it is clearly outperformed by LBE. In Table III, we
see that SBE has in general a much larger ART. In Table I
we observe that LBE performs significantly better than any
B LAST configuration. LBE performed best also in finding the
error paths (cf. Table II), outperforming both SBE and B LAST.
In summary, the experiments show that the LBE approach
outperforms the SBE approach, both for correct and defective
programs. This provides evidence of the benefits of a “more
symbolic” analysis as performed in the LBE approach. One
might argue that our CPACHECKER-based SBE implementation might be sub-optimal although it uses the same implementation and execution environment as LBE; in fact, both
implementations currently suffer from some inefficiencies and
have room for several optimizations. Therefore, we compare
also with B LAST. By looking at Tables I and II, we see that
LBE outperforms also B LAST, despite the fact that the latter
is the result of several years of fine-tuning. B LAST in turn is
much more efficient than SBE. However, the performance gap
between B LAST and SBE highly depends on the commandline options used for B LAST.
We conclude the section by discussing the scope of the
experimental evaluation. The LBE techniques proposed in this
paper bear substantial similarities to the SSA-based encodings
used in tools like SATABS [15], C ALYSTO [1] or S PEC # [4].
For lack of space, we chose to not include a comparison with
such tools; rather, we focussed on the more relevant issue of
the impact of LBE on ART-based model checking.
V. C ONCLUSION
AND
Acknowledgments. We thank Roman Manevich for interesting
discussions about B LAST’s performance bottlenecks.
R EFERENCES
[1] D. Babic and A. J. Hu, “C ALYSTO: Scalable and precise extended static
checking,” in Proc. ICSE. ACM, 2008, pp. 211–220.
[2] T. Ball, A. Podelski, and S. K. Rajamani, “Boolean and cartesian
abstractions for model checking C programs,” in Proc. TACAS, ser.
LNCS 2031. Springer, 2001, pp. 268–283.
[3] T. Ball and S. K. Rajamani, “The S LAM project: Debugging system
software via static analysis,” in Proc. POPL. ACM, 2002, pp. 1–3.
[4] M. Barnett and K. R. M. Leino, “Weakest-precondition of unstructured
programs,” in Proc. PASTE. ACM, 2005, pp. 82–87.
[5] D. Beyer, A. Cimatti, A. Griggio, M. E. Keremoglu, and R. Sebastiani,
“Software model checking via large-block encoding, Tech. Rep. SFUCS-2009-09/DISI-09-026/FBK-irst-2009.04.005, April 2009. Available:
http://arxiv.org/abs/0904.4709
[6] D. Beyer, T. A. Henzinger, R. Jhala, and R. Majumdar, “The software
model checker B LAST: Applications to software engineering,” Int. J.
Softw. Tools Technol. Transfer, vol. 9, no. 5-6, pp. 505–525, 2007.
[7] D. Beyer, T. A. Henzinger, and G. Théoduloz, “Configurable software
verification: Concretizing the convergence of model checking and program analysis,” Proc. CAV, LNCS 4590. Springer, 2007, pp. 504–518.
[8] D. Beyer and M. E. Keremoglu, “CPA CHECKER: A tool for configurable
software verification,” Simon Fraser University, Tech. Rep. SFU-CS2009-02, January 2009. Available: http://arxiv.org/abs/0902.0019
[9] D. Beyer, D. Zufferey, and R. Majumdar, “CSI SAT: Interpolation for
LA+EUF,” in Proc. CAV, LNCS 5123. Springer, 2008, pp. 304–308.
[10] R. Bruttomesso, A. Cimatti, A. Franzén, A. Griggio, and R. Sebastiani,
“The M ATH SAT 4 SMT solver,” in Proc. CAV, ser. LNCS 5123.
Springer, 2008, pp. 299–303.
[11] R. Cavada, A. Cimatti, A. Franzén, K. Kalyanasundaram, M. Roveri, and
R. K. Shyamasundar, “Computing predicate abstractions by integrating
BDDs and SMT solvers,” in Proc. FMCAD. IEEE, 2007, pp. 69–76.
[12] A. Cimatti, A. Griggio, and R. Sebastiani, “Efficient interpolant generation in satisfiability modulo theories,” in Proc. TACAS, ser. LNCS 4963.
Springer, 2008, pp. 397–412.
[13] E. M. Clarke, O. Grumberg, S. Jha, Y. Lu, and H. Veith,
“Counterexample-guided abstraction refinement for symbolic model
checking,” J. ACM, vol. 50, no. 5, pp. 752–794, 2003.
[14] E. M. Clarke, D. Kroening, and F. Lerda, “A tool for checking ANSI-C
programs,” in Proc. TACAS, LNCS 2988. Springer, 2004, pp. 168–176.
[15] E. M. Clarke, D. Kroening, N. Sharygina, and K. Yorav, “S ATA BS :
SAT-based predicate abstraction for ANSI-C,” in Proc. TACAS, ser.
LNCS 3440. Springer, 2005, pp. 570–574.
[16] T. A. Henzinger, R. Jhala, R. Majumdar, and K. L. McMillan, “Abstractions from proofs,” in Proc. POPL. ACM, 2004, pp. 232–244.
[17] T. A. Henzinger, R. Jhala, R. Majumdar, and G. Sutre, “Lazy abstraction,” in Proc. POPL. ACM, 2002, pp. 58–70.
[18] S. K. Lahiri, R. Nieuwenhuis, and A. Oliveras, “SMT techniques for
fast predicate abstraction,” in Proc. CAV, ser. LNCS 4144. Springer,
2006, pp. 424–437.
[19] K. L. McMillan, “Lazy abstraction with interpolants,” in Proc. CAV, ser.
LNCS 4144. Springer, 2006, pp. 123–136.
[20] G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer, “C IL: Intermediate language and tools for analysis and transformation of C programs,”
in Proc. CC, ser. LNCS 2304. Springer, 2002, pp. 213–228.
[21] R. Sebastiani, “Lazy satisfiability modulo theories,” J. Satisfiability,
Boolean Modeling and Computation, vol. 3, 2007.
F UTURE W ORK
We have proposed LBE as an alternative to the SBE modelchecking approach, based on the idea that transitions in the abstract space should represent larger fragments of the program.
Our novel approach results in significantly smaller ARTs,
where abstract successor computations are more involved, and
thus trading cost of many explicit enumerations of program
paths with the cost of symbolic successor computations. A
thorough experimental evaluation shows the advantages of
LBE against both our implementation of SBE and the stateof-the-art B LAST system.
The existing experimental results can now be summarized as
follows: (i) the combination of Cartesian predicate abstraction
with SBE is successful on many practical programs [3], [6],
but is not efficient on programs with nested conditional branching, (ii) the combination of Boolean predicate abstraction
with SBE is intractably expensive [2], (iii) the combination
of Cartesian predicate abstraction with joining paths is too
imprecise [7], and (iv) the combination of Boolean predicate
abstraction with LBE is the most promising combination for
ART-based predicate abstraction (Tables I and II).
In our future work, we plan to implement McMillan’s
interpolation-based lazy-abstraction approach [19], and experiment with SBE versus LBE versions of his algorithm.
32
Authorized licensed use limited to: Universitat Passau. Downloaded on May 31,2010 at 14:00:24 UTC from IEEE Xplore. Restrictions apply.