From M INI Z INC to Optimization Modulo Theories,
and Back
Francesco Contaldo, Patrick Trentin, Roberto Sebastiani
DISI, University of Trento, Italy
Abstract. Optimization Modulo Theories (OMT) is an extension of SMT that allows for finding models that optimize objective functions. In this paper we aim at
bridging the gap between Constraint Programming (CP) and OMT, in both directions. First, we have extended the OMT solver O PTI M ATH SAT with a F LAT Z INC
interface – which can also be used as F LAT Z INC-to-OMT encoder for other OMT
solvers. This allows OMT tools to be used in combination with MZN 2 FZN on the
large amount of CP problems coming from the M INI Z INC community. Second,
we have introduced a tool for translating SMT and OMT problems on the linear
arithmetic and bit-vector theories into M INI Z INC. This allows M INI Z INC solvers
to be used on a large amount of SMT/OMT problems.
We have discussed the main issues we had to cope with in either directions. We
have performed an extensive empirical evaluation comparing three state-of-theart OMT-based tools with many state-of-the-art CP tools on (i) CP problems coming from the M INI Z INC challenge, and (ii) OMT problems coming mostly from
formal verification. This analysis also allowed us to identify some criticalities,
in terms of efficiency and correctness, one has to cope with when addressing CP
problems with OMT tools, and vice versa.
1
Introduction
The last two decades have witnessed the rise of Satisfiability Modulo Theories (SMT)
[11] as efficient tool for dealing with several applications of industrial interest, in particular in the contexts of Formal Verification (FV). SMT is the problem of finding value
assignments satisfying some formula in first-order logic wrt. some background theory.
Optimization Modulo Theories (OMT) [40,51,34,32,54,15,37,31] is a more-recent extension of SMT searching for the optimal value assignment(s) w.r.t. some objective
function(s), by means of a combination of SMT and optimization procedures. (Since
OMT extends SMT, hereafter we often simply say “OMT” for both SMT and OMT.)
Several distinctive traits of OMT solvers –like, e.g., the efficient combination of
Boolean and arithmetical reasoning, incrementality, the availability of decision procedures for infinite-precision arithmetic and the ability to produce conflict explanations–
are a direct consequence of their tight relationship with the FV domain and its practical
needs. On the whole, it appears that OMT can be a potentially interesting and efficient
technology for dealing with Constraint Programming (CP) problems as well. At the
same time, modeling CP problems for OMT solvers requires a higher-level of expertise, because the same CP instance can have many possible alternative formulations, s.t.
the performance of SMT solvers on each encoding are hardly predictable [27,26,24].
On the other hand, the availability, efficiency and expressiveness of CP tools makes
them of potential interest as backend engines also for FV applications (e.g., [21,22,29]),
in particular with SW verification, where currently SMT is the dominating backend
technology, s.t. a large amount of SMT-encoded FV problems are available [10].
In this paper we aim at bridging the gap between CP and OMT, in both directions.
In the CP-to-OMT direction, we have extended the state-of-the-art OMT solver O P TI M ATH SAT [56] with a F LAT Z INC interface (namely “ FZN 2 OMT ”). In combination
with the standard MZN 2 FZN encoder [36], this new interface can be used to either (i)
solve CP models with O PTI M ATH SAT directly or (ii) generate OMT formulas encoded
in the SMT-LIB [23] format with optimization extensions, to be fed to other OMT
solvers, such as BCLT [16] and Z3 [15]. This allows state-of-the-art OMT technology
to be used on M INI Z INC problems coming from the CP community.
In the OMT-to-CP direction, we have introduced a tool for translating SMT and
OMT problems on the theories of linear arithmetic over the integers and rational (LIRA)
and bit-vector (BV) into M INI Z INC models (hereafter “OMT 2 MZN”). This allows M INI ZINC solvers to be used on OMT problems, giving them access to a large amount of OMT
problems, mostly coming from formal verification.
With both directions, we first present and discuss the challenges we encountered and
the solutions we adopted to address the differences between the two formalisms. Then
we present an extensive empirical evaluation comparing three OMT tools with many
state-of-the-art CP tools on (i) CP problems coming from the M INI Z INC challenge,
and (ii) OMT problems coming mostly from formal verification. This analysis allowed
us to identify some criticalities, in terms of efficiency and correctness, one has to cope
with when addressing CP problems with OMT tools, and vice versa.
Overall, our new compilers FZN 2 OMT and OMT 2 MZN in combination with the standard compiler MZN 2 FZN [36] provide a framework for translating problems encoded in
the SMT-LIB or the M INI Z INC format in either direction. This framework enables also
for a comparison between OMT solvers and CP tools on problems that do not belong
to their original application domain. To the best of our knowledge, this is the first time
that such a framework has been proposed, and that the OMT and CP technologies have
been extensively compared on problems coming from both fields.
Related Work. The tight connection between SMT and Constraint Programming (CP)
has been known for a relatively long period of time [41] and it has previously been
subject to investigation. Some works considered a direct encoding of CP [27,26] and
weighted CP [7] into SMT and M AX SMT, or an automatic framework for translating M INI Z INC –a standard CP modeling language [38]– into SMT-LIB –the standard
SMT format– [18,17]. Other works explored the integration of typical SAT and SMT
techniques within CP solvers [43,25]. Nowadays, several M INI Z INC solvers –like, e.g.,
H AIFACSP [59] and P ICAT [60]– are at least partially based on SAT technology.
To this extent, our first contribution FZN 2 OMT also obviates the loss, due to obsolescence, of the FZN 2 SMT compiler proposed by Bofill et al. in [18,17]. FZN 2 SMT is not
compatible with the changes that have been introduced to the M INI Z INC and F LAT ZINC standards starting from version 2.0 of the M INI Z INC distribution. Since some of
these changes are not backward compatible, it is also not possible to use FZN 2 SMT in
2
conjunction with an older version of the MZN 2 FZN compiler when dealing with recent
M INI Z INC models. Furthermore, FZN 2 SMT translates satisfaction problems into the
Version 1 of the SMT-LIB standard and produces no SMT-LIB output in the case of
optimization problems, that are solved directly. However, the optimization interface of
modern OMT solvers is based on the Version 2 of the SMT-LIB standard. This makes
it difficult to use it together with OMT solvers. Unfortunately, the FZN 2 SMT compiler is
closed source, with only the binaries being freely distributed, and seemingly no longer
maintained. This made it necessary to provide a new alternative solution to FZN 2 SMT.
To this extent, our new F LAT Z INC interface of O PTI M ATH SAT, FZN 2 OMT, translates
both satisfaction and optimization problems in the Version 2 of the SMT-LIB standard
enriched with the optimization extensions for OMT described in [56].
Content. The rest of the paper is organized as follows. In §2 we provide some background on OMT, M INI Z INC and F LAT Z INC. In §3 we describe the process from M INI ZINC to OMT. In §4 we describe the process from OMT to M INI Z INC . In §5 we describe
an empirical evaluation comparing a OMT-based tool with many state-of-the-art CP
tools. Finally, in §6 we conclude and point out some further research directions.
A longer and more detailed version of this paper is publicly available as [23].
2
Background
Satisfiability Modulo Theories (SMT) is the problem of deciding the satisfiability of
a first-order formula ϕ with respect to a combination of decidable first-order theories.
Typical theories of SMT interest are (the theory of) linear arithmetic over the rationals (LRA), the integers (LIA) or their combination (LIRA), non-linear arithmetic
over the rationals (N LRA) or the integers (N LIA), arrays (AR), bit-vectors (BV),
floating-point arithmetic (FP), and their combinations thereof. (See [42,50,11] for an
overview.). The last two decades have witnessed the development of very efficient SMT
solvers based on the so-called lazy-SMT schema [50,11]. This has brought previouslyintractable problems to the reach of state-of-the-art SMT solvers.
Optimization Modulo Theories (OMT), [40,51,34,32,54,15,37,58], is an extension
to SMT that allows for finding a model of a first-order formula ϕ that is optimal with
respect to some objective function expressed in some background theory, by means
of a combination of SMT and optimization procedures. State-of-the art OMT tools allow optimization in a variety of theories, including linear arithmetic over the rationals
(OMT(LRA)) [51] and the integers (OMT(LIA)) [15,54], bit-vectors (OMT(BV))
[37] and floating-point numbers (OMT (FP)) [58].
A relevant strict subcase of OMT(LRA)
is OMT with Pseudo-Boolean objective
P
functions (OMT(PB)) in the form i wi Ai s.t. wi are rational values and Ai are
Boolean variables whose values are interpreted as {0, 1}. Notice that OMT(PB) is also
equivalent to (partial weighted) M AX SMT, the SMT extension of M AX SAT, and that
OMT(PB) and M AX SMT can be encoded into OMT(LRA) but not vice versa [52].
Encoding OMT(PB)/M AX SMT into OMT(LRA), however, is not the most efficient
way to solve them, so that modern OMT solvers such as BCLT [16], O PTI M ATH SAT
[56] and Z3 [15] implement specialized OMT(PB)/M AX SMT procedures which are
much more efficient than general-purpose OMT(LRA) ones [15,55,56].
3
We stress the fact that —unlike with purely-combinatorial problems, which are encoded into SAT or M AX SAT and are thus solved by purely-Boolean search– typically
OMT problems involve the interleaving of both Boolean and arithmetical search: search
not only for the best truth-value assignment to the atomic subformulae, but also for the
best values to the numerical variables compatible with such truth-value assignment [52].
To this date, few OMT solvers exist, namely BCLT [16], C EGIO [8], H AZEL [37],
O PTI M ATH SAT [56], P ULI [31], S YMBA [34] and Z3 [15]. To this aim, we observe that
(i) some of these solvers are quite recent, (ii) most of these solvers focus on different,
partially overlapping, niche subsets of Optimization Modulo Theories, and (iii) the lack
of an official Input/Output interface for OMT makes it hard to compare some of these
tools with one another. OMT finds applications in the context of static analysis [19,30],
formal verification and model checking [35,46], scheduling and planning with resources
[31,33,48,44], software security and requirements engineering [39], workflow analysis
[12], machine learning [57], and quantum computing [13].
A distinctive trait of SMT (and OMT) solvers is the trade-off of speed against the
ability to certify the correctness of the result of any computation, which is particularly
important in the contexts of Formal Verification (FV) and Model Checking (MC). When
dealing with linear arithmetic in particular, SMT solvers employ infinite-precision arithmetic software libraries to avoid numerical errors and overflows.
SMT-LIB [23] is the standard input format by SMT solvers, it provides a standardized definition of the most prominent theories supported by SMT solvers and the
corresponding language primitives to use these features. At present, there is no standard
input format for modeling optimization problems targeting OMT solvers, although there
exist only minor syntactical differences between the major OMT solvers. The tools presented in this paper conform to the extended SMT-LIB format for OMT presented in
[56], that includes language primitives for modeling objectives.
M INI Z INC [38,36] is a widely adopted high-level declarative language for modeling
Constraint Satisfaction Problems (CSP) and Constraint Optimization Problems (COP).
The M INI Z INC format defines three scalar types (bool, int and float) and two compound
types (sets and fixed-size arrays of some scalar type). The standard provides an extensive list of predefined global constraints, a class of high-level language primitives that
allows one to encode complex constraints in a compact way.
F LAT Z INC is a lower-level language whose purpose is to bridge the gap between the
high-level modeling in M INI Z INC and the need for a fixed, and easy-to-parse, format
that can simplify the implementation of the input interface of M INI Z INC solvers. A
M INI Z INC model is typically flattened into a F LAT Z INC instance using the MZN 2 FZN
compiler [36], and then solved with some M INI Z INC tool.
3
From M INI Z INC to OMT
We consider the problem of translating M INI Z INC models into OMT problems first.
Similarly to other M INI Z INC solvers, we assume that the M INI Z INC model is first
translated into F LAT Z INC using the MZN 2 FZN standard compiler, as depicted in Figure 1. We describe the main aspects of FZN 2 OMT, focusing on the challenges we have
encountered and on the solutions we have adopted.
4
Fig. 1. Circular translation schema from M INI Z INC to SMT-LIB and back, resulting
from the composition of MZN 2 FZN, O PTI M ATH SAT and OMT 2 MZN. In this picture,
O PTI M ATH SAT acts both as a F LAT Z INC/OMT solver, and also as a F LAT Z INC to
SMT-LIB compiler.
F LAT Z INC data-types. The first challenge is to find a suitable representation of the
data-types supported by F LAT Z INC in SMT-LIB.
One possible choice for modeling the three basic scalar types of F LAT Z INC –namely
bool, int and float– with SMT-LIB are the Boolean, bit-vector and floating-point
theories respectively. However, the decision procedures for the bit-vector and floatingpoint numbers theories can be significantly more resource demanding than the decision
procedure for the linear arithmetic theory (LIRA), in particular when dealing with
a substantial amount of arithmetic computations. For this reason, we have opted to
model F LAT Z INC int and float data-types with the SMT-LIB integer and rational
types respectively, by default. For the case in which no substantial linear arithmetic
computation is performed, we also optionally allow for encoding the F LAT Z INC int
data-type as a SMT-LIB bit-vector.
For what concerns the two compound types of F LAT Z INC, that is the set and
array data-types, we have chosen to proceed as follows. Given that O PTI M ATH SAT
lacks a decision procedure for the theory of finite sets [6], we model a set using the
Boolean and integer theories, similarly to what has been done in [17]. The basic idea
is to introduce a fresh Boolean variable for each element in the domain of a set, and
use such variable as a placeholder for the membership of an integer element to the set
instance. Differently from [17], we make an extensive use of cardinality networks [9]
to encode constraints over the sets because they are handled more efficiently, for their
nice arc-consistency properties. No action is required to encode a F LAT Z INC array
into SMT-LIB, because it is used only as a container for other variables.
Floating-Point precision. A consequence of encoding the F LAT Z INC int and float
data-types with the linear arithmetic theory is that all of our computation is performed
with infinite-precision arithmetic. This can result in a performance disadvantage wrt.
other M INI Z INC solvers using finite-precision arithmetic, due to the increased cost of
each operation, but it has the benefit of guaranteeing the correctness of the final result
of the computation.
Currently, the M INI Z INC language does not allow one to express a certain quantity
as an infinite-precision fraction between two constant numbers. Instead, the MZN 2 FZN
compiler computes on-the-fly the result of any division operation between two constant
integers or floating-point numbers applying the rules corresponding to the type of the
operands. However, there are some instances in which we really need to be able to both
express quantities and perform computation with infinite-precision arithmetic. One of
5
such situations is to double-check the correctness of the M INI Z INC models generated
by the OMT 2 MZN compiler described in Section §4 (we have done this for the experimental evaluations in Section §5.2). In order to get around this limitation we developed
a simple wrapper around the MZN 2 FZN compiler, called EMZN 2 FZN [4], that replaces
any fraction among two constant floats with a fresh variable, and then sets its value
using the F LAT Z INC constraint float_div().
F LAT Z INC constraints. The SMT-LIB encoding of the majority of F LAT Z INC constraints in O PTI M ATH SAT follows their definition in the F LAT Z INC Standard, with the
exception of Pseudo-Boolean constraints, which we examine in detail later on. Several
global constraints are also supported in the same way, because the OMT-solver currently lacks ad hoc and efficient decision procedures for dealing with them. Constant
values and alias variables (e.g. those arising from the definition of some arrays) are
propagated through the formula, so as to keep the set of problem variables as compact
as possible. Those constraints requiring non-linear arithmetic –like, e.g., trigonometric,
logarithmic and exponential functions– are currently not supported; this situation may
change soon due to the recent extension of M ATH SAT5 with a procedure for it [20].
Pseudo-Boolean
constraints. When dealing with Pseudo-Boolean sums of the form
Pi=N
i=1 Ai · wi , where Ai is a Boolean variable and wi is a numerical weight, the
MZN 2 FZN compiler associates a fresh 0/1 variable ai to each Ai , and encodes the sum
Pi=N
as i=1 ai ·wi . Notice that the original Ai s may not be eliminated from the F LAT Z INC
model, because they typically occur elsewhere in the problem, i.e. as part of a Boolean
formula. From our own experience, this situation arises frequently, because PseudoBoolean sums are typically used to express cardinality constraints that have a variety of
uses. As described in [55], one limitation of this naive approach is that SMT and OMT
solvers do not typically handle this encoding efficiently. The main reason is that the
pruning power of the conflict clause resulting from a conflicting assignment is typically
limited to one specific Boolean assignment at a time, meaning that a large number of
conflict clauses (possibly exponential) has to be generated along the search. As shown
in [55], SMT and OMT solvers can benefit from encoding Pseudo-Boolean constraints
with cardinality networks.
FZN 2 OMT goes through some effort in order to recognize Pseudo-Boolean sums
over the integers, and replace the naive encoding with one based on cardinality networks. We note that using this technique generally results in a trade-off between solving
time and the overhead of generating cardinality networks prior to starting the search,
especially when dealing with a large number of variables.
Multi-objective optimization. FZN 2 OMT allows for multiple optimization goals, of heterogeneous type, being defined within the same F LAT Z INC model. This is a non-standard
extension to the F LAT Z INC format. Multiple objectives can be solved independently
from one another, or combined into a Lexicographic or Pareto optimization goal. We
refer the reader to [56] for details on the input encoding and the solver configuration.
Functionality. Given a satisfiability or optimization problem encoded in the F LAT Z INC
format, O PTI M ATH SAT can be used in the following ways (Figure 1):
6
– to directly solve the problem, optionally enumerating any sub-optimal solution
found during the search or all possible solutions with the same optimal value;
– to produce an OMT problem encoded with the extended SMT-LIB format described in [56]. This problem can be directly solved with O PTI M ATH SAT or, with
minor transformations1 , fed as input to other OMT solvers such as BCLT and Z3.
4
From OMT to M INI Z INC
In this section, we consider the problem of translating OMT formulas, encoded in the
optimization-extended SMT-LIB format of [56], into M INI Z INC models. Hereafter, we
describe the main challenges we have faced and the solutions we have adopted. Further
details about this conversion are available in [3].
General Translation Approach. The main challenge is to design an encoding from OMT
to M INI Z INC that is correct (i.e., it preserves in full the semantics of the input OMT
problems), effective (i.e., it produces as output M INI Z INC models which are as compact
and easy-to-solve as possible), and efficient (i.e. it does it with the least consumption
of time and memory). To this extent, one critical design choice is the way in which the
internal representation of the input OMT formula is organized and converted in terms of
M INI Z INC primitives. After a preliminar experimental evaluation we determined that
the sweet-spot, in terms of compactness and easiness to solve of the resulting M INI Z INC
model, is to adopt what we call “≥2-father DAG-ification”: a Directed-Acyclic-Graph
(DAG) internal representation of the formula where a fresh label is associated to all and
only DAG nodes with at least two fathers, inlining all other nodes
Theories Restriction. The SMT-LIB standard describes a wide number of SMT theories, most of which have no direct counterpart in M INI Z INC due to the few data-types
supported (see §2). Hence, hereafter we deal only with the theory of linear rational
and integer arithmetic, and their combination. On this regard, we note that even though
OMT 2 MZN can also handle the theory of bit-vectors, we do not cover it here because it
is not used in the experimental evaluation in Section §5 (We cover it in the long version
of this paper [23]). We leave the handling of other SMT theories to future work.
Linear Arithmetic Theory. On the surface, encoding linear arithmetic constraints coming from OMT in M INI Z INC, using the int and float data-types, looks like a trivial task.
In reality, this poses several challenges and it is subject to several limitations, due to a
couple of facts.
First, in SMT-LIB the linear arithmetic theory requires the capability to perform
infinite-precision computations. Unfortunately, to the best of our knowledge, no M INI ZINC solver provides infinite-precision arithmetic reasoning, and the MZN 2 FZN compiler
itself prevents representing arbitrarily-large and arbitrarily-precise quantities (e.g. the
fine-grained decimal weights of the machine learning application in [57]).
Second, in OMT linear arithmetic variables are not required to be bounded and have
quite often no explicit domain (i.e. they lack a lower-bound, an upper-bound or both),
1
To make this step as easy as possible, we collected our scripts into a public repository [1].
7
because it is not necessary for the problem at hand or it is implied by other constraints.
This is in contrast with M INI Z INC, whereby linear arithmetic variables are expected to
have a finite domain and, when they lack one, their domain appears to be capped with
some solver-dependent pair of values.
These restrictions are currently part of the M INI Z INC language and the target application domain, and we do not see any obvious work-around solving them. We note
that although there exist methods for bounding all variables in a given LP problem
(e.g. [45]), these have been deemed too impractical at this stage of our investigation.
Nonetheless, we have chosen to translate SMT-LIB linear arithmetic constraints with a
corresponding M INI Z INC encoding based on the int and float data-types. Although the
encoding is not always applicable, it does still allow one to correctly translate a number
of interesting OMT problems into M INI Z INC, as witnessed by our experimental evaluation in Section §5.2. More in detail, the translation is done as follows. We declare each
integer variable as unbounded, and then extend the M INI Z INC model with the appropriate constraints bounding its domain when the input OMT formula contains any such
information. Our empirical observation is that M INI Z INC models generated in this way
are correctly handed by all M INI Z INC solvers which we have tried, with the exception
of G UROBI, which returns an “unsupported” message. Floating-Point variables, instead,
are always declared with a user-defined domain. This is because all of the M INI Z INC
solvers we have tried, among those that can handle floating-point constraints, require
such information.
Other OMT Functionalities. Several problems of OMT interest require the capability
of dealing with soft-constraints (i.e. Weighted M AX SMT) and also with multiple objectives, that are either considered independent goals or combined in a Lexicographic or
Pareto-like fashion. To the best of our knowledge, the M INI Z INC standard does not allow for an explicit encoding of soft-constraints, nor to deal with more than one objective
function at a time.
We encode (weighted) M AX SMT problems using a standard Pseudo-Boolean encoding, such as the one used in [51]. When dealing with OMT problems that contain
N goals obj1 , ..., objN , for N > 1, we use the following approach. If these objectives
are independent targets, we generate N M INI Z INC models, each with a different goal
obji , and separately solve each model. If instead the multiple objectives belong to a
Lexicographic OMT problem, then we generate a unique M INI Z INC model that leverages the lexicographic-optimization functionality provided by M INI S EARCH [47]. (In
all other cases, M INI S EARCH is not used). We do not have any encoding for dealing
with Pareto-optimization, yet.
5
Experimental Evaluations
In this section we present an extensive empirical evaluation comparing OMT tools with
many state-of-the-art CP tools on CP problems coming from the M INI Z INC challenge
(§5.1), and on OMT problems coming mostly from formal verification (§5.2).
The OMT solvers under evaluation are BCLT, O PTI M ATH SAT (v. 1.6.0) and Z3
(v. 4.8.5). These are compared with some of the top-scoring solvers that participated
8
at recent editions of the M INI Z INC challenge, including C HOCO (v. 4.0.4), C HUFFED,
G12( FD ) (v. 1.6.0), G ECODE (v. 6.0.1), G UROBI (v. 8.0.1), H AIFACSP (v. 1.3.0), JA C O P (v. 4.5.0), I Z PLUS (v. 3.5.0), OR-T OOLS (v. 6.7.4981) and P ICAT (v. 2.4).
Remark 1. We could not include FZN 2 SMT [18,17] in our experimental evaluation because it is not compatible with the features of M INI Z INC that have been added since
version 2.0.
We run all these experimental evaluations on two identical 8-core 2.20Ghz Xeon
machines with 64GB of RAM and running Ubuntu Linux. All the benchmark-sets, the
tools and the scripts used to run these experiments, and some of the plots for the results
in Tables 1-3 which could not fit into this paper, can be downloaded from [2].
We stress the fact that the goal of these experiments is not to establish a winner
among OMT and M INI Z INC tools; rather, it is to assess the correctness, effectiveness
and efficiency of our OMT-to-CP and CP-to-OMT encoders and, more generally, to
investigate the feasibility of solving M INI Z INC problems with OMT tools and vice
versa, and to identify the criticalities in terms of efficiency and correctness in these
processes.
5.1
Evaluation on M INI Z INC Benchmark Sets
We consider the benchmark-sets used at the M INI Z INC Challenge of 2016 (MC16) and
2019 (MC19), each comprised by 100 instances. For compatibility reasons, the version
of MZN 2 FZN used to convert the problems to the F LAT Z INC format differs between the
two benchmark-sets. We use version 2.2.1 and 2.3.2 (with patches) for the problems
in MC16 and MC19 respectively. Due to recent changes in the F LAT Z INC format that
affect the benchmarks in MC19, the version of some M INI Z INC tools differs from what
described in Section §5 (see Table 1). In some cases, we had to download and compile
the latest source available for the tool, i.e. the “nightly” version.
We run each M INI Z INC solver with the corresponding directory of global constraints, and we run each M INI Z INC and OMT tool with the default options. We consider two OMT encodings of the original F LAT Z INC problems, LA and BV. The first
encodes the F LAT Z INC int type with the theory of linear integer arithmetic, whereas
the second is based on the theory of bit-vectors. We evaluate each OMT solver on both
SMT-LIB encodings, except for BCLT that has no support for bit-vector optimization.
For uniformity reasons with the other OMT solvers, we evaluate O PTI M ATH SAT using its SMT-LIB interface only, using thus its FZN 2 OMT interface as an external tool,
like with the other OMT solvers. We note that the solving time for all OMT solvers
includes the time required for translating the formula from the F LAT Z INC to the SMTLIB format. Each solver, either OMT or M INI Z INC, is given up to 1200s. to solve each
problem, not including the time taken by MZN 2 FZN to flatten it.
We verify the correctness of the results by automatically checking that all terminating solvers agree on the (possibly optimal) solution and, when this is not the case, we
manually investigate the inconsistency.
9
tool, configuration & encoding
terminated
inst. timeout error unsup. incor. correct time (s.)
s | o s| o s | o s| o s| o s | o
s|
o
BT1
s| o
BT2
s | o
P ICAT ( CP )
G12( FD )
C HOCO ()
I Z PLUS ()
C HUFFED ()
JAC O P()
G UROBI ()
H AIFACSP()
P ICAT ( SAT )
OR-T OOLS ( SAT )
VIRTUAL BEST (M INI Z INC )
15
15
15
15
15
15
15
15
15
15
15
|
|
|
|
|
|
|
|
|
|
|
M INI Z INC Challenge 2016
85 9 | 70 0 | 0 0 | 0 0
85 4 | 71 1 | 3 0 | 0 0
85 3 | 50 0 | 0 0 | 0 0
85 6 | 44 0 | 0 0 | 0 0
85 2 | 40 0 | 0 5 | 0 0
85 3 | 39 0 | 0 0 | 0 0
85 6 | 22 0 | 0 0 | 0 0
85 4 | 23 0 | 0 0 | 0 0
85 1 | 26 0 | 0 0 | 0 0
85 1 | 15 0 | 0 0 | 0 0
85 0 | 7 0 | 0 0 | 0 0
|
|
|
|
|
|
|
|
|
|
|
0
0
0
0
0
0
0
0
0
0
0
6
10
12
9
8
12
9
11
14
14
15
|
|
|
|
|
|
|
|
|
|
|
15
11
35
41
45
46
63
62
59
70
78
2281
4436
4256
999
635
3411
2346
591
151
555
146
|
|
|
|
|
|
|
|
|
|
|
6043
4220
11423
5492
4187
12825
3037
4444
7293
1338
3514
0
0
1
3
0
0
0
0
10
1
-
|
|
|
|
|
|
|
|
|
|
|
0
0
0
4
5
0
15
11
1
45
-
0
0
1
3
0
0
0
0
10
1
-
|
|
|
|
|
|
|
|
|
|
|
0
0
0
4
5
0
15
11
1
45
-
O PTI M ATH SAT( INT )
O PTI M ATH SAT( BV )
BCLT ( INT )
Z3( INT )
Z3( BV )
VIRTUAL BEST (OMT)
15
15
15
15
15
15
|
|
|
|
|
|
85
85
85
85
85
85
|
|
|
|
|
|
0
0
0
0
0
0
5
13
5
5
10
14
|
|
|
|
|
|
47
43
52
53
57
64
604
3664
1117
676
2938
3842
|
|
|
|
|
|
4856
8561
5998
10424
11113
6432
1
11
0
0
2
-
|
|
|
|
|
|
20
2
15
11
19
-
0
0
0
0
0
-
|
|
|
|
|
|
0
0
2
0
0
-
VIRTUAL BEST ( ALL )
15 | 85 0 |
-|
-
-|
-
10
2
10
10
5
1
|
|
|
|
|
|
38
42
33
32
28
21
0
0
0
0
0
0
|
|
|
|
|
|
0
0
0
0
0
0
0
0
0
0
0
0
|
|
|
|
|
|
7 0 | 0 0 |
0
0
0
0
0
0
0
0
0
0
0
0
0 0 | 0 15 | 78 146 | 3514
P ICAT ( CP ) [2.7 B 12]
I Z PLUS ()
G12( FD )
C HOCO ( STD )
G ECODE () [6.2.0]
JAC O P() [4.8]
H AIFACSP()
C HUFFED () [ NIGHTLY ]
G UROBI () [8.1.1]
P ICAT ( SAT ) [2.7 B 12]
OR-T OOLS ( SAT ) [ NIGHTLY ]
VIRTUAL BEST (M INI Z INC )
10
10
10
10
10
10
10
10
10
10
10
10
|
|
|
|
|
|
|
|
|
|
|
|
M INI Z INC Challenge 2019
90 8 | 67 0 | 11 0 | 5 0
90 5 | 71 0 | 4 0 | 0 0
90 5 | 64 0 | 10 0 | 0 0
90 4 | 63 0 | 5 0 | 0 0
90 4 | 63 0 | 0 0 | 0 0
90 4 | 55 0 | 6 0 | 0 0
90 0 | 47 0 | 10 0 | 0 2
90 0 | 43 0 | 0 5 | 10 0
80 0 | 48 0 | 0 0 | 0 0
90 0 | 45 0 | 5 0 | 0 0
90 5 | 42 0 | 3 0 | 0 0
90 0 | 29 0 | 0 0 | 0 0
O PTI M ATH SAT( INT ) [1.6.4.1]
O PTI M ATH SAT( BV ) [1.6.4.1]
BCLT ( INT )
Z3( INT )
Z3( BV )
VIRTUAL BEST (OMT)
10
10
10
10
10
10
|
|
|
|
|
|
90
90
90
90
90
90
VIRTUAL BEST ( ALL )
10 | 90 0 | 29 0 | 0 0 |
5
4
5
5
0
0
|
|
|
|
|
|
62
59
60
64
55
48
0
0
0
0
0
0
|
|
|
|
|
|
0
5
0
0
2
2
0
0
0
0
0
0
|
|
|
|
|
|
0
0
0
0
0
0
0
0
0
0
0
0
|
|
|
|
|
|
|
|
|
|
|
|
0
0
0
0
0
0
2
0
0
1
0
0
2
5
5
6
6
6
8
5
10
10
5
10
|
|
|
|
|
|
|
|
|
|
|
|
7
15
16
22
27
29
31
37
32
39
45
61
|
|
|
|
|
|
|
|
|
|
|
|
0
3
0
0
6
1
7
19
6
7
13
-
1
1
0
0
0
0
4
3
2
0
0
-
|
|
|
|
|
|
|
|
|
|
|
|
0
3
0
0
6
1
4
19
4
5
11
-
|
|
|
|
|
|
0
0
0
0
0
0
5
6
5
5
10
10
|
|
|
|
|
|
28
4 | 3650 2 |
26 484 | 7271 0 |
30
6 | 3369 0 |
26
4 | 5358 3 |
33 1629 | 7550 5 |
40 1624 | 5179 - |
10
1
6
6
17
-
0
0
0
0
0
-
|
|
|
|
|
|
0
0
5
1
3
-
-|
-
0 0 | 0 10 | 61
54
14
323
415
420
260
2
1
705
275
8
9
|
|
|
|
|
|
|
|
|
|
|
|
1440
3077
4010
4312
5094
5467
6408
4886
2895
9894
7239
5247
9 | 4919
1
1
0
0
0
0
4
3
2
0
0
-
-|
-
Table 1. M INI Z INC Challenge formulas. The columns list the total number of instances
(inst.), of timeouts (timeout), of run-time errors (error), of unsupported problems (unsup.), of incorrectly solved instances (incor.), of correctly solved instances (correct),
the total solving time for all solved instances (time), the number of instances solved in
the shortest time within the same category (BT1) and those solved in the shortest time
considering all tools (BT2).
Experiment Results. The results of this experiment are shown in Table 1, with separate
numbers for satisfiability (s) and optimization (o) instances in each benchmark-set. Using the experimental data, we separately computed the virtual best configuration among
all M INI Z INC solvers (i.e. VIRTUAL BEST (M INI Z INC )), all OMT solvers (i.e. VIRTUAL
BEST (OMT)), and also the virtual best among all tools considered in the experiment
(i.e. VIRTUAL BEST ( ALL )). The last two columns in the table list the number of prob10
lems solved by the given configuration in the same amount of time as the VIRTUAL
BEST () of each group (col. BT1) and as the VIRTUAL BEST ( ALL ) (col. BT2).
We start by looking at the M INI Z INC solvers in Table 1. The performance ladder is dominated by OR-T OOLS ( SAT ) and P ICAT ( SAT ), closely followed by G UROBI,
H AIFACSP and C HUFFED (in MC19). By looking at column BT1, we observe that
the top-performing M INI Z INC solvers tend to dominate over all the others. Looking
at the results of the MC19 experiment, we notice a significant increase in the number
of errors with respect to the benchmark-set of the MC16 edition, as well as a handful
of problems solved incorrectly. In the case of G UROBI and P ICAT ( SAT ), the MZN 2 FZN
compiler encountered an error over a few instances. As a consequence, the total number
of problems is smaller than 100 for both tools. After taking a closer look, we ascribe
this phenomenon to the recent changes in the M INI Z INC/F LAT Z INC format, that has
created some minor issues with some tools that have not been adequately updated.
Looking at the OMT tools only, we observe that Z3 has leading performance over
the other solvers. When compared to the M INI Z INC solvers, the OMT solvers place
themselves in the middle of the rank on both benchmark-sets. Given the fact that none
of the OMT solvers has specialized procedures or encodings for dealing with global
constraints, we consider this an interesting result.
5.2
Evaluation on OMT Benchmark Sets
In this experimental evaluation we use OMT formulas taken from well-known, publicly
available, repositories. We characterize these benchmark-sets as follows:
– SAL [integers]: 66 SMT-based Bounded Model Checking and K-Induction parametric problems created with the SAL model checker [5];
– SAL [rationals]: as above, with problems on the rationals;
– Symba [rationals]: 2632 bounded2 software verification instances derived from a
set of C programs used in the Software Verification Competition of 2013 [34];
– Jobshop and Strip Packing [rationals]: 190 problems taken from [49,51];
– Machine Learning [rationals]: 510 OMT instances generated with the PY LMT tool
based on Machine Learning Modulo Theories [57].
The first benchmark-set is on the integers, whereas the other four are on the rationals.
We stress the fact that all formulas contained in all benchmark-sets are satisfiable.
Remark 2. Although there exists a repository of multi-objective OMT formulas (e.g.
[34,54]), we have chosen to not include these in our experimental evaluation. The reason
for this is twofold. First, such comparison would likely be unfair wrt. CP tools because
that the workaround for dealing with multi-independent OMT formulas described in
Section §4 is not competitive with the integrated optimization schema provided by OMT
solvers [34,54]. In fact, the experimental evidence in [34,54] collected on a group of
OMT solvers indicates that the latter approach can be an order of magnitude faster than
the former one. Second, the workaround for dealing with lexicographic-optimization is
limited by the fact that M INI S EARCH is not fully compatible with recent versions of
M INI Z INC, and it only works with a restricted set of tools.
2
We discarded any unbounded instance in the original benchmark-set in [34].
11
We have used the OMT 2 MZN tool described in Section §4 to translate each OMT
formula to the M INI Z INC format. OMT 2 MZN is written in Python and it is built on top
of PY SMT [28], a general-purpose Python library for solving SMT problems, and it is
available at [3]. During this step, it has been necessary to impose a finite domain to any
unconstrained SMT-LIB rational variable, because otherwise none of the M INI Z INC
solvers would have been able to deal with them. We have experimented with two different domains: the largest feasible domain for floating-point variables of 32 bits (i.e.
±3.402823e + 38) for the first two benchmark-sets, and the largest feasible domain for
integer variables (i.e. ±231 ) for the last two.
We consider two O PTI M ATH SAT configurations: O PTI M ATH SAT( SMT ), solving
the original OMT formulas, and O PTI M ATH SAT( FZN ), executed on the generated
M INI Z INC instances. The benefits of this choice is two-fold. First, we can double-check
the correctness of such encoding, by comparing the optimum models generated in the
two cases. Second, we can verify whether there is any performance loss caused by the
encoding of the formula.
Only four of the M INI Z INC solvers listed in Section §5 support floating-point reasoning. This limited the number of tools that could be used with some OMT benchmarksets. The running-time of each M INI Z INC solver reported in these experiments (including O PTI M ATH SAT( FZN )) is comprehensive of the time taken by the MZN 2 FZN compiler, because the latter can sometime solve the input formulas on its own. The overall
timeout is set to 600s.
Notice that the optimal solutions found by O PTI M ATH SAT( SMT ) have been previously independently verified with a third-party SMT tool as reported in previous publications [53,54,55].3 Therefore, we verify the correctness of the results found by any
other configuration by comparing them with those found by O PTI M ATH SAT( SMT ),
and otherwise mark the result as “unverified”.
Experimental Results over the Integers. In this experiment, we evaluate the SAL (over
integers) benchmark-set. The results are collected in Table 2.
We notice first that O PTI M ATH SAT( FZN ) always produces correct results and it
shows comparable performances in terms on number of problems solved wrt. the baseline O PTI M ATH SAT( SMT ), solving even 4 problems more. (We conjecture that the
latter fact should be attributed to the limited, but effective, deduction capabilities of the
MZN 2 FZN compiler, that may have helped O PTI M ATH SAT in solving the input formulas.) This suggests that, at least on problems on the integers, OMT 2 MZN is efficient and
effective and does not affect correctness.
In general, M INI Z INC solvers do not seem to deal efficiently with this benchmarkset. Some tools have experienced some internal error (e.g. dumped-core, segmentation
fault), some others have been killed to to a high memory consumption (over 32GB),
whereas the majority of the remaining tools had a timeout.
We explain this behavior with the fact that the given benchmark set is characterized by the presence of a heavy Boolean structure combined with arithmetical con3
For every OMT problem hϕ, obji s.t. O PTI M ATH SAT( SMT ) returns a minimum value min
for obj on the formula ϕ, we say min is correct iff ϕ ∧ (obj = min) is satisfiable and ϕ ∧ (obj <
min) is unsatisfiable. (Dual for maximization.)
12
Terminated
inst. timeout tool-er. unsupp. incor. correct time(s)
66
0
0
66
0
0
0
66
0
66
0
0
0
0
66
0
66
0
0
0
0
66
0
66
0
0
0
0
66
19
47
0
0
0
0
66
57
9
0
0
0
0
66
66
0
0
0
0
0
66
66
0
0
0
0
0
66
66
0
0
0
0
0
66
66
0
0
0
0
0
66
63
0
0
0
3
166
66
62
0
0
0
4 1667
66
62
0
0
0
4
718
Tools & Configuration
G UROBI()
G12( FD )
I Z PLUS()
JAC O P()
C HUFFED()
OR-T OOLS ( SAT )
C HOCO()
H AIFACSP()
P ICAT ( CP )
G ECODE()
G UROBI ( L )
P ICAT ( SAT )
V IRTUAL B EST (M INI Z INC )
O PTI M ATH SAT( FZN )
V IRTUAL B EST ( FZN )
66
66
18
18
0
0
0
0
0
0
48
48
7113
7113
O PTI M ATH SAT( SMT )
V IRTUAL B EST ( ALL )
66
66
22
16
0
0
0
0
0
0
44
50
2657
5037
Table 2. SAL over integers. A SAT result is marked as correct when the objective value
matches the reference solution provided by O PTI M ATH SAT( SMT ) (when run without
a timeout), as incorrect otherwise.
23 317 2765 90 90 86 39
8 3961 2684 44 32 32 1
0 10800
0 0 0 0 0
23 13800
- - - - -
0
0
0
-
Time(s)
∆ ≥ 101
∆ ≥ 10−1
∆ ≥ 10−3
unsat
∆ ≥ 10−6
Verified
Unverified
Incorrect
SAL, Symba, Jobshop and Strippacking
2888 10 0 2855
0
2888 48 0 2728 104
2888 2733 0
0 155
2888
0 0 2628 237
Incorrect Results
∆ ≥ 100
G12( MIP )
G UROBI()
G ECODE()
V IRTUAL B EST (M INI Z INC )
Timeout
Tool-errors
Tools & Configuration
Instances
Terminated
O PTI M ATH SAT( FZN )
V IRTUAL B EST ( FZN )
2888
2888
31 0
0 0
0 2854 3 22320
11 2854 23 20673
0
-
0
-
0
-
0
-
0
-
0
-
O PTI M ATH SAT( SMT )
V IRTUAL B EST ( ALL )
2888
2888
23 0
0 0
0 2865 0 15675
0 2865 23 15182
0
-
-
-
-
-
-
G12( MIP )
G UROBI()
G ECODE()
V IRTUAL B EST (M INI Z INC )
Machine Learning
510 108 0
510
9 0
510 322 0
510
9 0
400
476
164
468
0
1
0
1
224
135
10
317
400 0
468 8
147 17
- -
0
8
2
-
0
6
0
-
0
4
0
-
0
0
0
-
35 0
68 0
72 1
32
80
359
177
0
-
0
0
-
0
0
-
0
0
-
0
0
-
0
0
-
0 500 0 7707
0 500 1 7737
0
-
-
-
-
-
-
O PTI M ATH SAT( FZN )
O PTI M ATH SAT( FZN + E )
V IRTUAL B EST ( FZN )
510 298 0 177
510 442 0
0
510
9 0 428
O PTI M ATH SAT( SMT )
V IRTUAL B EST ( ALL )
510
510
10 0
9 0
2
24
24
32
Table 3. OMT Problems defined over the rationals. A SAT result is marked as correct when the objective value matches the reference solution provided by O PTI M ATH SAT( SMT ) with an absolute error ∆ < 10−6 . A result is marked as unverified when we
have no reference solution and incorrect if neither of the previous two conditions apply.
straints, which requires the efficient combination of strong Boolean-reasoning capabilities (e.g., efficiently handling chains of unit propagations) with strong arithmeticalsolving&optimization capabilities, which is a typical feature of OMT solvers.
13
None of the input formulas was initially supported by G UROBI. After restricting
the bound of every integer variable to ±106 , G UROBI ( L ) was able to solve 3 instances
within the timeout. Among the M INI Z INC solvers, the best result is obtained by P I CAT ( SAT ), that solved 4 problems out of 66.
Experimental Results Over the Rationals. We consider first the first three benchmarksets over the rationals: SAL over rationals, Symba, JobShop&Strip-Packing. (Separate
tables for the four benchmarks are reported in the extended version of this paper [23].)
Of all M INI Z INC solvers we have tried, only three are able to deal with floating-point
constraints. The results are shown in Table 3. Since each of the input formulas is satisfiable, we consider a result incorrect either when it is equal to UNSAT, or when the relative
def |o
−of zn |
, osmt and of zn being the optimal value
error ∆ exceeds 10−6 , s.t.: ∆ = smt
|osmt |
found by O PTI M ATH SAT( SMT ) and the optimal value found by the M INI Z INC solver
under test respectively. (Recall that the former was previously checked to be correct.)
Similarly to the previous experiment on the integers, O PTI M ATH SAT( FZN ) always produces correct results, and display comparable performance wrt. O PTI M ATH SAT( OMT ) in terms of number of instances being solved, solving somewhat fewer
problems. This is not the case of the other three M INI Z INC solvers. Among these,
G ECODE experienced a timeout on the majority of the formulas being considered,
G12( MIP ) returned mostly incorrect answers, whereas G UROBI seems to have the best
performance, in particular on the third benchmark-set.
We attribute the large number of incorrect results returned by all three M INI Z INC
solvers to the fact that these tools use finite-precision floating-point arithmetic internally. The incorrect behavior of some of these solvers (e.g. G UROBI) can also be partially explained with the large domain of floating-point variables in these problems.
However, given the nature of these input instances, it was not possible for us to assign
a smaller domain to each variable in the problem a priori.
We analyze separately the results for the last benchmark-set reported in Table 3. The
peculiar aspect of the Machine Learning benchmark-set [57] is that it is characterized by
Pseudo-Boolean sums over rational weights, and by very fine-grained rational values4 .
Unfortunately, these fine-grained rational values are rounded by the standard MZN 2 FZN
compiler, which causes the incorrect results even of O PTI M ATH SAT( FZN ) in Table 3,
despite the fact that O PTI M ATH SAT uses infinite-precision arithmetic.
In order to overcome this issue, we leverage the EMZN 2 FZN compiler described
in Section §3 to preserve the original fractional values, and show that with this approach O PTI M ATH SAT does not produce incorrect results any longer (configuration
O PTI M ATH SAT( FZN + E ) in Table 3), solving correctly 33 problems more than O PTI M ATH SAT( FZN ).
Despite this improvement, however, O PTI M ATH SAT( FZN + E ) still solves much less
problems than O PTI M ATH SAT( SMT ). We have analyzed this fact, and we noticed that
the performance gap is due to the structural changes introduced by the MZN 2 FZN com4
For example,
1799972218749879
2251799813685248
is a sample weight value from problems in [57].
14
piler, which prevents O PTI M ATH SAT( FZN +E) to use some efficient solving techniques
when dealing with the same problem given to O PTI M ATH SAT( SMT ). 5
Overall, since there are at least 177 formulas affected by the above issue with the
MZN 2 FZN compiler, we avoid an in-depth discussion of the results obtained by the other
M INI Z INC solvers. However, at a first glance the situation does not seem to differ from
the other benchmark-sets over the rationals.
5.3
Discussion
On the whole, from our experiments, OMT tools appear to be still at some disadvantage
when dealing with M INI Z INC problems wrt. specific tools, and vice versa.
On the one hand, OMT solvers seem to be penalized by their lack of efficient ad hoc
decision procedures for dealing with global constraints. Moreover, the approach taken
by the MZN 2 FZN compiler, that creates lots of alias Boolean, integer and floating-point
variables for dealing with Pseudo-Boolean constraints, is particularly challenging to
deal with efficiently by an OMT solver.
On the other hand, M INI Z INC solvers seem to suffer with problems needing an
arithmetic-reasoning component combined with heavy Boolean-reasoning component.
Even more importantly, the lack of infinite-precision linear arithmetic procedures causes
a number of incorrect results when dealing with OMT problems over the rationals. Both
of these points need to be addressed in order to deal with the vast number of Formal
Verification and Model Checking applications in the SMT/OMT domain.
6
Conclusions & Future Work
In this paper we have taken a first step forward towards bridging the M INI Z INC and the
OMT communities. The ultimate goal is to obtain a correct, effective and efficient fullyautomated system for translating problems from one community to the other, so as to
extend the application domain of both communities. With our experimental evaluation,
we have identified some criticalities that need to be addressed by each community in
order to solidify this union.
We plan to push this investigation forward as follows. In the short term, we plan
to address the inefficient handling of Pseudo-Boolean constraints over the rationals
revealed by the experimental evaluation in Section §5.2. In order to deal with those
F LAT Z INC constraints that require non-linear arithmetic, we envisage an opportunity to
either extend O PTI M ATH SAT with proper handling of the non-linear arithmetic theory
[20] or to experiment with an encoding based on the floating-point theory [58]. This
objective goes hand in hand with the extension of OMT 2 MZN to deal with other SMT
theories. In the long term, OMT solving may also benefit from adopting efficient ad hoc
decision procedures for frequently used global constraints. Finally, we plan to broaden
the scope of our investigation and include other OMT solvers in our study.
P
For example, an equality like (t = i Fi · wi ) where wi are float values, t in an unbounded
float variable
P and Fi ∈ {0.0, 1.0} are float variables, is rewritten by MZN 2 FZN into the equality
(0.0 = i Fi · wi + t · −1.0), where the Pseudo-Boolean sum is added to an arithmetic term,
preventing O PTI M ATH SAT( FZN +E) from recognizing it as the definition of a Pseudo-Boolean
sum, and hence from activating the specialized solving technique for Pseudo-Boolean equalities.
5
15
References
1. FZN 2 OMT. https://github.com/PatrickTrentin88/fzn2omt.
2. Benchmarks, Tools and Data. http://disi.unitn.it/trentin/resources/
cpaior2020.tar.xz.
3. omt2mzn. https://github.com/cespio/omt2mzn.
4. EMZN 2 FZN Repository. https://github.com/PatrickTrentin88/emzn2fzn.
5. Sal, symbolic analysis laboratory. http://sal.csl.sri.com.
6. SMT-LIB Format for Finite Lists, Sets and Maps. https://www.cprover.org/
SMT-LIB-LSM/.
7. C. Ansótegui, M. Bofill, M. Palahí, J. Suy, and M. Villaret. Solving weighted CSPs with
meta-constraints by reformulation into Satisfiability Modulo Theories. Constraints, 18(2),
2013.
8. R. Araújo, I. Bessa, L. C. Cordeiro, and J. E. C. Filho. SMT-based Verification Applied
to Non-convex Optimization Problems. In 2016 VI Brazilian Symposium on Computing
Systems Engineering (SBESC), Nov 2016.
9. R. Asín, R. Nieuwenhuis, A. Oliveras, and E. Rodríguez-Carbonell. Cardinality Networks:
a theoretical and empirical study. Constraints, 16(2), 2011.
10. C. Barrett, S. Ranise, A. Stump, and C. Tinelli. The satisfiability modulo theories library
(smt-lib). http://www.smtlib.org, 2010.
11. C. Barrett, R. Sebastiani, S. A. Seshia, and C. Tinelli. Satisfiability Modulo Theories, chapter 26, pages 825–885. Volume 185 of Biere et al. [14], February 2009.
12. C. Bertolissi, D. R. dos Santos, and S. Ranise. Solving Multi-Objective Workflow Satisfiability Problems with Optimization Modulo Theories Techniques. In SACMAT. ACM, 2018.
13. Z. Bian, F. Chudak, W. Macready, A. Roy, R. Sebastiani, and S. Varotti. Solving SAT and
MaxSAT with a Quantum Annealer: Foundations and a Preliminary Report. In Frontiers of
Combining Systems, volume 10483 of LNCS. Springer, 2017.
14. A. Biere, M. J. H. Heule, H. van Maaren, and T. Walsh, editors. Handbook of Satisfiability.
IOS Press, February 2009.
15. N. Bjorner, A.-D. Phan, and L. Fleckenstein. νZ - An Optimizing SMT Solver. In Proc.
TACAS, volume 9035 of LNCS. Springer, 2015.
16. M. Bofill, R. Nieuwenhuis, A. Oliveras, E. R. guez Carbonell, and A. Rubio. The Barcelogic
SMT Solver. In 20th International Conference on Computer Aided Verification, CAV’08,
volume 5123 of LNCS. Springer, 2008.
17. M. Bofill, M. Palahí, J. Suy, and M. Villaret. Solving constraint satisfaction problems with
SAT modulo theories. Constraints, 17(3), 2012.
18. M. Bofill, J. Suy, and M. Villaret. A System for solving constraint satisfaction problems with
SMT. Theory and Applications of Satisfiability Testing–SAT 2010, 2010.
19. L. Candeago, D. Larraz, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Speeding up
the Constraint-Based Method in Difference Logic. In SAT, volume 9710 of LNCS. Springer,
2016.
20. A. Cimatti, A. Griggio, A. Irfan, M. Roveri, and R. Sebastiani. Incremental linearization
for satisfiability and verification modulo nonlinear arithmetic and transcendental functions.
ACM Trans. Comput. Logic, 19(3), Aug. 2018.
21. H. Collavizza and M. Rueher. Exploration of the capabilities of constraint programming for
software verification. 03 2006.
22. H. Collavizza, M. Rueher, and P. Van Hentenryck. Cpbpv: a constraint-programming framework for bounded program verification. Constraints, 15(2), Apr 2010.
23. F. Contaldo, P. Trentin, and R. Sebastiani. From minizinc to optimization modulo theories,
and back (extended version), 2019. https://arxiv.org/abs/1912.01476.
i
24. H. A. Elgabou and A. M. Frisch. Encoding The Lexicographic Ordering Constraint in SAT
Modulo Theories. In In Proc. of Thirteenth International Workshop on Constraint Modelling
and Reformulation, 09 2014.
25. T. Feydy and P. J. Stuckey. Lazy Clause Generation Reengineered. In Proceedings of the 15th
International Conference on Principles and Practice of Constraint Programming, CP’09,
Berlin, Heidelberg, 2009. Springer-Verlag.
26. A. M. Frisch and P. A. Giannaros. SAT Encodings of the At-Most-k Constraint Some Old ,
Some New , Some Fast , Some Slow. 2010.
27. A. M. Frisch and M. Palahí. Anomalies in SMT Solving: Difficulties in Modelling Combinatorial Problems. In In Proc. of Thirteenth International Workshop on Constraint Modelling
and Reformulation, 09 2014.
28. M. Gario and A. Micheli. Pysmt: a solver-agnostic library for fast prototyping of smt-based
algorithms. In SMT workshop 2015, 13th International Workshop on Satisfiability Modulo
Theories, 2015.
29. P. J. Grinchtein O., Carlsson M. A constraint optimisation model for analysis of telecommunication protocol logs. In Tests and Proofs. TAP 2015., volume 9154 of LNCS. Springer,
2015.
30. G. E. Karpenkov. Finding inductive invariants using satisfiability modulo theories and convex optimization. Theses, Université Grenoble Alpes, Mar. 2017.
31. G. Kovásznai, B. Erdélyi, and C. Biró. Investigations of graph properties in terms of wireless sensor network optimization. In 2018 IEEE International Conference on Future IoT
Technologies (Future IoT), Jan 2018.
32. D. Larraz, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Minimal-Model-Guided
Approaches to Solving Polynomial Constraints and Extensions. In SAT, 2014.
33. F. Leofante, E. Abraham, T. Niemueller, G. Lakemeyer, and A. Tacchella. Integrated Synthesis and Execution of Optimal Plans for Multi-Robot Systems in Logistics. Information
Systems Frontiers, May 2018.
34. Y. Li, A. Albarghouthi, Z. Kincad, A. Gurfinkel, and M. Chechik. Symbolic Optimization
with SMT Solvers. In POPL, 2014.
35. T. Liu, S. S. Tyszberowicz, B. Beckert, and M. Taghdiri. Computing Exact Loop Bounds for
Bounded Program Verification. In SETTA, volume 10606 of LNCS. Springer, 2017.
36. MiniZinc. www.minizinc.org.
37. A. Nadel and V. Ryvchin. Bit-Vector Optimization. In Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2016, volume 9636 of LNCS. Springer, 2016.
38. N. Nethercote, P. J. Stuckey, R. Becket, S. Brand, G. J. Duck, and G. Tack. MiniZinc:
Towards a Standard CPModelling Language. In Principles and Practice of Constraint Programming – CP 2007, volume 4741 of LNCS. Springer Berlin Heidelberg, 2007.
39. C. M. Nguyen, R. Sebastiani, P. Giorgini, and J. Mylopoulos. Multi-objective reasoning
with constrained goal models. Requirements Engineering, 2016. In print. Published online
24 December 2016. DOI: http://dx.doi.org/10.1007/s00766-016-0263-5.
40. R. Nieuwenhuis and A. Oliveras. On SAT Modulo Theories and Optimization Problems. In
Proc. Theory and Applications of Satisfiability Testing - SAT 2006, volume 4121 of LNCS.
Springer, 2006.
41. R. Nieuwenhuis, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Challenges in Satisfiability Modulo Theories. In Proc. RTA’07, volume 4533 of LNCS. Springer, 2007.
42. R. Nieuwenhuis, A. Oliveras, and C. Tinelli. Solving SAT and SAT Modulo Theories: from
an Abstract Davis-Putnam-Logemann-Loveland Procedure to DPLL(T). Journal of the ACM,
53(6), November 2006.
43. O. Ohrimenko, P. J. Stuckey, and M. Codish. Propagation via lazy clause generation. Constraints, 14(3), 2009.
ii
44. R. S. Oliver, S. S. Craciunas, and W. Steiner. IEEE 802.1Qbv Gate Control List Synthesis
Using Array Theory Encoding. In 2018 IEEE Real-Time and Embedded Technology and
Applications Symposium (RTAS), April 2018.
45. C. H. Papadimitriou. On the complexity of integer programming. J. ACM, 28(4), 1981.
46. S. Ratschan. Simulation Based Computation of Certificates for Safety of Dynamical Systems. In Formal Modeling and Analysis of Timed Systems - 15th International Conference,
FORMATS 2017, Berlin, Germany, September 5-7, 2017, Proceedings, volume 10419 of
LNCS. Springer, 2017.
47. A. Rendl, T. Guns, P. J. Stuckey, and G. Tack. Minisearch: A solver-independent meta-search
language for minizinc. In Principles and Practice of Constraint Programming - 21st International Conference, CP 2015, Cork, Ireland, August 31 - September 4, 2015, Proceedings,
2015.
48. S. F. Roselli, K. Bengtsson, and K. Åkesson. SMT Solvers for Job-Shop Scheduling Problems: Models Comparison and Performance Evaluation. In 2018 IEEE 14th International
Conference on Automation Science and Engineering (CASE), Aug 2018.
49. N. W. Sawaya and I. E. Grossmann. A cutting plane method for solving linear generalized
disjunctive programming problems. Computers & Chemical Engineering, 29(9), 2005.
50. R. Sebastiani. Lazy Satisfiability Modulo Theories. Journal on Satisfiability, Boolean Modeling and Computation, JSAT, 3(3-4), 2007.
51. R. Sebastiani and S. Tomasi. Optimization in SMT with LA(Q) Cost Functions. In IJCAR,
volume 7364 of LNAI. Springer, July 2012.
52. R. Sebastiani and S. Tomasi. Optimization Modulo Theories with Linear Rational Costs.
ACM Transactions on Computational Logics, 16(2), March 2015.
53. R. Sebastiani and P. Trentin. OptiMathSAT: A Tool for Optimization Modulo Theories. In
Proc. International Conference on Computer-Aided Verification, CAV 2015, volume 9206 of
LNCS. Springer, 2015.
54. R. Sebastiani and P. Trentin. Pushing the Envelope of Optimization Modulo Theories with
Linear-Arithmetic Cost Functions. In Proc. Int. Conference on Tools and Algorithms for the
Construction and Analysis of Systems, TACAS’15, volume 9035 of LNCS. Springer, 2015.
55. R. Sebastiani and P. Trentin. On Optimization Modulo Theories, MaxSMT and Sorting Networks. In Proc. Int. Conference on Tools and Algorithms for the Construction and Analysis
of Systems, TACAS’17, volume 10205 of LNCS. Springer, 2017.
56. R. Sebastiani and P. Trentin. OptiMathSAT: A Tool for Optimization Modulo Theories.
Journal of Automated Reasoning, Dec 2018.
57. S. Teso, R. Sebastiani, and A. Passerini. Structured learning modulo theories. Artif. Intell.,
244, 2017.
58. P. Trentin and R. Sebastiani. Optimization Modulo the Theory of Floating-Point Numbers.
In 27th International Conference on Automated Deduction - CADE-27, volume 11716 of
LNCS. Springer, 2019.
59. M. Veksler and O. Strichman. Learning general constraints in CSP. Artif. Intell., 238, 2016.
60. N.-F. Zhou and H. Kjellerstrand. Optimizing SAT Encodings for Arithmetic Constraints. 08
2017.
iii