SAA For JCC
SAA For JCC
DOI 10.1007/s10957-009-9523-6
S. Ahmed · A. Shapiro
Georgia Institute of Technology, Atlanta, GA 30332, USA
S. Ahmed
e-mail: sahmed@isye.gatech.edu
A. Shapiro
e-mail: ashapiro@isye.gatech.edu
400 J Optim Theory Appl (2009) 142: 399–416
1 Introduction
1 We use the same notation ξ to denote a random vector and its particular realization. Which of these two
meanings will be used in a particular situation will be clear from the context.
J Optim Theory Appl (2009) 142: 399–416 401
(cdf) of standard normal random variable and by zα the corresponding critical value,
i.e., (zα ) = 1 − α, for α ∈ (0, 1),
k
N
B(k; p, N) := p i (1 − p)N −i , k = 0, . . . , N, (2)
i
i=0
2 Theoretical Background
In order to simplify the presentation we assume in this section that the constraint func-
tion G : Rn × → R is real valued. Of course, a number of constraints Gi (x, ξ ) ≤ 0,
i = 1, . . . , m, can be equivalently replaced by one constraint with
where
p(x) := P {G(x, ξ ) > 0}.
Now let ξ 1, . . . , ξ N
be an independent identically
distributed (iid) sample of N real-
izations of random vector ξ and PN := N −1 N j =1 (ξ j ) be the respective empirical
measure. Here (ξ ) denotes measure of mass one at point ξ , and hence PN is a dis-
crete measure assigning probability 1/N to each point ξ j , j = 1, . . . , N . The sample
average approximation p̂N (x) of function p(x) is obtained by replacing the ‘true’
distribution P by the empirical measure PN . That is, p̂N (x) := PN {G(x, ξ ) > 0}.
Let 1(0,∞) : R → R be the indicator function of (0, ∞), i.e.,
1, if t > 0,
1(0,∞) (t) :=
0, if t ≤ 0.
1
N
p̂N (x) = EPN [1(0,∞) (G(x, ξ ))] = 1(0,∞) (G(x, ξ j )).
N
j =1
402 J Optim Theory Appl (2009) 142: 399–416
That is, p̂N (x) is equal to the proportion of times that G(x, ξ j ) > 0. The problem
associated with the generated sample ξ 1 , . . . , ξ N , is
We refer to problems (4) and (5) as the true and SAA problems, respectively, at
the respective significance levels α and γ . Note that, following [11], we allow the
significance level γ ≥ 0 of the SAA problem to be different from the significance
level α of the true problem. Next we discuss the convergence of a solution of the
SAA problem (5) to that of the true problem (4) with respect to the sample size N
and the significance level γ . A convergence analysis of the SAA problem (5) has
been given in [11]. Here we present complementary results under slightly different
assumptions.
Recall that a sequence fk (x) of extended real valued functions is said to epicon-
e
verge to a function f (x), written fk → f , if for any point x the following two con-
ditions hold:
(i) for any sequence xk converging to x, one has
Note that, by the (strong) law of large numbers (LLN), we have that, for any x, p̂N (x)
converges w.p.1 to p(x).
Proposition 2.1 Let G(x, ξ ) be a Carathéodory function. Then the functions p(x)
e
and p̂N (x) are lower semicontinuous, and p̂N → p w.p.1. Moreover, suppose that
for every x̄ ∈ X the set {ξ ∈ : G(x̄, ξ ) = 0} has P -measure zero, i.e., G(x̄, ξ )
= 0
w.p.1. Then the function p(x) is continuous at every x ∈ X and p̂N (x) converges to
p(x) w.p.1 uniformly on any compact set C ⊂ X, i.e.,
Proof Consider function ψ(x, ξ ) := 1(0,∞) (G(x, ξ )). Recall that p(x) =
EP [ψ(x, ξ )] and p̂N (x) = EPN [ψ(x, ξ )]. Since the function 1(0,∞) (·) is lower
semicontinuous and G(·, ξ ) is a Carathéodory function, it follows that the function
ψ(x, ξ ) is random lower semicontinuous2 (see, e.g., [12, Proposition 14.45]). Then
by Fatou’s lemma we have for any x̄ ∈ Rn ,
lim inf p(x) = lim inf ψ(x, ξ )dP (ξ )
x→x̄ x→x̄
This shows lower semicontinuity of p(x). Lower semicontinuity of p̂N (x) can be
shown in the same way.
e
The epiconvergence p̂N → p w.p.1 is a direct implication of Artstein and Wets
[13, Theorem 2.3]. Note that, of course, |ψ(x, ξ )| is dominated by an integrable func-
tion since |ψ(x, ξ )| ≤ 1. Suppose, further, that for every x̄ ∈ X, G(x̄, ξ )
= 0 w.p.1,
which implies that ψ(·, ξ ) is continuous at x̄ w.p.1. Then by the Lebesgue Dominated
Convergence Theorem we have for any x̄ ∈ X,
lim p(x) = lim ψ(x, ξ )dP (ξ )
x→x̄ x→x̄
= lim ψ(x, ξ )dP (ξ ) = ψ(x̄, ξ )dP (ξ ) = p(x̄),
x→x̄
which shows continuity of p(x) at x = x̄. Finally, the uniform convergence (8)
follows by a version of the uniform law of large numbers (see, [9, Proposition 7,
p. 363]).
By lower semicontinuity of p(x) and p̂N (x) we have that the feasible sets of
the ‘true’ problem (4) and its SAA counterpart (5) are closed sets. Therefore, if the
set X is bounded (i.e., compact), then problems (4) and (5) have nonempty sets of
optimal solutions denoted, respectively, as S and ŜN , provided that these problems
have nonempty feasible sets. We also denote by ϑ ∗ and ϑ̂N the optimal values of the
true and the SAA problems, respectively. The following result shows that for γ = α,
under mild regularity conditions, ϑ̂N and ŜN converge w.p.1 to their counterparts of
the true problem.
We make the following assumption.
(A) There is an optimal solution x̄ of the true problem (4) such that for any ε > 0
there is x ∈ X with x − x̄ ≤ ε and p(x) < α.
In other words the above condition (A) assumes existence of a sequence {xk } ⊂ X
converging to an optimal solution x̄ ∈ S such that p(xk ) < α for all k, i.e., x̄ is an
accumulation point of the set {x ∈ X : p(x) < α}.
Proposition 2.2 Suppose that the significance levels of the true and SAA problems
are the same, i.e., γ = α, the set X is compact, the function f (x) is continuous,
G(x, ξ ) is a Carathéodory function, and condition (A) holds. Then ϑ̂N → ϑ ∗ and
D(ŜN , S) → 0 w.p.1 as N → ∞.
Proof By the condition (A), the set S is nonempty and there is x ∈ X such that
p(x) < α. We have that p̂N (x) converges to p(x) w.p.1. Consequently p̂N (x) < α,
and hence the SAA problem has a feasible solution, w.p.1 for N large enough. Since
p̂N (·) is lower semicontinuous, the feasible set of SAA problem is compact, and
hence ŜN is nonempty w.p.1 for N large enough. Of course, if x is a feasible solution
404 J Optim Theory Appl (2009) 142: 399–416
of an SAA problem, then f (x) ≥ ϑ̂N . Since we can take such point x arbitrary close
to x̄ and f (·) is continuous, we obtain that
Now let x̂N ∈ ŜN , i.e., x̂N ∈ X, p̂N (x̂N ) ≤ α and ϑ̂N = f (x̂N ). Since the set X is
compact, we can assume by passing to a subsequence if necessary that x̂N converges
e
to a point x̄ ∈ X w.p.1. Also we have that p̂N → p w.p.1, and hence
It follows that p(x̄) ≤ α and hence x̄ is a feasible point of the true problem, and thus
f (x̄) ≥ ϑ ∗ . Also f (x̂N ) → f (x̄) w.p.1, and hence
It follows from (9) and (10) that ϑ̂N → ϑ ∗ w.p.1. It also follows that the point x̄ is an
optimal solution of the true problem and then we have D(ŜN , S) → 0 w.p.1.
Condition (A) is essential for the consistency of ϑ̂N and ŜN . Think, for example,
about a situation where the constraint p(x) ≤ α defines just one feasible point x̄
such that p(x̄) = α. Then arbitrary small changes in the constraint p̂N (x) ≤ α may
result in that the feasible set of the corresponding SAA problem becomes empty. Note
also that condition (A) was not used in the proof of inequality (10). Verification of
condition (A) can be done by ad hoc methods.
Suppose now that γ > α. Then by Proposition 2.2 we may expect that with in-
crease of the sample size N , an optimal solution of the SAA problem will approach
an optimal solution of the true problem with the significance level γ rather than α.
Of course, increasing the significance level leads to enlarging the feasible set of the
true problem, which in turn may result in decreasing of the optimal value of the true
problem. For a point x̄ ∈ X we have that p̂N (x̄) ≤ γ , i.e., x̄ is a feasible point of the
SAA problem, iff no more than γ N times the event “G(x̄, ξ j ) > 0” happens in N
trials. Since probability of the event “G(x̄, ξ j ) > 0” is p(x̄), it follows that
It follows that if p(x̄) ≤ α and γ > α, then 1 − prob{p̂N (x̄) ≤ γ } approaches zero at
a rate of exp(−κN ), where κ := (γ − α)2 /(2α). Of course, if x̄ is an optimal solution
of the true problem and x̄ is feasible for the SAA problem, then ϑ̂N ≤ ϑ ∗ . That is,
if γ > α, then the probability of the event “ϑ̂N ≤ ϑ ∗ ” approaches one exponentially
fast. Similarly, we have that if p(x̄) = α and γ < α, then probability that x̄ is a
feasible point of the corresponding SAA problem approaches zero exponentially fast
(cf., [11]).
J Optim Theory Appl (2009) 142: 399–416 405
In order to get a lower bound for the optimal value ϑ ∗ we proceed as follows. Let
us choose two positive integers M and N , let
θN := B γ N; α, N
B(L − 1; θN , M) ≤ β. (12)
and hence calculate its optimal value ϑ̂Nm , m = 1, . . . , M. That is, solve M times the
corresponding SAA problem at the significance level γ . It may happen that problem
(13) is either infeasible or unbounded from below, in which case we assign its optimal
value as +∞ or −∞, respectively. We can view ϑ̂Nm , m = 1, . . . , M, as an iid sample
of the random variable ϑ̂N , where ϑ̂N is the optimal value of the respective SAA
problem at significance level γ . Next we rearrange the calculated optimal values in
(1) (M) (1) (2)
the nondecreasing order as follows ϑ̂N ≤ · · · ≤ ϑ̂N , i.e., ϑ̂N is the smallest, ϑ̂N is
the second smallest etc, among the values ϑ̂Nm , m = 1, . . . , M. We use the random
quantity ϑ̂N(L) as a lower bound of the true optimal value ϑ ∗ . It is possible to show
(L)
that with probability at least 1 − β, the random quantity ϑ̂N is below the true optimal
value ϑ ∗ , i.e., ϑ̂N is indeed a lower bound of the true optimal value with confidence
(L)
at least 1 − β (see3 [8]). We will discuss later how to choose the constants M, N
and γ .
X := {x ∈ Rn : eT x = 1, x ≥ 0}.
3 In [8] this lower bound was derived for γ = 0. It is straightforward to extend the derivations to the case
of γ > 0.
406 J Optim Theory Appl (2009) 142: 399–416
Note that, of course, E[r T x] = r T x, where r := E[r] is the corresponding mean vec-
tor. That is, the objective function of problem.
The motivation to study (14) is the portfolio selection problem going back to
Markowitz [15]. The vector x represents the percentage of a total wealth of one dol-
lar invested in each of n available assets, r is the vector of random returns of these
assets and the decision agent wants to maximize the mean return subject to having a
return greater or equal to a desired level v, with probability at least 1 − α. We note
that problem (14) is not realistic because it does not incorporate crucial features of
real markets such as cost of transactions, short sales, lower and upper bounds on the
holdings, etc. However, it will serve to our purposes as an example of an application
of the SAA method. For a more realistic model we can refer the reader, e.g., to [16].
First assume that r follows a multivariate normal distribution with mean vector r
and covariance matrix , written r ∼ N (r, ). In that case r T x ∼ N (r T x, x T x),
and hence (as it is well known) the chance constraint in (14) can be written as a
convex second order conic constraint (SOCC). Using the explicit form of the chance
constraint, one can efficiently solve the convex problem (14) for different values of α.
An efficient frontier of portfolios can be constructed in an objective function value
versus confidence level plot, that is, for every confidence level α we associate the
optimal value of problem (14). The efficient frontier dates back to Markowitz [15].
If r follows a multivariate lognormal distribution, then no closed form solution is
available. The sample average approximation (SAA) of problem (14) can be written
as
max r T x, s.t. p̂N (x) ≤ γ , (15)
x∈X
where
N
p̂N (x) := N −1 1(0,∞) (v − riT x)
i=1
and γ ∈ [0, 1). The reason we use γ instead of α is to suggest that for a fixed α, a
different choice of the parameter γ in (15) might be suitable. For instance, if γ = 0,
then the SAA problem (15) becomes the linear program
A recent paper by Campi and Garatti [17], building on the work of Calafiore and
Campi [18], provides an expression for the probability of an optimal solution x̂N of
the SAA problem (5), with γ = 0, to be infeasible for the true problem (4). That
is, under the assumptions that the set X and functions f (·) and G(·, ξ ), ξ ∈ , are
convex and that w.p.1 the SAA problem attains unique optimal solution, we have that,
for N ≥ n,
prob{p(x̂N ) > α} ≤ B(n − 1; α, N ), (17)
J Optim Theory Appl (2009) 142: 399–416 407
and the above bound is tight. Thus, for a confidence parameter β ∈ (0, 1) and a sample
size N ∗ such that
B(n − 1; α, N ∗ ) ≤ β, (18)
the optimal solution of problem (16) is feasible for the corresponding true problem
(14) with probability at least 1 − β.
For γ > 0, problem (15) can be written as the mixed-integer linear program
max r T x, (19a)
x,z
with one binary variable for each sample point. To see that problems (15) and (19)
are equivalent, let (x, z1 , . . . , zN ) be feasible for problem (19). Then, from the first
constraint of (19), we have zi ≥ 1(0,∞) (v − riT x), and so from the second constraint
of (19) we have
N
γ ≥ N −1 zi ≥ N −1 1(0,∞) (v − riT x) = p̂N (x). (20)
i
Thus, x is feasible to (15) and has the same objective value as in (19). Conversely,
let x be feasible for problem (15). Defining zi = 1(0,∞) (v − riT x), i = 1, . . . , N , we
have that (x, z1 , . . . , zN ) is feasible for problem (19) with the same objective value.
Given a fixed α in (14), it is not clear what are the best choices of γ and N for (19).
We believe it is problem dependent and numerical investigations will be performed
with different values of both parameters. We know from Proposition 2.2 that, for
γ = α the larger the N the closer we are to the original problem (14). However, the
number of samples N must be chosen carefully because problem (19) is a binary
problem. Even moderate values of N can generate instances that are very hard to
solve.
First we perform numerical experiments for the SAA method with γ = 0 (problem
(16)) assuming that r ∼ N (r, ). We considered 10 assets (n = 10) and the data
for the estimation of the parameters was taken from historical monthly returns of 10
US major companies. We wrote the codes in GAMS and solved the problems using
CPLEX 9.0. The computer was a PC with an Intel Core 2 processor and 2 GB of
RAM.
Let us fix α = 0.10 and β = 0.01. For these values, the sample size suggested by
(18) is N ∗ = 183. We ran 20 independent replications of (16) for each of the sample
408 J Optim Theory Appl (2009) 142: 399–416
sizes N = 30, 40, . . . , 200 and for N ∗ = 183. We also build an efficient frontier plot
of optimal portfolios with an objective value versus prob{r T xα ≥ v}, where xα is the
optimal solution of problem (14) for a given α. We show in the same plot (Fig. 1) the
corresponding objective function values and prob{r T x̂N ≥ v} for each optimal solu-
tion x̂N found for the SAA (16). To identify each point with a sample size, we used a
gray scale that attributes light tones of gray to smaller sample sizes and darker ones to
larger samples. The efficient frontier curve is calculated for α = 0.8, 0.81, . . . , 0.99
and then connected by lines. The vertical and horizontal lines are for reference only:
they represent the optimal value for problem (14) with α = 0.10 and the 90% relia-
bility level, respectively.
Figure 1 shows interesting features of the SAA (16). Although larger sample sizes
always generate feasible points, the value of the objective function, in general, is
quite small if compared with the optimal value 1.004311 of problem (14) with α =
0.10. We also observe the absence of a convergence property: if we increase the
sample size, the feasible region of problem (16) gets smaller and the approximation
becomes more and more conservative and therefore suboptimal. The reason is that for
increasingly large samples the condition riT x ≥ v for all i approaches the condition
prob{r T x ≥ v} = 1.
We performed similar experiments for the lognormal case. For each point obtained
in the SAA, we estimated the probability by Monte-Carlo techniques. The reader is
referred to [20] for detailed instructions of how to generate samples from a multi-
variate lognormal distribution. Since in the lognormal case one cannot compute the
efficient frontier, we also included in Fig. 2 the upper bounds for α = 0.02, . . . , 0.20,
calculated according to (12). The detailed computation of the upper bounds will be
given in the next subsection.
In order to find better candidate solutions for problem (14), we need to solve the
SAA with γ > 0, (problem (19)), which is a combinatorial problem. Since our portfo-
lio problem is a linear one, we still can solve problem (15) efficiently for a moderate
number (e.g., 200 constraints) of instances. We performed tests for problem (15) with
both distributions, fixing γ = 0.05 and 0.10 and changing N .
J Optim Theory Appl (2009) 142: 399–416 409
The best candidate solutions to (14) were obtained with γ = 0.05. We considered
different sample sizes from 30 to 200. Although several points are infeasible to the
original problem, we observe in Figs. 3 and 4 that whenever a point is feasible it is
close to optimal solution in the normal case or to the upper bound under lognormality.
For γ = 0.10, almost all generated points were infeasible in both cases, as seen in
Figs. 5 and 6.
To investigate the different possible choices of γ and N in problem (19), we cre-
ated a three dimensional representation which we will call γ N -plot. The domain is a
discretization of values of γ and N , forming a grid with pairs (γ , N ). For each pair
we solve an instance of problem (19) with these parameters and stored the optimal
value and the approximate probability of being feasible to the original problem (14).
The z-axis represents the optimal value associated to each point in the domain in the
grid. Finally, we created a surface of triangles based on this grid as follows. Let i be
the index for the values of γ and j for the values of N . If candidate points associ-
410 J Optim Theory Appl (2009) 142: 399–416
A method to compute lower bounds of chance constrained problems of the form (1)
was suggested in [8]. We summarized that procedure at the end of Sect. 2 leaving
the question of how to choose the constants L, M and N . Given β, M and N , it is
straightforward to specify L: it is the largest integer that satisfies (12). For a given N ,
the larger M the better because we are approximating the L-th order statistic of the
random variable ϑ̂N . However, note that M represents the number of problems to be
solved and this value is often constrained by computational limitations.
412 J Optim Theory Appl (2009) 142: 399–416
Approximation (21) suggests that for small values of α we should take N of order
O(α −1 ). If N is much bigger than 1/α then we would have to choose a very large
M in order to honor inequality (12). For instance if α = 0.10, β = 0.01 and N = 100
instead of N = 1/α = 10 or N = 2/α = 20, we need M to be greater than 100 000 in
order to satisfy (12), which can be computationally intractable for some problems. If
N = 200 then M has to be grater then 109 , which is impractical for most applications.
In [11], the authors applied the same technique to generate bounds on probabilistic
versions of the set cover problem and the transportation problem. To construct the
bounds they varied N and used M = 10 and L = 1. For many instances they obtained
lower bounds slightly smaller (less than 2%) or even equal to the best optimal values
generated by the SAA. In the portfolio problem, the choice L = 1 generated poor
upper bounds as we will see.
Since problem (14) is a maximization problem, we calculated upper bounds fixing
β = 0.01 for all cases and by choosing three different values for the constants L, M
and N . First we fixed L = 1 and N = 1/α (solid line in Fig. 2). The constant M
was chosen to satisfy the inequality (12). The results were not satisfactory, mainly
because M ended up being too small. Since the constant M defines the number of
samples from v̂N and since our problem is a linear one, we decided to fix M = 1 000.
Then we chose N = 1/α (dashed line) and 2/α (dotted line) in the next two
experiments. The constant L was chosen to be the largest integer such that (12) is
satisfied. Figure 2 shows the generated points for the SAA with γ = 0 along with the
upper bounds.
It is harder to construct upper bounds with γ > 0. The difficulty lies in an ap-
propriate choice of the parameters since we cannot have very large values of M or
N when solving binary programs. Our experience shows that it is not significantly
better than the bounds obtained with γ = 0.
4 Blending Problem
min x1 + x2 , (25a)
x1 ≥0,x2 ≥0
where N is the number of samples, ω1i and ω2i are samples from ω1 and ω2 , γ ∈ (0, 1)
and K is a positive constant positive constant greater or equal than 7.
We performed experiments similar to the ones for the portfolio problem so we will
present the results without details. In Fig. 8 we generated approximations for prob-
lem (22) with α = 0.05 using the SAA. The sample points were obtained by solving a
414 J Optim Theory Appl (2009) 142: 399–416
SAA with γ = 0.025 and sample sizes N = 60, 70, . . . , 150. The Campi-Garatti (18)
suggested value is N ∗ = 130. We included the efficient frontier for problem (22).
We will not show the corresponding figures for other values of γ , but the pattern ob-
served in the portfolio problem repeated: with γ = 0 almost every point was feasible
but far from the optimal, with γ = α = 0.05 almost every point was infeasible. The
parameter choice that generated the best candidate solutions was γ = α/2 = 0.025.
We also show the γ N -plot for the SAA of problem (22). We tested γ values in the
range 0, 0.005, . . . , 0.05 and N = 60, 70, . . . , 150. We included a plane representing
the optimal value of problem (22) for α = 0.05, which is readily obtained by applying
formula (23).
In accordance with Fig. 7, in Fig. 9 the best candidate solutions are the ones with
γ around 0.025. Even for very small sample sizes we have feasible solutions (dark
gray triangles) close to the optimal plane. On the other hand, this experiment gives
more evidence that the SAA with γ = 0 is excellent to generate feasible solutions
(dark gray triangles) but the quality of the solutions is poor. As shown in Fig. 9, the
J Optim Theory Appl (2009) 142: 399–416 415
high peaks associated with γ = 0 persist for any sample size, generating points far
form the optimal plane. In agreement with Fig. 7, the candidates obtained for γ > α
are in their vast majority infeasible.
5 Conclusions
We have discussed chance constrained problems with a single constraint and proved
convergence results about the SAA method. We applied the SAA approach to a port-
folio chance constrained problem with random returns. In the normal case, we can
compute the efficient frontier and use it as a benchmark solution. Experiments show
that the sample size suggested by [17] was too conservative for our problem: a much
smaller sample can yield feasible solutions. We observed that the quality of the solu-
tions obtained was poor. Similar results were obtained for the lognormal case, where
upper bounds were computed using a method developed in [8].
As another illustration of the use of the SAA method, we presented a two dimen-
sional blending problem and modeled it as a joint chance constrained problem. We
use the problem as a benchmark to test the SAA approach and also to show how one
can use the SAA methodology to approximate joint chance constrained problems.
In both cases we observe that the choice γ = α/2 gave very good candidate so-
lutions. Even though it generated more infeasible points if compared to the choice
γ = 0, the feasible ones were of better quality. Using the γ N -plot we were able to
confirm these empirical findings for our two test problems. Figures 7 and 9 tells us
that relatively small sample sizes (e.g, if compared to Campi-Garatti estimates) can
yield good candidate solutions. This is extremely important since for γ > 0 the SAA
problem is an integer program and large values of N could quickly make the SAA
approach intractable. Upper bounds were also constructed for the portfolio problem
using the SAA with γ = 0 by solving several continuous linear programs. Since no
closed solution is available for the portfolio problem with lognormal returns, having
an upper bound is an important information about the variability of the solution.
We believe that the SAA methodology is suitable to approximate chance con-
strained problems. Such problems are usually impossible to be solved explicitly
and the proposed method can yield good candidate solutions. One advantage of the
method is the fairly general assumption on the distribution of the random variables
of the problem. One must only be able to sample from the given distribution in order
to build the corresponding SAA program. The main contribution of the paper is to
present the method and its theoretical foundations and to suggest parameter choices
for the actual implementation of the procedure.
References
1. Charnes, A., Cooper, W.W., Symmonds, G.H.: Cost horizons and certainty equivalents: an approach
to stochastic programming of heating oil. Manag. Sci. 4, 235–263 (1958)
2. Prékopa, A.: Stochastic Programming. Kluwer Academic, Dordrecht (1995)
3. Dupačová, J., Gaivoronski, A., Kos, Z., Szántai, T.: Stochastic programming in water management: a
case study and a comparison of solution techniques. Eur. J. Oper. Res. 52, 28–44 (1991)
416 J Optim Theory Appl (2009) 142: 399–416
4. Henrion, R., Li, P., Möller, A., Steinbach, S.G., Wendt, M., Wozny, G.: Stochastic optimization for
operating chemical processes under uncertainty. In: Grötschel, M., Krunke, S., Rambau, J. (eds.)
Online Optimization of Large Scale Systems, pp. 457–478. Springer, Berlin (2001)
5. Henrion, R., Möller, A.: Optimization of a continuous distillation process under random inflow rate.
Comput. Math. Appl. 45, 247–262 (2003)
6. Dentcheva, D., Prékopa, A., Ruszczyński, A.: Concavity and efficient points of discrete distributions
in probabilistic programming. Math. Program. 89, 55–77 (2000)
7. Luedtke, J., Ahmed, S.: A sample approximation approach for optimization with probabilistic con-
straints. SIAM J. Optim. 19, 674–699 (2008)
8. Nemirovski, A., Shapiro, A.: Convex approximations of chance constrained programs. SIAM J. Op-
tim. 17(4), 969–996 (2006)
9. Shapiro, A.: Monte Carlo sampling methods. In: Ruszczyński, A., Shapiro, A. (eds.) Stochastic Pro-
gramming. Handbooks in OR & MS, vol. 10, pp. 353–425. North-Holland, Amsterdam (2003)
10. Atlason, J., Epelman, M.A., Henderson, S.G.: Optimizing call center staffing using simulation and
analytic center cutting plane methods. Manag. Sci. 54, 295–309 (2008)
11. Luedtke, J., Ahmed, S.: A sample approximation approach for optimization with probabilistic con-
straints. SIAM J. Optim. 19, 674–699 (2008)
12. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)
13. Artstein, Z., Wets, R.J.-B.: Consistency of minimizers and the SLLN for stochastic programs. J. Con-
vex Anal. 2, 1–17 (1996)
14. Chernoff, H.: A measure of asymptotic efficiency for tests of a hypothesis based on the sum observa-
tions. Ann. Math. Stat. 23, 493–507 (1952)
15. Markowitz, H.: Portfolio selection. J. Financ. 7, 77–97 (1952)
16. Wang, Y., Chen, Z., Zhang, K.: A chance-constrained portfolio selection problem under t -distribution.
Asia Pac. J. Oper. Res. 24(4), 535–556 (2007)
17. Campi, M.C., Garatti, S.: The exact feasibility of randomized solutions of robust convex programs.
Optimization online (www.optimization-online.org) (2007)
18. Calafiore, G., Campi, M.C.: The scenario approach to robust control design. IEEE Trans. Autom.
Control. 51, 742–753 (2006)
19. Klein Haneveld, W.K., van der Vlerk, M.H.: Stochastic Programming (lecture notes) (2007)
20. Law, A., Kelton, W.D.: Simulation Modeling and Analysis. Industrial Engineering and Management
Science Series. McGraw–Hill Science/Engineering/Math, New York (1999)