A Genetic Algorithm Based Augmented Lagrangian Method for

Accurate, Fast and Reliable Constrained Optimization

Kalyanmoy Deb

and Soumil Srivastava

Kanpur Genetic Algorithms Laboratory (KanGAL)
Indian Institute of Technology Kanpur
Kanpur, UP, India, PIN 208016
deb@iitk.ac.in, soumil.iitk@gmail.com
http://www.iitk.ac.in/kangal/pub.htm
KanGAL Report Number 2011012
Abstract
Among the penalty based approaches for constrained optimization, augmented Lagrangian
(AL) methods are better in at least three ways: (i) they have theoretical convergence prop-
erties, (ii) they distort the original objective function minimally, thereby providing a better
function landscape for search, and (iii) they can result in computing optimal Lagrange mul-
tiplier for each constraint as a by-product. Instead of keeping a constant penalty parameter
throughout the optimization process, these algorithms update the parameters (called multipli-
ers) adaptively so that the corresponding penalized function dynamically changes its optimum
from the unconstrained minimum point to the constrained minimum point with iterations.
However, the ip side of these algorithms is that the overall algorithm requires a serial applica-
tion of a number of unconstrained optimization tasks, a process that is usually time-consuming
and tend to be computationally expensive. In this paper, we devise a genetic algorithm based
parameter update strategy to a particular AL method. The proposed strategy updates critical
parameters in a self-adaptive manner based on population statistics. Occasionally, a classical
optimization method is used to improve the GA-obtained solution, thereby providing the re-
sulting hybrid procedure its theoretical convergence property. The GAAL method is applied
to a number of constrained test problems taken from the evolutionary algorithms (EAs) liter-
ature. The number of function evaluations required by GAAL in most problems is found to
be smaller than that needed by a number of existing evolutionary based constrained handling
methods. GAAL method is found to be accurate, computationally fast, and reliable over mul-
tiple runs. Besides solving the problems, the proposed GAAL method is also able to nd the
optimal Lagrange multiplier associated with each constraint for all test problems as an added
benet a matter that is important for a sensitivity analysis of the obtained solution, but
has not yet been paid adequate attention in the past evolutionary constrained optimization
studies.
1 Introduction
Constrained optimization problems are the most important and ubiquitous type of engineering
optimization problems. Evolutionary algorithms (EA) have been applied extensively for tackling
these problems with various degrees of success. The penalty function approach is a relatively
simple and remarkably easy to implement and therefore has been popularly used with an EA. But

K. Deb is also an adjunct professor at the Department of Information and Service Economy, Aalto University,
Helsinki, Finland.
1
the approach also has several functional and operational drawbacks which limit its precision and
speed of convergence [6, 19]. First, the penalty function approaches do not take into account the
mathematical form (linear or non-linear, convex or non-convex, unimodal or multi-modal, etc.) of
the constraints. Second, the addition of the penalty term on the objective function usually makes a
distortion in the objective function, which may create articial non-linearity and multi-modalities
that may thwart the progress of an optimization algorithm to the true constrained minimum
point. Third, most penalty function methods do not have a convergence proof and works under
the assumption that optimal solutions of the subsequent penalized functions approach the true
constrained minimum of the problem. Various types of sophisticated penalty function methods
have been devised in the past to overcome some of these drawbacks in the context of evolutionary
algorithms, such as multi-level penalty functions [11], dynamic penalty functions [12], and others.
But according to a study by Michalewicz and Schoenauer [16], such sophisticated methods are
less generic and work well on some problems but not on others.
In 2000, the rst author had proposed a parameter-less approach [7] for constrained handling
within a GA that eliminated the need for a penalty parameter and was received quite favorably.
Other popular approaches used a two-objective method [24] or, in general, a multi-objective
approach [4] for handling constraints. Other approaches used other evolutionary algorithms,
such as the particle swarm optimization (PSO) [25], dierential evolution (DE) [21] and reported
excellent performance. Most of these studies in evolutionary algorithms that used a penalized
technique for handling infeasible solutions have used some form of a penalty function. However,
the augmented Lagrangian methods have received a large attention in the recent past for solving
constrained optimization problems [1, 2, 17, 23, 26]. The augmented Lagrangian methods are
better than direct penalty function approaches in a number of ways:
1. AL methods usually have a theoretical convergence property to a Karush-Kuhn-Tucker
(KKT) point. The particular AL approach used in this study has such a property.
2. The AL approach does not distort the original objective function, instead it has an eect
of shifting the objective function in a manner so that its optimum moves from the uncon-
strained minimum to the constrained minimum point. This does not change the complexity
of the problem too much from start to nish and makes it easier to choose a single opti-
mization algorithm for the entire problem-solving task.
3. As a by-product, AL methods nd the Lagrange multiplier value for each constraint. La-
grange multipliers are important for subsequent post-optimality analysis and for sensitivity
analysis [18, 19, 13]. The ability to compute Lagrange multipliers provides AL methods a
denite edge over penalty function approaches and for that matter over most other opti-
mization algorithms.
However, one of the criticisms of AL methods is the need for solving a number of serial opti-
mization tasks, which eventually makes the overall approach computationally expensive. Also, a
current optimization task depends largely on the accuracy of previous optimization tasks, thereby
requiring to solve every optimization problem with an appropriate accuracy. Another matter
which is not adequately discussed in the literature, but is also an important factor is that particu-
larly in solving multi-modal constrained problems, if an earlier optimization task did not converge
to an appropriate optimum point that would eventually lead the process to the true constrained
optimum, the value of the associated parameters computed at the false optimum will be dierent
from that are required for the algorithm to converge to the true constrained minimum point. The
use of a more global optimization algorithm to solve every intermediate problem may alleviate
such a false convergence.
In this paper, we revisit the use of a classical augmented Lagrangian (AL) approach, but
instead of using a classical unconstrained optimization algorithm to solve the resulting penal-
ized functions one at a time, we extend our previous pilot study [20] here and employ a genetic
2
algorithm to seamlessly solve penalized functions by updating the associated parameters adap-
tively. For each constraint (of equality and inequality type), the proposed AL method involves
a dynamically changed parameter. The population approach of the proposed GA-based method
maintains diversity of solutions and allows a more global search of the individual optimization
tasks. Furthermore, the switching from one parameter setting to another is made completely in
a self-adaptive manner based on population statistics so that once started the user does not have
to control the switching from one optimization task to another, as it is the case in classical way
of applying the AL method. However, to induce convergence property of the proposed algorithm,
a classical optimization procedure (the fmincon routine of MATLAB) is embedded within the
proposed GA-based procedure to terminate a run.
In the remainder of the paper, we rst describe the proposed AL-based constraint handling
method and the various operators (such as the adaptive mutation and self-adaptive penalty pa-
rameter update procedure) incorporated within it to ensure a fast and accurate convergence.
We then recommend values for the parameters based on two parametric studies. Thereafter, we
present the performance of the proposed method on a number of standard constrained test prob-
lems taken from the EA literature, compare its results to the classical method, and highlight the
benets of the proposed approach. Conclusions of the study are presented next.
2 Augmented Lagrangian Method and the Proposed Algorithm
A general notation for a constrained optimization problem is:
Minimize f(x),
subject to g
j
(x) 0, j = 1, 2, . . . , J,
h
k
(x) = 0, k = 1, 2, . . . , K,
x
l
i
x
i
x
u
i
, i = 1, 2, . . . , n.
(1)
In a particular classical augmented Lagrangian method for constrained optimization, the objective
function f(x) is modied as follows:
P(x,
t
,
t
) = f(x) + R
J

j=1
_
_
g
j
(x) +
t
j

_
2

t
j
_
2
_
+ R
K

k=1
_
_
h
k
(x) +
t
k
_
2

t
k
_
2
_
,
(2)
where g(x) = g(x) if g(x) < 0 else it is zero. The superscript t indicates the iteration counter.
In solving the t-th optimization problem, multipliers
t
j
and
t
k
are kept xed. The initial values
of these multipliers are set to zero, or
0
j
= 0 and
0
k
= 0. However, these multipliers are updated
at the end of t-th optimization run and are kept xed for the (t + 1)-th iteration, as follows:

t+1
j
=
t
j
+ g
j
(x
t
), j = 1, 2, . . . , J, (3)

t+1
k
=
t
k
+ h
k
(x
t
), k = 1, 2, . . . , K, (4)
where x
t
is the optimum solution of the t-th penalized function P. It is also important to
mention that the starting solution of the initial optimization task can be chosen as any arbitrary
solution, whereas the initial solution to the t-th (t 1) optimization problem is the optimum
solution found at (t 1)-th optimization task. This sequential optimization procedure continues
till an insignicant change in the optimum solutions of the two consecutive optimization tasks is
observed. It is also important to mention that the penalty parameter R is usually kept constant
throughout the entire optimization process.
3
A major result which can be obtained from a KKT analysis of the above penalized optimization
problem is that at a theoretical termination of the algorithm (say, occurring at iteration T), the
optimal Lagrangian multipliers for the original problem can be calculated by using the following
relationships:
u
j
= 2R
T
j
, (5)
v
k
= 2R
T
k
. (6)
Although the above description and its associated theoretical result are attractive, there are
some practical implementational issues which we discuss next. First, since the value of R is kept
xed throughout the optimization process, it may be apparent that its value can be xed to any
arbitrary value. Although theoretically it may be justied, in practice the performance of an
algorithm largely depends on the chosen value of R. Notice that all constraints are multiplied
by the same penalty parameter R. It is discussed enough in the context of penalty parameter
approaches that this causes some constraints to behave as more important than others. Due to
the possibility of constraints taking values of dissimilar orders of magnitude, unequal treatment
of constraints may not allow the overall algorithm to achieve a systematic convergence to the
constrained minimum. Constraints taking higher magnitude of values will get emphasized more.
Thus, there is a need for normalizing the constraint functions before using them in Equation 2
[6, 10]. Second, the value of R makes a balance between the objective function f and the constraint
violation given by the second and third terms of the right side of Equation 2. Unless a proper
balance is established, either the objective function is not emphasized enough or the constraint
violation is not adequately emphasized. The above scenario will result in a premature convergence
to an arbitrary feasible solution or to an infeasible solution, respectively. Importantly, for arriving
at a computationally ecient approach, such a balance must be made adaptively with more
emphasis to constraint violation in the beginning of the optimization process and then a relatively
larger emphasis to the objective function towards later generations. As done in penalty function
approaches, in AL method as well, there is a need for updating R with the optimization iteration
counter t. We have proposed here a self-adaptive approach here for updating R by using a statistics
of the current population objective values. Once a new R is set at the end of some iterations,
the corresponding multipliers are also updated by using the theoretical relationships presented in
Equations 5 and 6. Since optimal Lagrange multipliers u
j
and v
k
are properties of the supplied
optimization problem, the equations suggest that the new
j
and
k
values must be changed in
an inversely proportionate manner with the new R value.
Based on these discussions, we suggest a GA based AL method with the following properties:
1. GAAL self-adaptively alters the value of R whenever a sucient convergence is observed.
2. GAAL self-adaptively determines the update of multiplier values
j
from one iteration to
another and based on a theoretical basis whenever there is a change in R value.
3. GAAL uses a termination condition based on solutions obtained for successive R updates
and its subsequent classical optimization run.
4. GAAL is seamless in that no user intervention is needed once the algorithm is started.
We now discuss the details of the proposed GAAL method.
2.1 Proposed GAAL Method
In this study, we restrict ourselves to inequality constraints only, however extensions of the current
study can be made for equality constraints as well. In the following, we rst present the proposed
algorithm and then discuss its individual operators:
4
1. Initialization
(a) Set t = 0. Initialize the GA population P
t
with N members randomly.
(b) Set
t
j
= 0 for all constraints, j = 1, 2, . . . , J.
(c) Set the initial penalty parameter R based on the statistics of the initial population as
follows:
R = 50

N
|f(x)|

N
CV(x)
, (7)
where CV(x) =

J
j=1
g
j
(x)
2
and g
j
is the normalized version of the constraint func-
tion g
j
. Set R
old
= R.
(d) Record the best population member x
b,t
and increment t by one.
2. For every generation t, perform the following steps:
(a) Perform one generation of the GA using the binary tournament selection operator
with the penalized function P() (equation 2) as tness measure to be minimized, the
simulated binary crossover (SBX) operator [8], and the adaptive polynomial mutation
operator [15] on population P
t
to create a new ospring population Q
t
of size N.
(b) Choose the best population member x
b,t
of Q
t
. Set P
t
= Q
t
and compute =

P(x
b,t
,
t
)P(x
b,t1
,
t1
)
P(x
b,t1
,
t1
)

.
(c) If (t mod ) = 0 and
1
,
Reset penalty parameter R as follows:
R =
10

N
|f(x)|
N
+ (1 ) R
old
, (8)
and update multipliers as follows:

t
j
=
t
j
R
old
R
, j = 1, 2, . . . , J.
Set R
old
= R and employ a classical optimization algorithm with the current
best population member (x
b,t
) as the initial solution to nd a new solution y
t
.
Replace the worst solution of the population P
t
by y
t
. Increment counter t
by one and go to Step 3.
Else go to Step 2(d).
(d) If
2
, update the multipliers using:

t+1
j
=
t
j
+ g
j
(x
b,t
), j = 1, 2, . . . , J.
Increment counter t by one and go to Step 2(a).
3. Termination check: If the absolute dierence between f(y
t
) and f(y
t1
) is less than or
equal to a pre-specied small number , report y
t
as the GAAL optimized solution, com-
pute Lagrange multipliers using Equation 5, and the procedure is terminated. Otherwise,
increment counter t by one and go to Step 2(a).
The automatic update of penalty parameter is an important feature of this algorithm. Initially,
the penalty parameter R is set such that the average constraint violation penalty is scaled to 50
(chosen after some trial and error experiments) times the average absolute objective function
value. This provides the algorithm adequate exibility to venture into both infeasible and feasible
regions while maintaining moderately high penalty on infeasible regions. Subsequently, during
5
the course of the run, the penalty parameter is updated according to Equation 8 as and when
the rate of improvement in penalized function value is below a certain threshold. The weighted
transition approach with a weight to the new calculated parameter allows a gradual change in
R. This facilitates in a smoother convergence of the algorithm, as we shall demonstrate later.
Importantly, we update the penalty parameter R in Step 2(c) and the multipliers
t
j
in
Step 2(d) at two dierent levels in this method. If the improvement in the best population
member (x
b,t
) is less than
2
(a user-dened parameter) then we update the multipliers alone and
continue. If this update does not yield better results in the succeeding generations below another
threshold (
1
), we make a change in the value of R for the algorithm to look for better solutions
elsewhere in the search space. Parameters
1
and
2
must be set such that
2

1
. This is to
ensure that the multiplier update (Step 2(d)) occurs more often than the R update (Step 2(c)).
From the relationship between u
j
,
j
and R given in Equation 5, we then update the multipliers
in a way so as to make the product of R and
j
constant.
The SBX operator is the same as that described in [8]. However, we use an adaptive mutation
operator in which the mutation probability is changed with the generation counter as follows [9]:
p
m
=
0.1
n
+
0.1t
t
max
_
1
1
n
_
, (9)
where n is the number of variables, t is the number of generations and t
max
is the maximum
number of generations set by the user during the run. This creates about 1% perturbance in the
solutions in the rst generation and subsequently decreases the perturbance in later generations.
The constant values of 50 and 10 used in Equations 7 and 8, respectively, are found to produce
better results through trial-and-error experiments. Other parameters used in the algorithm are
set as follows:
1. Population size N: We recommend the use of N = max(10n, 50), where n is the number of
variables in the problem.
2. Probability of crossover p
c
and SBX index
c
: We recommend p
c
= 0.9 and
c
= 2. They
are found to work well on all our test problems.
3. Mutation index
m
: We recommend
m
= 100 + t.
4. Update threshold
1
: Based on simulation studies, we recommend using
1
= 0.0001 or
0.01%. This parameter is used to decide on an update of R and start a classical optimization
method as a local search operator.
5. Update threshold
2
: Based on simulation studies, we recommend using
2
= 0.1 or 10%.
This parameter is used to decide on an update of the multipliers.
6. R update parameter : This parameter is used to prevent too frequent changes in R.
7. Weight parameter : This parameter determines the extent of transition towards the newly
determined penalty parameter. This is important as it directly aects the speed and nature
of convergence.
8. Terminating parameter : We choose = 0.0001.
After a solution with four decimal places of accuracy (with = 0.0001) in the objective value is
achieved, the optimized variable vector is presented here with six decimal places.
Next, we perform a parametric study to examine the eect of two important parameters
and . Other parameter values are xed at the standard values mentioned above or are not found
to make any signicant change in the performance of our proposed algorithm.
6
0 10 20 30 40 50 60 70 80 90 100
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
x 10
5

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 1: Function evaluations versus for g02. As is increased, more function evaluations are
needed.
2.2 Parametric Study :
We performed a parametric study of on three constrained optimization problems: g02, g08 and
g10, borrowed from the literature [14]. Figures 1, 2 and 3 present the results in a comprehensible
manner. It is observed that on increasing the value of , more number of function evaluations
are required in solving the problems. But, it is also observed that if is set to one, that is,
allowing fmincon routine to be invoked at every generation, the procedure leads to a premature
convergence for some problems (g08, for example). Besides, by performing the local search so
frequently does not allow the GA procedure enough time to improve the solutions between two
local search procedures. Based on the stated observations, we recommend = 5.
2.3 Parametric Study :
Next, we perform a parametric study of on the same three problems: g02, g08 and g10. The
parameter is varied within [0, 1]. Figures 4, 5 and 6 present the results for the three problems,
respectively. It can be observed that the required function evaluations are smaller for intermediate
values of . Based on the stated observations, we recommend = 0.5, in which an equal weight
( = 0.5) is assigned to both newly calculated R and the previous R. As depicted in the gures,
too much emphasis on the old value or the new value does not yield good results.
3 Application on Test Problems
By setting = 5 and = 0.5, we now apply our proposed GAAL algorithm to a number of
constrained test problems and compare the performance of GAAL with existing evolutionary
based constrained handling algorithms.
7
0 10 20 30 40 50 60 70 80 90 100
0
2
4
6
8
10
12
x 10
4

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 2: Function evaluations versus for g08. As is increased, more function evaluations are
needed but reaches a plateau after a value of around 50.
0 10 20 30 40 50 60 70 80 90 100
0
1
2
3
4
5
6
7
x 10
4

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 3: Function evaluations versus for g10. As is increased, more function evaluations are
needed.
8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
5
10
15
x 10
4

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 4: Function evaluations versus for g02. has little eect on function evaluations for
this problem.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
5
10
15
x 10
4

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 5: Function evaluations versus for g08. As is increased beyond 0.8 or decreased below
0.1, more function evaluations are needed.
9
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.5
1
1.5
2
2.5
3
3.5
4
x 10
4

F
u
n
c
t
i
o
n

e
v
a
l
u
a
t
i
o
n
s


Average
Median with minmax errorbars
Figure 6: Function evaluations versus for g10. As is decreased beyond 0.1, required number
of function evaluations increase rapidly.
3.1 Problem A: g18
This problem has nine variables and 13 inequality constraints. Refer to appendix for a detail
description of this problem. The best known optimum is as follows:
x

=(0.657776, 0.153419, 0.323414, 0.946258, 0.657776,

0.753213, 0.323414, 0.346463, 0.599795),
with f

= 0.866025. The optimal solution obtained by GAAL is shown below:

x
GAAL
=(0.490498, 0.871442, 0.999940, 0.010937, 0.490498,
0.871442, 0.999940, 0.010937, 0.000000),
with f
GAAL
= 0.866025. Clearly, although there is a dierence in the decision variable vectors,
both solutions are feasible and the objective function values are identical at least to six decimal
places of accuracy. This indicates that the problem is multi-modal having a number of feasi-
ble constrained optimal solutions and the best-reported solution lies on one optimum and our
GAAL-obtained solution lies on another optimum. The best, median and worst function values
obtained in 25 dierent runs started from dierent initial populations are 0.866025, 0.866025
and 0.674981, respectively. Only one out of the 25 runs did not nd the the optimum accurately.
The rest 24 runs of GAAL are within 0.01% of the optimum. The corresponding Lagrangian
multipliers for the above-mentioned GAAL solution are as follows: (0.14291952, 0, 0.13948934,
0.14266385, 0, 0.14418376, 0.14966718, 0, 0.14274691, 0, 1.13003191, 0, 0), thereby indicating
that constraints g
1
, g
3
, g
4
, g
6
, g
7
, g
9
, and g
11
are active at the optimum.
We choose this problem to demonstrate the self-adaptive behavior of the associate algorithmic
parameters such as the penalty parameter R and multipliers
j
with the generation counter.
10
Figure 7 shows the variation of population-best penalized function value P with the generation
counter. It is clear from the gure that after some initial uctuations, the penalized function value
stabilizes to its minimum value. A similar trend happens for the corresponding objective function
value f (Figure 8). In the initial few generations, the population-best solutions are infeasible,
hence the objective values are apparently smaller than f

= 0.866025, but when GAAL manages

to nd the feasible region, the corresponding objective value is higher. Eventually, the algorithm
is able to converge to an optimum solution having f

= 0.866025.
The variations of P and f shown above are typical for penalty-based constrained handling
algorithms. Let us now investigate how the algorithmic parameters (such as R and
j
) self-adapt
themselves with the generation counter so as to follow the established theoretical relationship
leading to the optimal Lagrange multiplier value for each constraint at the end. First, we plot the
variation of the penalty parameter R in Figure 9. Although R can get updated after every = 5
generations, the
1
condition does not allow the algorithm to have too frequent changes. At
generation 35, the penalty parameter R is increased to a value of 776.25 from 33.27. As Figure 8
reveals, the reason for this drastic change in R is due to a sharp increase in objective values (refer
to the R-update equation). Since an appropriate value of R is expected to make a balance between
the constraint violation value and the objective function value, our carefully designed R-update
rule (Equation 8) enables an increase in R so as to match the range of constraint values with that
of the objective values. Thereafter, despite some slight changes in R with further generations, a
self-adaptive drastic change of R at generation 35 shifted the focus of the algorithm from infeasible
region to feasible region and was an important and necessary event for the success of the algorithm
in getting to the constrained minimum solution.
Next, we show the variation of multipliers for a few constraints for the population-best solution
with the generation counter in Figures 10 to 15. Starting with with
0
j
= 0, all
t
j
values reduce as
the population-best solution remains infeasible during the initial generations. At generation 35,
the penalty parameter R is self-adapted to a high value, thereby making the absolute value of
all
j
values small (recall the inverse relationship between R and
j
values). Thereafter, the
0 20 40 60 80 100 120
5
0
5
10
15
20
25
generations
P
Figure 7: Problem g18: P versus generation counter.
11
0 20 40 60 80 100 120
10
9
8
7
6
5
4
3
2
1
0
generations
f
Figure 8: Problem g18: f versus generation counter.
multipliers gets adjusted by the AL rules given in Equation 3. Interestingly,
5
returns to a value
zero after being negative in the initial generations and make a correct computation of a zero value
0 20 40 60 80 100 120 140
0
100
200
300
400
500
600
700
800
generations
R
Figure 9: Problem g18: R versus generation counter.
12
0 50 100 150
0.2
0.15
0.1
0.05
0
generations

1
Figure 10: Problem g18:
1
versus generation
counter.
0 50 100 150
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
generations

4
Figure 11: Problem g18:
4
versus generation
counter.
0 50 100 150
0.1
0.08
0.06
0.04
0.02
0
generations

5
Figure 12: Problem g18 :
5
versus generation
counter.
0 50 100 150
0.25
0.2
0.15
0.1
0.05
0
generations

6
Figure 13: Problem g18:
6
versus generation
counter.
0 50 100 150
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
generations

7
Figure 14: Problem g18:
7
versus generation
counter.
0 50 100 150
0.12
0.1
0.08
0.06
0.04
0.02
0
generations

1
1
Figure 15: Problem g18:
11
versus generation
counter.
of the corresponding Lagrange multiplier value (u
5
). Besides the drastic change at generation 35
(mainly due to the change in R at this generation), subsequent changes in multiplier values can
also be observed, particularly for
11
in Figure 15. Since all Lagrange multipliers (u
j
) must be
non-negative at the constrained optimum solution, it is interesting to note that all multiplier
values (
j
) settle down to non-positive values.
13
The performance of GAAL in this problem amply demonstrates its self-adaptive property and
its ability to adjust all parameters associated with 13 constraints to reach near a constrained
optimum solution. The occasional updates of R to make a right balance between objective func-
tion and the constraint values help steer the search in the right direction. Thereafter, frequent
updates of multipliers (
j
) using the AL principle provided local improvements in the solutions.
Importantly, a seamless operation orchestrating the adjustments of multiple parameters in the
right direction is achieved by the use of theory of augmented Lagrangian approach and by the
global search power of genetic algorithms.
3.2 Problem B: g01
The next problem has 13 variables and nine inequality constraints. The appendix presents the
mathematical formulation of the problem. The best-known optimum for this problem is as follows:
x

= (1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1).
The corresponding f

= 15. Six constraints (all except g

4
, g
5
and g
6
) are active at this optimum.
The best optimized solution of GAAL is given below:
x
GAAL
= (1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 3, 3, 1),
with f
GAAL
= 15. This solution is identical to best-known solution. The best, median and
worst objective values obtained for 25 dierent runs are 15, 15 and 12.453125, respectively.
The proposed method has performed quite well in this problem as well. Only one out of the 25
runs is not able to nd the the optimum accurately. The rest 24 runs of GAAL were within 0.01%
of the optimum. The corresponding Lagrangian multipliers for the best optimized solution are as
follows: u = (0.0042346, 0.0046562, 0.0054315, 0, 0, 0, 1.021383, 1.012629, 0.990493), indicating
that constraints g
4
, g
5
, and g
6
are inactive. This agrees with the known result on this problem.
For brevity, we do not show any gure for this or other problems demonstrating the variation of
the algorithmic parameters with the generation counter.
It is clear from the two applications that a highlight of the use of an AL method in a GA is
that in addition to the optimized solution, the proposed GAAL method is also able to supply us
with the optimal Lagrange multiplier values which can indicate the relative importance of each
constraint to the obtained optimized solution a matter which is important from a practical
standpoint.
3.3 Problem C: g02
This problem has 20 variables and two inequality constraints. The best-known optimum (upto
nine decimal places of accuracy) for this problem is given as follows:
x

=(3.162460616, 3.128331428, 3.094792129, 3.061451495, 3.027929159,

2.993826067, 2.958668718, 2.921842273, 0.494825115, 0.488357110,
0.482316427, 0.476645751, 0.471295508, 0.466230993, 0.461420050,
0.456836648, 0.452458769, 0.448267622, 0.444247010, 0.440382860),
with f

= 0.803619104. Only constraint g

1
is active at this solution. The optimized solution of
GAAL method (upto six decimal places of accuracy) is given as follows:
x
GAAL
=(3.162461, 3.128331, 3.094792, 3.061451, 3.027929, 2.993826,
2.958669, 2.921842, 0.494825, 0.488357, 0.482316, 0.476645,
0.471296, 0.466231, 0.461420, 0.456837, 0.452459, 0.448268,
0.444247, 0.440383),
14
and f
GAAL
= 0.803619. Interestingly, the above GAAL solution is exactly the same as the
best-known solution upto at least six decimal places of accuracy. The best, median and worst
objective values were 0.803619, 0.803619 and 0.744767, respectively. Here, two out of the 25
runs are not able to nd the optimum accurately. But all 25 runs of GAAL were within 10% of
the best-known optimum solution. The corresponding Lagrangian multipliers are reported as u =
(0.074457, 0), indicating that only constraint g
1
is active at the optimized solution. Interestingly,
this agrees with the known result for this problem.
3.4 Problem D: g04
The next problem has ve variables and six inequality constraints. The best-known optimum
solution (upto nine decimal places of accuracy) is as follows:
x

= (78, 33, 29.995256026, 45, 36.775812906),

with f

= 30665.538671783. Constraints g
1
and g
6
are active at this point. The result of the
GAAL method is a follows:
x
GAAL
= (78, 33, 29.995256, 45, 36.775813),
with f
GAAL
= 30665.538672 a solution identical (upto six decimal places of accuracy) to the
best-known solution. The best, median and worst objective values obtained from 25 dierent
runs are 30665.538672, 30665.538672 and 30665.538672, respectively, meaning that all 25
runs have found an identical solution. The proposed GAAL method performed quite well in this
problem.
3.5 Problem E: g06
This problem has two variables and two inequality constraints. The best-known optimum is given
with nine decimal places of accuracy as follows:
x

= (14.095000000, 0.842960789),
and f

= 6961.813875580. Both constraints are active at this optimum. The GAAL optimized
solution is presented with six decimal places of accuracy as follows:
x
GAAL
= (14.095000, 0.842961),
and f
GAAL
= 6961.813876. Upto six decimal places of accuracy, GAAL solution is identical to
that of the best-known solution. The best, median and worst objective values are 6961.813876,
6961.813876 and 6961.813876, respectively, meaning once again that all 25 runs are able to
nd the same optimized solution. The corresponding Lagrangian multipliers are calculated as
u = (97739.988401, 172760.954414), indicating that both constraints are active at the GAAL
optimized solution.
3.6 Problem F: g07
This problem has 10 variables and eight inequality constraints. The best-known optimum (with
nine decimal places of accuracy) is as follows:
x

=(2.171996341, 2.363683041, 8.773925739, 5.095984437,

0.990654757, 1.430573929, 1.321644154, 9.828725265,
8.280091589, 8.375926648),
15
with f

= 24.306209068. Six constraints (g

1
to g
6
) are active at this optimum. The result of the
GAAL method (with six decimal places of accuracy) is given below:
x
GAAL
=(2.171996, 2.363683, 8.773926, 5.095984, 0.990655,
1.430574, 1.321644, 9.828726, 8.280092, 8.375927),
with f
GAAL
= 24.306209. The best, median and worst objective values for 25 runs are 24.306209,
24.306209 and 24.306209, respectively. It is evident that the proposed method has performed
quite well in this problem as well. The corresponding Lagrangian multipliers are computed to be
u = (1.715999, 0.471251, 1.356905, 0.022114, 0.304300, 0.273330, 0, 0), thereby suggesting that
the rst six constraints are active at the GAAL solution.
3.7 Problem G: g08
This problem has two variables and two inequality constraints. The best-reported optimum
solution is given below:
x

= (1.227971353, 4.245373366),
with f

= 0.0958250414. The result of the GAAL method is given as follows:

x
GAAL
= (1.227971, 4.245373),
with f
GAAL
= 0.095825. The best, median and worst objective values of 25 runs are 0.095825,
0.095825 and 0.0, respectively. Eight out of the 25 runs are able to locate the optimum accu-
rately. Due to multi-modal nature of this problem, the performance of GAAL in this problem is
not as good as that in the previous problems.
3.8 Problem H: g09
This problem has seven variables and four inequality constraints. The best-known optimum (upto
nine decimal places of accuracy) is as follows:
x

=(2.330499351, 1.951372368, 0.477541399, 4.365726249,

0.624486959, 1.038130994, 1.594226678),
with f

= 680.630057374. Two constraints (g

1
and g
4
) are active at this optimum. The GAAL
obtained solution (obtained with six decimal places of accuracy) is shown below:
x
GAAL
=(2.330499, 1.951372, 0.477541, 4.365726,
0.624487, 1.038131, 1.594227),
with f
GAAL
= 680.630057. The best, median and worst function values were 680.630057, 680.630057
and 680.630057, respectively. The proposed method performed quite well in this problem. All 25
runs found the the optimum accurately upto six decimal places of accuracy. The corresponding
Lagrangian multipliers are found to be u = (1.166538, 0, 0, 4.276302), indicating that constraints
g
1
and g
4
are active at the obtained optimized solution.
3.9 Problem I: g10
This problem has eight variables and six inequality constraints. The best-known solution with
nine decimal places of accuracy is given as follows:
x

=(579.306685018, 1359.970678079, 5109.970657431, 182.017699630,

295.601173703, 217.982300369, 286.416525928, 395.601173703),
16
with f

= 7049.248020529. All six constraints are active at this optimum. The GAAL optimized
solution is given below:
x
GAAL
=(579.306688, 1359.970651, 5109.970682, 182.017700,
295.601173, 217.982300, 286.416527, 395.601173),
with f
GAAL
= 7049.248021. The best, median and worst function values were 7049.248021,
7049.248021 and 7049.248021, respectively, meaning that all 25 runs found the identical solution.
The corresponding Lagrangian multipliers are computed to be u = (1723.265940, 5764.135170,
5218.5992782, 0.008269, 0.010103, 0.010055). This problem has been reported to be a dicult
problem to solve due to non-uniform scaling of the constraints [10], as can be seen from the
problem formulation given in the appendix. The rst three constraints are normalized and the
remaining three are not. Interestingly, the GAAL method did not have any diculty in solving the
problem, as in GAAL each constraint is associated with a dierent multiplier
j
which is evolved
independently. The Lagrange multiplier vector rightly indicates that the rst three values take
a large u
j
value due to having a small constraint value and the last three constraints take small
u
j
values due to large constraint values. It is the product of Lagrange multiplier value u
j
and
constraint value g
j
that must be of the similar order to the objective function value in order
for the composite penalized function P() to provide equal importance to all constraints and the
objective function. GAAL is able to adjust all multipliers
j
accordingly.
3.10 Problem J: g12
This problem has three variables and nine inequality constraints. The best-known solution is
given as follows:
x

= (5, 5, 5),
with f

= 1. The GAAL optimized solution is as follows:

x
GAAL
= (5.143422, 4.800712, 5.047059),
with f
GAAL
= 0.999375. The best, median and worst function values are 0.999375, 0.999375
and 0.999375, respectively. All 25 runs of GAAL are found to lie within 0.1% of the best-known
optimum.
3.11 Problem K: g24
This problem has two variables and two inequality constraints. The best-known optimum is given
below:
x

= (2.329520197, 3.178493074),
with f

= 5.508013271. Both constraints are active at this optimum. The GAAL optimized
solution is given below:
x
GAAL
= (2.329520, 3.178493),
with f
GAAL
= 5.508013. The best, median and worst function values were 5.508013, 5.508013
and 5.508013, respectively. The proposed method performed quite well in this problem. All 25
runs found the the optimum accurately upto six decimal places of accuracy. The correspond-
ing Lagrangian multipliers are computed to be u = (0.348751, 0.705709), indicating that both
constraints are active at this solution.
17
4 Comparative Results
The GAAL results presented in the previous section were obtained by setting a termination
condition that checked objective value of two consecutive local-searched solutions in Step 2(d). If
the absolute dierence in objective values were less than or equal to 10
4
, the run was terminated.
Importantly, in these runs, the termination was not made based on a comparison with best-known
solutions. Despite, the GAAL method was able to converge very close to the correct optimum
solution in all problems and in almost all 25 runs. This indicates the robustness and eciency of
the proposed method.
In this section, we are interested in comparing the performance of our GAAL method with
some well-known evolutionary constraint handling methods. Three state-of-the-art methods are
chosen for this purpose. They are taken from references [25], [21] and [3]. The termination
condition used in these studies were slightly dierent. If a solution having an objective value
within 10
4
from the best-known objective value is found, a run is terminated. This requires one
to know the best-known solutions and thus the approach is applicable for solving test problems
only. Nevertheless, we compare our results (in which the knowledge of the best-known solution
is not utilized) with the existing results reported in [25], [21] and [3] in Table 1. It can be
observed that the GAAL achieves the best-known optimum with the set accuracy level in smaller
number of function evaluations than all three competing evolutionary methods in almost all
problems. For problem g08, Takahama and Sakais algorithm [21] performs the best and GAAL
performs the worst. For problem g02, only two out of 25 runs comes close to 0.1% of the best-
known objective value. Both problems g02 and g08 have a periodic and multimodal objective
function and as discussed elsewhere [10], the multi-modality causes diculty to penalty based
approaches. The three competing evolutionary methods used a dierential evolution (DE) which
uses a dierence between two variable vectors to create a child solution. Particularly, for periodic
objective functions in which multiple optima lie in a periodic manner on the search space, DE
based methods are expected to perform better. However, the superior performance of GAAL
in most other problems having a large number of variables, a large number of constraints, non-
uniform scaling of constraints and other complexities, remains as an interesting outcome of this
study.
Table 1: Comparison of function evaluations needed by GAAL and by other best-known evolu-
tionary algorithms.
Prob Zavala et al.[25] Takahama & Sakai[21] Brest[3] Proposed Method
BestMedian Worst BestMedian Worst BestMedian Worst BestMedian Worst
g01 80776 90343 96669 18594 19502 19971 51685 55211 57151 12617 15870 34717
g02 87419 93359 99654 108303 114347 129255 175090 226789253197 15849 38937 66785
g04 3147 103308110915 12771 13719 14466 56730 62506 67383 1861 3611 4111
g06 95944 109765130293 5037 5733 6243 31410 34586 37033 1005 2755 12355
g07 114709 138767208751 60873 67946 75569 184927 197901221866 10408 20219 30218
g08 2270 4282 5433 621 881 1173 1905 4044 4777 4605 100606100809
g09 94593 103857119718 19234 21080 21987 79296 89372 98062 2953 6450 8205
g10 109243 135735193426 87848 92807 107794 203851 220676264575 9371 10578 16175
g12 482 6158 9928 2901 4269 5620 364 6899 10424 1209 1705 1957
g18 97157 107690124217 46856 57910 60108 139131 169638191345 11461 14161 26306
g24 11081 18278633378 1959 2451 2739 9359 12844 14827 1005 1505 3911
Finally, we compare our proposed GAAL method with the classical AL method solved using
MATLABs fmincon routine. In this approach, each
j
is initialized to zero. Thereafter, the
penalized function (Equation 2) is formulated and solved using the fmincon routine. A random
18
solution is used as an initial solution. Thus, to make the comparison fair, the algorithm is run
25 times and each time starting with a dierent initial random solution. Based on the optimum
solution obtained for the rst penalized function, the multipliers are updated using Equation 3
and another optimization run is performed to solve the revised penalized function. This time,
the optimal solution obtained from the previous optimization of penalized function is used as
the initial solution. An identical termination condition (a solution having at most 10
4
dierence
between optimal objective values of two consecutive penalized functions) is used here. However, if
such a scenario is not achieved within a maximum of 5,000 iterations, the nal solution at the end
of 5,000 iteration is reported. Note that in such an event, the reported solution can be infeasible as
well. Table 2 shows the comparison of the results of the proposed GAAL method with the results
obtained from the above classical AL method. It can be observed that in only a few cases the
classical AL approach is able to nd a solution close to the best-known solution and surprisingly
in many cases ends up getting stuck to an infeasible solution. Only problem g09 is solved in 25
out of 25 runs. This table clearly indicates the usefulness of a more global and population-based
genetic algorithm as an ecient constrained optimization algorithm that can be used through the
augmented Lagrangian approach. The use of a seamless and self-adaptive penalty parameter and
multiplier update and an embedding of an occasional local search operation to provide convergence
property are all necessary ingredients that have helped the overall algorithm to work well on a
variety of dicult constrained optimization problems.
4.1 An Engineering Problem: Welded Beam Design
Finally, GAAL is applied to the well-known welded beam design problem presented in [7]. This
problem has four variables and ve inequality constraints. For a welded beam, the cost of fabri-
cation is required to be minimized subject to two stress constraints, one physical constraint, one
buckling constraint, and one deection constraint. The physical constraint states that two of the
variables (h and t) must be related as h b.
We set the population size as 50 and maximum generations as 500 for this problem. The
best-known optimum (with four decimal places of accuracy) for this problem is as follows:
(h, , t, b)

= (0.2444, 6.2187, 8.2915, 0.2444),

having a cost objective value of f

= 2.38117 units. The rst four constraints are active at this

optimum.
The best GAAL solution is found to be as follows:
(h, , t, b)
GAAL
= (0.244369, 6.218607, 8.291472, 0.244369),
with f
GAAL
= 2.381134 units an 0.001% improvement from the best-known solution. Despite
the introduction of this problem to classical optimization literature in eighties [19], to evolutionary
computation literature in nineties [5], and extensive studies made thereafter, the GAAL obtained
solution is better than the best-reported solution to this problem so far.
The best, median and worst function values over 25 runs are 2.381134, 2.381134 and 2.381134
units, respectively, thereby depicting the robustness of the proposed algorithm. The minimum,
median and maximum number of function evaluations required to achieve termination in all 25
runs are 3,909, 5,911 and 10,515, respectively. These numbers are small compared to that needed
in other evolutionary constrained optimization studies 320,000 needed in [7] and a median of
8,574 in a recent study [10]. The corresponding Lagrangian multipliers are reported as u =
(1.235694, 1.069852, 13.081068, 0.017638, 0), meaning that the rst four constraints are active at
this solution. Interestingly, the above GAAL solution requires that h = b, thereby making the
corresponding (third) constraint active at the GAAL solution. A large value of u
3
indicates that
the third constraint (h b) is most sensitive to the optimum solution, that is, a small change
19
Table 2: Comparison of function evaluations needed by GAAL and a classical AL method. Suc-
cessful run means result is within 0.1% of the best-known optimum and is feasible. Infeasible
results are shown in italics.
Prob. Best-known Proposed method, GAAL
optimum f

Function evaluations Succ.

Best Median Worst MinMedian Max runs
g01 -15 -15 -15 -12.453125 12617 15870 34717 24
g02 -0.803619 -0.803619 -0.803619 -0.744767 15849 38937 66785 2
g04-30665.538671 -30665.538672-30665.538672-30665.538672 1861 3611 4111 25
g06 -6961.813876 -6961.813876 -6961.813876 -6961.813876 1005 2755 12355 25
g07 24.306209 24.306209 24.306209 24.306209 10408 20219 30218 25
g08 -0.095825 -0.095825 -0.095825 0 4605 100606 100809 8
g09 680.630057 680.630057 680.630057 680.630057 2953 6450 8205 25
g10 7049.248021 7049.248021 7049.248021 7049.248021 9371 10578 16175 25
g12 -1 -0.999375 -0.999375 -0.999375 1209 1705 1957 25
g18 -0.866025 -0.866025 -0.866025 -0.674981 11461 14161 26306 24
g24 -5.508013 -5.508013 -5.508013 -5.508013 1005 1505 3911 25
Prob. Best-known Classical ALM
optimum f

Function evaluations Succ.

Best Median Worst MinMedian Max runs
g01 -15 -7.697937 -6 -6 225 529 1313 0
g02 -0.803619 -0.411793 -0.213328 -0.186381 455 1084 2247 0
g04-30665.538671 -32217.431037 -30665.541418-29083.524056 391 546 80381 6
g06 -6961.813876 -6961.813662 -6961.760172 -4518.090906 925 962 152087 17
g07 24.306209 24.304042 24.30512 24.313922 8106 160203 160203 1
g08 -0.095825 -58.385784 -0.025741 0.024911 89 89 102790 0
g09 680.630057 680.629056 680.659357 680.982856 2522 2920 53700 25
g10 7049.248021 2100 7049.312618 7049.493868 8125 191132779221 14
g12 -1 -1 -1 -0.25 198 30117 30738 1
g18 -0.866025 -0.695361 0 0 1815 2570 9740 0
g24 -5.508013 -5.508022 -5.507574 -3 16 145 100750 5
in this constraint will make a large sacrice in the optimal objective value. Thus, adhering to
dimensions h = b = 0.244369 inches is an important design goal for this problem. The fth
constraint involves a deection constraint which does not get satised with the equality sign, as
an arbitrary large threshold (0.25 inches) on allowable deection is chosen in the formulation. It
is clear from the results that not only GAAL is able to nd an improved solution, the method is
also reliable and computationally fast.
5 Conclusions
In this paper, we have proposed a hybrid and self-adaptive genetic algorithm that follows the
principle of augmented Lagrangian (AL) method in solving constrained optimization problems.
The proposed algorithm (GAAL) updates both the associated penalty parameter (which is usually
kept xed in a classical AL method) and multipliers for all constraints in a seamless manner. Oc-
casionally, the population-best solution is modied by applying a local search using a well-known
numerical optimization algorithm to provide convergence property to the overall algorithm. The
proposed GAAL procedure has shown to work better and more reliably than three competing
20
evolutionary approaches in most of the standard test problems of this study in terms of computa-
tional eorts. The obtained solutions are also found to be remarkably accurate (upto six decimal
places) to the best-known results. The method has also shown to work well on a practical design
problem. A comparison with the classical AL method has also revealed the importance of using
a genetic algorithm as an optimization method for constraint handling.
This method takes advantage of GAs population approach and classical optimization algo-
rithms convergence properties, and exploits the algorithmic advantage of the augmented La-
grangian approach. The extent of an optimization run for a xed value of multipliers is self-
adjusted and importantly the penalty parameter value, which is usually kept xed in a classical
AL approach, is also adjusted in a self-adaptive manner. A combination of all above has made
the overall GAAL method to be accurate, computationally fast, and reliable over multiple runs
on most problems of this study.
A by-product of the proposed methodology is that the Lagrange multipliers corresponding
to constraint functions have also been obtained without any additional eort, thereby giving us
information about the inuence and sensitivity of individual constraints on the optimized solution.
Despite some recent interests on nding Lagrange multipliers [22] for GA based approaches,
evolutionary constrained optimization researchers have not paid much attention towards this
important and pragmatic aspect of computing Lagrange multipliers. Evolutionary algorithms
have potential for this purpose and more EAs now must be devised for not only nding the
optimal solution, but also to compute Lagrange multipliers eciently. This paper remains as an
important step in this direction.
Acknowledgments
This study is supported by Academy of Finland under grant 133387.
References
[1] Amstutz, S.: Augmented lagrangian for cone constrained topology optimization. Computa-
tional Optimization and Applications 49(1), 101122 (2011)
[2] Birgin, E.G., Martnez, J.M.: Augmented lagrangian method with nonmonotone penalty
parameters for constrained optimization. Computational Optimization and Applications
(2011). DOI: 10.1007/s10589-011-9396-0
[3] Brest, J.: Constrained Real-Parameter Optimization with Self-Adaptive Dierential Evo-
lution, pp. 7394. springer (2009)
[4] Coello, C.A.C.: Treating objectives as constraints for single objective optimization. Engi-
neering Optimization 32(3), 275308 (2000)
[5] Deb, K.: Optimal design of a welded beam structure via genetic algorithms. AIAA Journal
29(11), 20132015 (1991)
[6] Deb, K.: Optimization for Engineering Design: Algorithms and Examples. New Delhi:
Prentice-Hall (1995)
[7] Deb, K.: An ecient constraint handling method for genetic algorithms. Computer Methods
in Applied Mechanics and Engineering 186(24), 311338 (2000)
[8] Deb, K., Agrawal, R.B.: Simulated binary crossover for continuous search space. Complex
Systems 9(2), 115148 (1995)
21
[9] Deb, K., Agrawal, S.: A niched-penalty approach for constraint handling in genetic algo-
rithms. In: Proceedings of the International Conference on Articial Neural Networks and
Genetic Algorithms (ICANNGA-99), pp. 235243. Springer-Verlag (1999)
[10] Deb, K., Datta, R.: A fast and accurate solution of constrained optimization problems using
a hybrid bi-objective and penalty function approach. In: Proceedings of the IEEE World
Congress on Computational Intelligence (WCCI-2010), pp. 165172 (2010)
[11] Homaifar, A., Lai, S.H.V., Qi, X.: Constrained optimization via genetic algorithms. Simula-
tion 62(4), 242254 (1994)
[12] Joines, J.A., Houck, C.R.: On the use of nonstationary penalty functions to solve nonlinear
constrained optimization problems with GAs. In: Proceedings of the International Conference
on Evolutionary Computation, pp. 579584 (1994)
[13] Kowalewski, A., Lasiecka, I., Sokolowski, J.: Sensitivity analysis of hyperbolic optimal control
problems. Computational Optimization and Applications (2011). DOI: 10.1007/s10589-010-
9375-x
[14] Liang, J.J., Runarsson, T.P., Mezura-Montes, E., Clerc, M., Suganthan, P.N.,
Coello, C.A.C., Deb, K.: Special session on constrained real-parameter optimization
(http://www.ntu.edu.sg/home/epnsugan/) (2006)
[15] Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Berlin:
Springer-Verlag (1992)
[16] Michalewicz, Z., Schoenauer, M.: Evolutionary algorithms for constrained parameter opti-
mization problems. Evolutionary Computation Journal 4(1), 132 (1996)
[17] Pillo, G.D., Liuzzi, G., Lucidi, S., Palagi, L.: A truncated newton method in an augmented
lagrangian framework for nonlinear programming. Computational Optimization and Appli-
cations 45(2), 311352 (2010)
[18] Rao, S.S.: Genetic algorithmic approach for multiobjective optimization of structures. In:
Proceedings of the ASME Annual Winter Meeting on Structures and Controls Optimization,
vol. 38, pp. 2938 (1993)
[19] Reklaitis, G.V., Ravindran, A., Ragsdell, K.M.: Engineering Optimization Methods and
Applications. New York : Wiley (1983)
[20] Srivastava, S., Deb, K.: A genetic algorithm based augmented lagrangian method for compu-
tationally fast constraint optimization. In: Proceedings of the First International Conference
on Swarm, Evolutionary and Memetic Computing (SEMCCO 2010), pp. 182189. Berlin:
Springer-Verlag (2010)
[21] Takahama, T., Sakai, S.: Constrained optimization by the constrained dierential evolution
with gradient-based mutation and feasible elites. In: 2006 IEEE Congress on Evolutionary
Computation, pp. 18. Piscatway: IEEE Press (2006)
[22] Tulshyan, R., Arora, R., Deb, K., Dutta, J.: Investigating ea solutions for approximate kkt
conditions for smooth problems. In: Proceedings of Genetic and Evolutionary Algorithms
Conference (GECCO-2010), pp. 689696 (2010)
[23] Volkwein, S.: Application of the augmented lagrangian-sqp method to optimal control prob-
lems for the stationary burgers equation. Computational Optimization and Applications
16(1), 5781 (2010)
22
[24] Y. Zhou Y. Li, J.H., Kang, L.: Multi-objective and mgg evolutionary algorithm for con-
strained optimization. In: Proceedings of Congress on Evolutionary Computation, pp. 15
(2003)
[25] Zavala, A.E.M., Aguirre, A.H., Diharce, E.R.V.: Continuous Constrained Optimization with
Dynamic Tolerance Using the COPSO Algorithm, pp. 124. Heidelberg, Germany: Springer
(2009)
[26] Zhou, Y.Y., Yang, X.Q.: Augmented lagrangian functions for constrained optimization prob-
lems. Journal of Global Optimization (2010). DOI: 10.1007/s10898-011-9688-z
A Problem Formulations
These problems are taken from [14].
A.1 Problem g01
The problem is given as follows:
min. f(x) = 5

4
i=1
x
i
5

4
i=1
x
2
i
+ 5

13
i=5
x
i
,
s.t. g
1
(x) 2x
1
+ 2x
2
+ x
10
+ x
11
10 0,
g
2
(x) 2x
1
+ 2x
3
+ x
10
+ x
12
10 0,
g
3
(x) 2x
2
+ 2x
3
+ x
11
+ x
12
10 0,
g
4
(x) 8x
1
+ x
10
0,
g
5
(x) 8x
2
+ x
11
0,
g
6
(x) 8x
3
+ x
12
0,
g
7
(x) 2x
4
x
5
+ x
10
0,
g
8
(x) 2x
6
x
7
+ x
11
0,
g
9
(x) 2x
8
x
9
+ x
12
0,
(10)
where 0 x
i
1 for i = 1, 2, . . . , 9, 0 x
i
100 for i = 10, 11, 12, and 0 x
13
1.
A.2 Problem g02
The problem is given as follows:
min. f(x) =

P
20
i=1
cos
4
(x
i
)2
20
i=1
cos
2
(x
i
)

P
20
i=1
ix
2
i

,
s.t. g
1
(x) 0.75
20
i=1
x
i
0,
g
2
(x)

20
i=1
x
i
150 0,
0 x
i
10, i = 1, 2, . . . , 20.
(11)
23
A.3 Problem g04
The problem is given as follows:
min. f(x) = 5.3578547x
2
3
+ 0.8356891x
1
x
5
+ 37.293239x
1
40792.141,
s.t. g
1
(x) 85.334407 + 0.0056858x
2
x
5
+ 0.0006262x
1
x
4
0.0022053x
3
x
5
92 0,
g
2
(x) 85.334407 0.0056858x
2
x
5
0.0006262x
1
x
4
+0.0022053x
3
x
5
0,
g
3
(x) 80.51249 + 0.0071317x
2
x
5
+ 0.0029955x
1
x
2
+0.0021813x
2
3
110 0,
g
4
(x) 80.51249 0.0071317x
2
x
5
0.0029955x
1
x
2
0.0021813x
2
3
+ 90 0,
g
5
(x) 9.300961 + 0.0047026x
3
x
5
+ 0.0012547x
1
x
3
+0.0019085x
3
x
4
25 0,
g
6
(x) 9.300961 0.0047026x
3
x
5
0.0012547x
1
x
3
0.0019085x
3
x
4
+ 20 0,
78 x
1
102, 33 x
2
45, 27 (x
3
, x
4
, x
5
) 45.
(12)
A.4 Problem g06
The problem is given as follows:
min. f(x) = (x
1
10)
3
+ (x
2
20)
3
,
s. t. g
1
(x) (x
1
5)
2
(x
2
5)
2
+ 100 0,
g
2
(x) (x
1
6)
2
+ (x
2
5)
2
82.81 0,
13 x
1
100, 0 x
2
100.
(13)
A.5 Problem g07
The problem is given as follows:
min. f(x) = x
2
1
+ x
2
2
+ x
1
x
2
14x
1
16x
2
+ (x
3
10)
2
+4(x
4
5)
2
+ (x
5
3)
2
+ 2(x
6
1)
2
+ 5x
2
7
+ 7(x
8
11)
2
+2(x
9
10)
2
+ (x
10
7)
2
+ 45,
s.t.
g
1
(x) 105 + 4x
1
+ 5x
2
3x
7
+ 9x
8
0,
g
2
(x) 10x
1
8x
2
17x
7
+ 2x
8
0,
g
3
(x) 8x
1
+ 2x
2
+ 5x
9
2x
10
12 0,
g
4
(x) 3(x
1
2)
2
+ 4(x
2
3)
2
+ 2x
3
2
7x
4
120 0,
g
5
(x) 5x
2
1
+ 8x
2
+ (x
3
6)
2
2x
4
40 0,
g
6
(x) x
2
1
+ 2(x
2
2)
2
2x
1
x
2
+ 14x
5
6x
6
0,
g
7
(x) 0.5(x
1
8)
2
+ 2(x
2
4)
2
+ 3x
2
5
x
6
30 0,
g
8
(x) 3x
1
+ 6x
2
+ 12(x
9
8)
2
7x
10
0,
10 x
i
10, i = 1, 2, . . . , 10.
(14)
A.6 Problem g08
The problem is given as follows:
min. f(x) =
sin
3
(2x
1
) sin(2x
2
)
x
3
1
(x
1
+x
2
)
,
s.t. g
1
(x) x
2
1
x
2
+ 1 0,
g
2
(x) 1 x
1
+ (x
2
4)
2
0,
0 x
i
10, i = 1, 2.
(15)
24
A.7 Problem g09
The problem is given as follows:
min. f(x) = (x
1
10)
2
+ 5(x
2
12)
2
+ x
4
3
+ 3(x
4
11)
2
+10x
6
5
+ 7x
2
6
+ x
4
7
4x
6
x
7
10x
6
8x
7
,
s.t. g
1
(x) 127 + 2x
2
1
+ 3x
4
2
+ x
3
+ 4x
2
4
+ 5x
5
0,
g
2
(x) 282 + 7x
1
+ 3x
2
+ 10x
3
2
+ x
4
x
5
0,
g
3
(x) 196 + 23x
1
+ x
2
2
+ 6x
2
6
8x
7
0,
g
4
(x) 4x
2
1
+ x
2
2
3x
1
x
2
+ 2x
2
3
+ 5x
6
11x
7
0,
10 x
i
10, i = 1, 2, . . . , 7.
(16)
A.8 Problem g10
The problem is given as follows:
min. f(x) = x
1
+ x
2
+ x
3
,
s.t. g
1
(x) 1 + 0.0025(x
4
+ x
6
) 0,
g
2
(x) 1 + 0.0025(x
5
+ x
7
x
4
) 0,
g
3
(x) 1 + 0.01(x
8
x
5
) 0,
g
4
(x) x
1
x
6
+ 833.33252x
4
+ 100x
1
83333.333 0,
g
5
(x) x
2
x
7
+ 1250x
5
+ x
2
x
4
1250x
4
0,
g
6
(x) x
3
x
8
+ 1250000 + x
3
x
5
2500x
5
0,
100 x
1
10000, 1000 (x
2
, x
3
) 10000,
10 (x
4
, . . . , x
8
) 1000.
(17)
A.9 Problem g12
The problem is given as follows:
min. f(x) = (100 (x
1
5)
2
(x
2
5)
2
(x
3
5)
2
)/100,
s.t. g
i
(x) (x
1
p
i
)
2
+ (x
2
q
i
)
2
+ (x
3
r
i
)
2
0.0625 0,
i = 1, 2, . . . , 9,
0 x
i
100, i = 1, 2, 3.
(18)
A.10 Problem g18
The problem is given as follows:
min. f(x) = 0.5 (x
1
x
4
x
2
x
3
+ x
3
x
9
x
5
x
9
+ x
5
x
8
x
6
x
7
) ,
s.t. g
1
(x) x
2
3
+ x
2
4
1 0,
g
2
(x) x
2
9
1 0,
g
3
(x) x
2
5
+ x
2
6
1 0,
g
4
(x) x
2
1
+ (x
2
x
9
)
2
1 0,
g
5
(x) (x
1
x
5
)
2
+ (x
2
x
6
) 0,
g
6
(x) (x
1
x
7
)
2
+ (x
2
x
8
) 0,
g
7
(x) (x
3
x
5
)2 + (x
4
x
6
)
2
10,
g
8
(x) (x
3
x
7
)
2
+ (x
4
x
8
)
2
0,
g
9
(x) x
2
+ (x
8
x
9
)
2
1 0,
g
10
(x) x
2
x
3
x
1
x
4
0,
g
11
(x) x
3
x
9
0,
g
12
(x) x
5
x
9
0,
g
13
(x) x
6
x
7
x
5
x
8
0,
10 x
i
10, i = 1, 2, . . . , 8,
0 x
9
20.
(19)
25
A.11 Problem g24
The problem is given as follows:
min. f(x) = x
1
x
2
,
s.t. g
1
(x) 2x
4
1
+ 8x
3
1
8x
2
1
+ x
2
2 0,
g
2
(x) 4x
4
1
+ 32x
3
1
88x
2
1
+ 96x
1
+ x2 36 0,
0 x
1
3, 0 x
2
4.
(20)
A.12 Problem Weld
The problem is given as follows (x = (h, l, t, b)
T
) [19, 5]:
min. f
1
(x) = 1.10471h
2
l + 0.04811tb(14.0 + l),
s. t. g
1
(x) 13, 600 (x) 0,
g
2
(x) 30, 000 (x) 0,
g
3
(x) b h 0,
g
4
(x) P
c
(x) 6, 000 0,
g
5
(x) 0.25 (x) 0,
0.125 h, b 5,
0.1 l, t 10,
(21)
where,
(x) =
_
(

)
2
+ (

)
2
+ (l

)/
_
0.25(l
2
+ (h + t)
2
),

=
6, 000

2hl
,

=
6, 000(14 + 0.5l)
_
0.25(l
2
+ (h + t)
2
)
2[0.707hl(l
2
/12 + 0.25(h + t)
2
)]
,
(x) =
504, 000
t
2
b
, (x) =
2.1952
t
3
b
,
P
c
(x) = 64, 746.022(1 0.0282346t)tb
3
.
26