Practical Optimization Using Evolutionary Methods

Kalyanmoy Deb
Kanpur Genetic Algorithms Laboratory (KanGAL)
Department of Mechanical Engineering
Indian Institute of Technology Kanpur
Kanpur, PIN 208016, India
deb@iitk.ac.in
http://www.iitk.ac.in/kangal
KanGAL Report Number 2005008

Abstract

Many real-world problem solving tasks, including

CFD problems, involve posing and solving optimization problems, which are usually non-linear,
non-differentiable, multi-dimensional, multi-modal,
stochastic, and computationally time-consuming. In
this paper, we discuss a number of such practical
problems which are, in essence, optimization problems and review the classical optimization methods
to show that they are not adequate in solving such
demanding tasks. On the other hand, in the past
couple of decades, new yet practical optimization
methods, based on natural evolutionary techniques,
are increasingly found to be useful in meeting the
challenges. These methods are population based,
stochastic, and flexible, thereby providing an ideal
platform to modify them to suit to solve most optimization problems. The remainder of the paper illustrates the working principles of such evolutionary optimization methods and presents some results in support of their efficacy. The breadth of their application domain and ease and efficiency of their working
make evolutionary optimization methods promising
for taking up the challenges offered by the vagaries
of various practical optimization problems.

Optimization is an activity which does not belong to

any particular discipline and is routinely used in almost all fields of science, engineering and commerce.
Chambers dictionary describes optimization as an act
of making the most or best of anything. Theoretically speaking, performing an optimization task in
a problem means finding the most or best suitable
solution of the problem. Mathematical optimization
studies spend a great deal of effort in trying to describe the properties of such an ideal solution. Engineering or practical optimization studies, on the other
hand, thrive to look for a solution which is as similar to such an ideal solution as possible. Although
the ideal optimal solution is desired, the restrictions
on computing power and time often make the practitioners happy with an approximate solution.
Serious studies on practical optimization begun as
early as the second World war, when the need for efficient deployment and resource allocation of military
personnel and accessories became important. Most
development in the so-called classical optimization
field was made by developing step-by-step procedures
for solving a particular type of an optimization problem. Often fundamental ideas from geometry and
calculus were borrowed to reach the optimum in an
iterative manner. Such optimization procedures have

INTRODUCTION

enjoyed a good 50 years of research and applications and are still going strong. However, around the
middle of eighties, completely unorthodox and lessmathematical yet intriguing optimization procedures
have been suggested mostly by computer scientists.
It is not surprising because these non-traditional optimization methods exploit the fast and distributed
computing machines which are getting increasingly
available and affordable like slide-rules of sixties.
In this paper, we focus on one such non-traditional
optimization method which takes the lions share of
all non-traditional optimization methods. This socalled evolutionary optimization (EO) mimics the
natural evolutionary principles on randomly-picked
solutions from the search space of the problem and iteratively progresses towards the optimum point. Natures ruthless selective advantage to fittest individuals and creation of new and more fit individuals
using recombinative and mutative genetic processing with generations is well-mimicked artificially in
a computer algorithm to be played on a search space
where good and bad solutions to the underlying problem coexist. The task of an evolutionary optimization algorithm is then to avoid the bad solutions in
the search space, take clues from good solutions and
eventually reach close to the best solution, similar to
the genetic processing in natural systems.
Like the existence of various natural evolutionary
principles applied to lower and higher level species,
researchers have developed different kinds of evolutionary plans resulting in a gamut of evolutionary
algorithms (EAs) some emphasizing the recombination procedure, some emphasizing the mutation operation and some using a niching strategy, whereas
some using a mating restriction strategy. In this article, we only give a description of a popular approach
genetic algorithm (GA). Other approaches are also
well-established and can be found from the EA literature [42, 1, 19, 29, 44, 27].
In the remainder of the paper, we describe an optimization problem and cite a number of commonlyused practical problems, including a number of CFD
problems, which are in essence optimization problems. A clear look at the properties of such practical optimization problems and a description of the
working principles of classical optimization methods

reveal that completely different optimization procedures are in order for such problem solving. Thereafter, the evolutionary optimization procedure is described and its suitability in meeting the challenges
offered by various practical optimization problems is
demonstrated. The final section concludes the study.

AN
OPTIMIZATION
PROBLEM AND ITS NOTATIONS

Throughout this paper, we describe procedures for

finding the optimum solution of a problem of the following type:
Minimize f (x),
Subject to gj (x) 0,
hk (x) = 0,
xmin x xmax .

(1)

Here, f (x) is the objective function (of n variables)

which is to be minimized. A maximization problem
can be converted to a minimization problem by multiplying the function by 1. The inequality constraints
g and equality constraints h demand a solution x to
be feasible only if all constraints are satisfied. Many
problems also require that the search is restricted
within a prescribed hypervolume defined by lower
and upper bound on each variable. This region is
called the search space and the set of all feasible solutions is called feasible space. Usually, there exists
at least one solution x in the feasible space which
corresponds to the minimum objective value. This
solution is called the optimum solution.
Thus, the task in an optimization process is to start
from one or a few random solutions in the search
space and utilize the function and constraint functions to drive its search towards the feasible region
and finally reach near the optimum solution by exploring as small as a set of solutions as possible.

SCOPE OF OPTIMIZATION IN PRACTICE

Optimal control: In this activity, control parameters, as functions of time, distance, etc., are
decision variables. The objective functions are
usually some estimates computed at the end of
Many researchers and practitioners may not know,
process, such as the quality of end-product, timebut they often either use or required to use an opaveraged
drag or lift, and stability or some other
timization method. For example, in fitting a linear
performance
measure across the entire process.
relationship y = mx + c between an input parameter
For
example,
in the design of a varying nozzle
x and output parameter y through a set of n data
shape
for
achieving
minimum average drag from
points, we often almost blindly use the following relaunch
to
full
throttle,
parameters describing the
lationships:
shape
of
the
nozzle
as
a
function of time are the
P
P P
decision variables. To evaluate a particular solun xy x y
P
P
,
(2)
m =
tion, the CFD system must have to be solved in
n x2 ( x)2
P
P
a
time sequence from launch to full throttle and
ym x
c =
.
(3)
the drag experienced at every time step must
n
have to be recorded for the computation of the
It is interesting to know that the above relationaverage drag coefficient for the entire flight. Staship has been derived by solving a unconstrained
bility or other performance indicators must also
quadratic optimization problem of minimizing the
be computed and recorded at every time step for
overall vertical error between the actual and the prechecking if the system is safe and perform satisdicted output values. In this section, we briefly disfactorily over the entire flight.
cuss different practical problem solving tasks in which
Modeling: Often in practice, systems involve coman optimization procedure is usually used.
plex processes which are difficult to express by
Optimal design: Instead of arriving at a solution
using exact mathematical relationships. Howwhich is simply a functionally satisfactory one,
ever, in such cases, either through pilot case
an optimization technique can be used to find
studies or through actual plant operation, nuan optimal design which will minimize or maximerous input-output data are available. To
mize a design goal. This activity of optimization
understand the system better and to improve
probably takes the lions share of all optimizathe system, it is necessary to find the relationtion activities and is most routinely used. Here
ships between input and output parameters in
the decision variables can be dimensions, shapes,
a process known as modeling. Such an opmaterials etc., which describe a design and obtimization task involves minimizing the error
jective function can be cost of production, enbetween the predicted output obtained by the
ergy consumption, drag, lift, reliability, stability
developed model and actual output. To commargin etc. For example, in the optimal design
pute the predicted output, a mathematical paof an airfoil shape for minimum drag (objective
rameterized model can be assumed based on
function), the shape of an airfoil can be represome information about the system, such as
sented by a few control parameters which can
y = a1 exp(a2 x), where a1 and a2 are the pabe considered as the decision variables. A minirameters to be optimized for modeling the exmum limit on the lift can be kept as a constraint
ponential relationship between the input x and
and the airfoil area can be bounded within ceroutput y. If a mathematical relationship is not
tain limits. Such a task will not only produce an
known at all, a relationship can be found by a
airfoil shape providing the minimum desired lift
sophisticated optimization method (such as geand area, but will also cause as small a drag as
netic programming method [29]) or by using an
possible.
artificial neural network.
3

Scheduling: Many practical problems involve finding a permutation which causes certain objectives optimum. In these problems, the sequence
of operations are of importance such as the absolute location of the operation on the permutation. For example, in a machining sequencing problem, it is important to know in what
order a job will flow from one machine to the
next, instead of knowing when exactly a job has
to be made on a particular machine. Such optimization tasks appear in time-tabling, planning, resource allocation problems and others,
and are usually known as combinatorial optimization problems.

smallest size classifiers which would be able correctly classify most samples of the data set into
various classes. Such problems often arise in bioinformatics problems [15] and can also be prevalent in other data-driven engineering tasks. In
a CFD problem such a classification task should
be able to find classifiers by which an investigation of a flow pattern should reveal the type of
flow (laminar or turbulent) or type of fluid or
kind of geometry associated with the problem.

Machine learning: In the age of automation, many

real-world systems are equipped with automated
and intelligent subsystems which can make decisions and adjust the system optimally, as and
Prediction and forecasting: Many time-varying
when an unforeseen scenario happens. In the
real-world problems are often periodic and predesign of such intelligent subsystems, often madictable. Past data for such problems can be
chine learning techniques involving different softanalyzed for finding the nature of periodicity so
computing techniques and artificial intelligence
that a better understanding of the problem can
techniques are combined. To make such a sysbe achieved. Using such a model, future foretem operate with a minimum change or with
casts can also be made judiciously. Although
minimum energy requirement, an optimization
such applications are plenty in financial domain,
procedure is needed. Since such subsystems are
periodicities in fluidic systems, such as repeated
to be used on-line and since on-line optimization
wake formation of a particular type and its preis a difficult proposition due to time restrictions,
diction of where and when will it reappear, are
such problems are often posed as an offline optialso of importance.
mization problem and solved by trying them on
a number of synthetic (or real) scenarios [32, 33].
Data mining: A major task in any real-world
In such optimization tasks, an optimal rule base
problem solving today is to analyze multiis learned which works the best on the chosen
dimensional data and discover the hidden usetest scenarios. It is then hypothesized that such
ful information they carry. This is by no means
an optimal rule base will also work well in real
is an easy task, simply because it is not known
scenarios. To make the optimal rule base reliable
what kind of information they would carry and
in real scenarios, such an optimization procedure
in most problems it is not even known what kind
can be used repeatedly with the new scenarios
of information one should look for. A major
and new optimal rule base can be learned.
task in such problems is to cluster functionally
similar data together and functionally dissimilar
data in separate clusters. Once the entire data 3.1 Properties of Practical Optimizaset is clustered, useful properties of iso-cluster
tion Problems
data can be deciphered for a better understandBased on the above discussion, we observe that the
ing of the problem. The clustering task is an oppractical optimization problems usually have the foltimization problem in which the objective is usulowing properties:
ally to maximize inter-cluster distance between
any pair of clusters and simultaneously minimize
1. They are non-smooth problems having their obtheir intra-cluster distances. Another task often
jectives and constraints are most likely to be
required to be performed is the identification of
non-differentiable and discontinuous.
4












Search
Space






























 

 

 

 Feasible
 

 







































































































































































 


 


 


 


 




























































































































 

 

 

 

 

 











































































































































































 

 

 

 

 

 
































































































































 

 

 

 

 

 














































































































Optimum




















































 

 

 

 

 

 











































































































































































































































































 

 

 

 

 

 



























































































































































































 

 

 

 

 

 






































































































 

























 

 
 

 
 

 

























































Initial
Guess















 

























 


 


 


 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

2. Often, the decision variables are discrete making

the search space discrete as well.
3. The problems may have mixed types (real, discrete, Boolean, permutation, etc.) of variables.
4. They may have highly non-linear objective and
constraint functions due to complicated relationships and equations which the decision variables
must form and satisfy. This makes the problems
non-linear optimization problems.
5. There are uncertainties associated with decision
variables, due to which the the true optimum
solution may not of much importance to a practitioner.

Constraints

6. The objective and constraint functions may also

be noisy and non-deterministic.

Figure 1: Most classical methods use a point-by-point

approach.

7. The evaluation of objective and constraint functions is computationally expensive.

Classical search and optimization methods can be

classified into two distinct groups mostly in the way
the directions are chosen: Direct and gradient-based
methods [6, 35, 38]. In direct methods, only objective function and constraints are used to guide the
search strategy, whereas gradient-based methods use
the first and/or second-order derivatives of the objective function and/or constraints to guide the search
process. Since derivative information is not used,
the direct search methods are usually slow, requiring
many function evaluations for convergence. For the
same reason, they can be applied to many problems
without a major change of the algorithm. On the
other hand, gradient-based methods quickly converge
to an optimal solution, but are not efficient in nondifferentiable or discontinuous problems. In addition,
there are some common difficulties with most of the
traditional direct and gradient-based techniques:

8. The problems give rise to multiple optimal solutions, of which some are globally best and many
others are locally optimal.
9. The problems involve multiple conflicting objectives, for which no one solution is best with respect to all chosen objectives.

CLASSICAL
OPTIMIZATION METHODS

Most classical point-by-point algorithms use a deterministic procedure for approaching the optimum solution. Such algorithms start from a random guess
solution. Thereafter, based on a pre-specified transition rule, the algorithm suggests a search direction,
which is often arrived at by considering local information. A one-dimensional search is then performed
along the search direction to find the best solution.
This best solution becomes the new solution and the
above procedure is continued for a number of times.
Figure 1 illustrates this procedure. Algorithms vary
mostly in the way the search directions are defined
at each intermediate solution.

Convergence to an optimal solution depends on

the chosen initial solution.
Most algorithms are prone to get stuck to a suboptimal solution.
An algorithm efficient in solving one problem
5

may not be efficient in solving a different problem.

more complicated objective function by means of natural genetics and natural selection than the search
and optimization problems we are interested in solving in practice. Thus, it is not surprising that the
computerized evolutionary optimization (EO) algorithm is not as complex as the natural genetics and
selection procedures, rather it is an abstraction of
the complex natural evolutionary process. Although
an EO is a simple abstraction, it is robust and has
been found to solve various search and optimization
problems of science, engineering, and commerce.

Algorithms are not efficient in handling problems having discrete variables or highly nonlinear and many constraints.
Algorithms cannot be efficiently used on a parallel computer.

MOTIVATION FROM NATURE AND EVOLUTION5.1 Evolutionary Optimization

ARY OPTIMIZATION

The idea of using evolutionary principles to constitute a computerized optimization algorithm was suggested by a number of researchers located in geographically distant places across the globe. The field
now known as genetic algorithms was originated by
John Holland of University of Michigan [24], a contemporary field evolution strategy was originated by
Ingo Rechenberg and Hans-Paul Schwefel of Technical University of Berlin [37, 41], the field of evolutionary programming was originated by Larry Fogel
[20], and others. Figure 2 provides an overview of
computational intelligence and its components. Also,
different subfields of evolutionary computing are also
shown. In this paper, we mainly discuss the principles of a genetic algorithm and its use in different
optimization problem solving.
Genetic algorithm (GA) is an iterative optimization procedure. Instead of working with a single solution in each iteration, a GA works with a number of
solutions (collectively known as a population) in each
iteration. A flowchart of the working principle of a
simple GA is shown in Figure 3. In the absence of
any knowledge of the problem domain, a GA begins
its search from a random population of solutions. If
a termination criterion is not satisfied, three different operators reproduction and variation operators
(crossover and mutation, and others) are applied to
update the population of solutions. One iteration of
these operators is known as a generation in the parlance of GAs. Since the representation of a solution
in a GA is similar to a natural chromosome and GA
operators are similar to genetic operators, the above

It is commonly believed that the main driving principle behind the natural evolutionary process is the
Darwins survival-of-the-fittest principle [3, 18]. In
most scenarios, nature ruthlessly follows two simple
principles:
1. If by genetic processing an above-average offspring is created, it usually survives longer than
an average individual and thus have more opportunities to produce offspring having some of its
traits than an average individual.
2. If, on the other hand, a below-average offspring
is created, it usually does not survive longer and
thus gets eliminated quickly from the population.
The principle of emphasizing good solutions and eliminating bad solutions seems to dovetail well with desired properties of a good optimization algorithm.
But one may wonder about the real connection between an optimization procedure and natural evolution! Has the natural evolutionary process tried
to maximize a utility function of some sort? Truly
speaking, one can imagine a number of such functions
which the nature may have been thriving to maximize: life span of a species, quality of life of a species,
physical growth, and others. However, any of these
functions is non-stationary in nature and largely depends on the evolution of other related species. Thus,
in essence, the nature has been optimizing a much

Computational

Begin

Intelligence

Step 1:
Fuzzy
Systems

Evolutionary

Neural
Networks

Algorithms

Representation scheme
Step 2:

Initialize Population
Genetic
Algorithms

Evolution
Strategies

Evolutionary
Programming

Genetic
Programming

t=0
Step 3:

Differential
Evolution

Particle
Swarm

Evaluation

Evolvable
Hardware

Cond?

Figure 2: Computational intelligence and evolutionary computation.

Yes
t=t+1

procedure is called a genetic algorithm.

5.1.1

No
Step 4:

Reproduction
Step 5:

Stop

Variation

Step 1: Representation of a solution

Representation of a solution describing the problem

is an important first step in a GA. The solution vector x can be represented as a vector of real numbers,
discrete numbers, a permutation of entities or others or a combination, as suitable to the underlying
problem. If a problem demands mixed real and discrete variables, a GA allows such a representation of
a solution.

Figure 3: A flowchart of working principle of a genetic

algorithm.

5.1.2

5.1.3

any suitable portion of the search space. In difficult problems with a known good solutions, the population can be created around the good solution by
randomly perturbing it [5].

Step 2: Initialization of a population of

solutions

Step 3: Evaluation of a solution

In this step, every solution is evaluated and a fitness

value
is assigned to the solution. This is where the soUsually, a set of random solutions (of size N ) are ini(L)
lution
is checked for its feasibility (by computing and
tialized in a pre-defined search space (xi xi
checking constraint functions gj and hk ). By simply
(U )
xi ). However, it is not necessary that the subseassigning a suitable fitness measure, a feasible can be
quent GA operations are confined to create solutions
given more importance over an infeasible solution or a
in the above range. To make sure that initial popubetter feasible solution can be given more importance
lation is well-distributed, any space-filling or Latinover a worse feasible solution. This is also the place
hypercube method can also be used. It is also not
where a single-objective GA dealing with a single obnecessary that the initial population is created ranjective function can be converted to a multi-objective
domly on the search space. If some problem inforGA in which multiple conflicting objectives can be
mation is available (such as knowledge about good
dealt with. In most practical optimization problems,
solutions), a biased distribution can be created in
this is also the most time-consuming step. Hence,

place-holders to position the i-th component. We illustrate this coding procedure by using an example
having six components (a to f) using six integers (0
to 5). Let us say that we have a string

any effort to complete this step quickly (either by using distributed computers or approximately) would
provide a substantial saving in the overall procedure.
The evaluation step requires to first decipher
the solution vector from the chosen representation
scheme. If the variables are represented directly as
real or discrete numbers, they are already directly
available. However, if a binary substring is used to
represent some discrete variables, first the exact variable value must be computed from the substring. For
example, the following is a string, representing n variables:
11010
010 . . . 0010
| {z } 1001001
| {z }
| {z } |{z}

(0 0 2 1 3 2)

and would like to decipher the corresponding permutation. The second component (i = 2) has two places
to be positioned (i) before the component a or (ii)
after the component a. The first case is denoted by a
0 and the second case is denoted by a 1 in the string.
Since the place-holder value for i = 2 (component b)
is zero in the above string, the component b appears
x3
x1
x2
xn
before the component a. Similarly, with the first two
The i-th problem variable is coded in a binary sub- components placed as (b a), there are three placestring of length `i , so that the total number of al- holders for the component c (i) before b (with a
ternatives allowed in that variable is 2`i . The lower value 0), (ii) between b and a (with a value 1), and
bound solution xmin
is represented by the solution (iii) after a (with a value 2). Since the string has
i
(00. . .0) and the upper bound solution xmax
is repre- a value 2 in the third place, component c is placed
i
sented by the solution (11. . .1). Any other substring after a. Continuing in this fashion, we obtain the
si decodes to a solution xi as follows:
permutation
(b d f a e c)
xmax
xmin
min
i
i
DV(si ),
(4)
xi = x i +
corresponding to the above string representation.
2 `i 1
The advantage of this coding is that simple crossover
where DV(si ) is the decoded value of the substring and mutation operators (to be discussed later) can be
si . Such a representation of discrete variables (even applied on the coding to create valid permutations.
real-parameter variables) has a traditional root and
also nature root. A binary string resembles a chro5.1.4 Step 4: Reproduction operator
mosome comprising of a number of genes taking one
of two values one or zero. Treating these strings Reproduction (or selection) is usually the first opas individuals, we can then think of mimicking natu- erator applied to the population. Reproduction seral crossover and mutation operators similar to their lects good strings in a population and forms a mating pool. The essential idea is to emphasize abovenatural counterparts.
To represent a permutation (of n integers), an average strings in the population. The so-called biinnovative coded representation procedure can be nary tournament reproduction operator picks two soadopted so that simple genetic operators can be ap- lutions at random from the population and the better
plied on the coding. In the proposed representation of the two is chosen according to its fitness value. Al[13], at the i-th position from the left of the string, an though a deterministic procedure can be chosen for
integer between zero and (i1) is allowed. The string this purpose (for example, the best 50% population
is decoded to obtain the permutation as follows. The members can be duplicated), usually a stochastic proi-th position denotes the placement of component i cedure is adopted in an EA to reduce the chance of
in the permutation. The decoding starts from the getting trapped into a local optimal solution.
left-most position and proceeds serially towards right.
While decoding the i-th position, the first (i1) components are already placed, thereby providing with i
8

5.1.5

to a 0 and vice versa, as happened in the fourth bit

of the following example:

Step 5: Variation operators

During the selection operator, good population members are emphasized at the expense of bad population members, but no new solution is created. The
purpose of variation operators is just to create new
and hopefully improved solutions by using the mating pool. A number of successive operations can be
used here. However, there are two main operators
which are mostly used.
In a crossover operator, two solutions are picked
from the mating pool at random and an information exchange is made between the solutions to create one or more offspring solutions. In a single-point
crossover operator applied to binary strings, both
strings are cut at an arbitrary place and the rightside portion of both strings are swapped among themselves to create two new strings:

Usually, a small value of pm 1/` or 1/n (n is the

number of variables) is used. Once again, the mutation operation uses random numbers but it is not an
entirely random process, as from a particular string
it is not possible to move to any other string in one
mutation event. Mutation uses a biased distribution
to be able to move to a solution close to the original
solution. The basic difference between the crossover
and mutation operations is that in the former more
than one parent solutions are needed, whereas in the
later only one solution is used directly or indirectly.
After reproduction and variation operators are applied to the whole population, one generation of a
GA is completed. These operators are simple and
0 0 1 1 1
0 0 0 0 0
straightforward. Reproduction operator selects good

1 1 1 1 1
1 1 0 0 0
strings and crossover operator recombines good substrings from two good strings together to hopefully
It is true that every crossover between any two soform a better substring. Mutation operator alters a
lutions from the new population is not likely to find
string locally to hopefully create a better string. Even
offspring better than both parent solutions, but the
though none of these claims are guaranteed and/or
chance of creating better solutions is far better than
tested while creating a new population of strings, it
random [21]. This is because the parent strings beis expected that if bad strings are created they will be
ing crossed are not any two arbitrary random strings.
eliminated by the reproduction operator in the next
These strings have survived tournaments played with
generation and if good strings are created, they will
other solutions during the earlier reproduction phase.
be retained and emphasized.
Thus, they are expected to have some good bit comAlthough the above illustrative discussion on the
binations in their string representations. Since, a
working of a genetic algorithm may seem to be a
single-point crossover on a pair of parent strings can
computer-savvy, nature-hackers game-play, there exonly create ` different string pairs (instead of all 2`1
ists rigorous mathematical studies where a complete
possible string-pairs) with bit combinations from eiprocessing of a finite population of solutions under
ther strings, the created offspring are also likely to be
GA operators is modeled by using Markov chains [45],
good strings. To reduce the chance of losing too many
using statistical mechanics approaches [34], and usgood strings by this process, usually a pair of soluing approximate analysis [22]. On different classes of
tions are participated in a crossover operator with
functions, the recent dynamical systems analysis [46]
a crossover probability, pc . Usually, a large value
treating a GA as a random heuristic search reveals
within [0.7, 1] is chosen for this parameter. A value
the complex dynamical behavior of a binary-coded
of pc = 0.8 means that 80% population members parGA with many meta-stable attractors. The interestticipate in crossovers to create new offspring and the
ing outcome is the effect of population size on the
20% parents are directly accepted as offspring.
degree of meta-stability. In the statistical mechanics
The mutation operator perturbs a solution to its
approach, instead of considering microscopic details
vicinity with a small mutation probability, pm . For
of the evolving system, several macroscopic variables
example, in a binary string of length `, a 1 is changed
9

describing the system, such as mean, standard deviation, skewness and kurtosis of the fitness distribution,
are modeled. Analyzing different GA implementations with the help of these cumulants provides interesting insights into the complex interactions among
GA operators [39]. Rudolph has shown that a simple GA with an elite-preserving operator and nonzero mutation probability converges to the global
optimum solution to any optimization problem [40].
Leaving the details of the theoretical studies (which
can be found from the growing EA literature), here
we shall illustrate the robustness (combined breadth
and efficiency) of a GA-based optimization strategy
in solving different types of optimization problems
commonly encountered in practice.

5.2

Real-Parameter
rithms

Genetic

AlgoFigure 4: Offspring created using the PCX operator.

When decision variables in an optimization problem

are real-valued, they can be directly represented in a
GA as mentioned above, instead of a binary representation. Such real-parameter GAs came to light
in nineties, when engineering applications of GAs
gained prominence. Although the GA flowchart remains the same, the crossover and mutation operations suitable to real-valued parameters were needed
to be developed.
Most studies used variable-wise crossover and mutation operators in the past. Recently, vector-wise
operators are suggested. Mutation operator perturbs
a parent solution in its vicinity by using a Gaussianlike probability distribution. However, sophisticated
crossover operators are also suggested. We proposed
a parent-centric crossover (PCX) operator [9] which
uses three or more parents and construct a probability distribution for creating offspring solutions. The
interesting feature of the probability distribution is
that it is defined based on the vector-difference of
parent solutions, instead of parent solutions themselves. Figure 4 shows the distribution of offspring
around parents under the PCX operator.
Since offspring solutions result in a diversity proportional to that in parent population, such a
crossover operator introduces a self-adaptive feature,
which allows a broad search early on and a focused

search later in an automatic manner. Such a property

is desired in an efficient optimization procedure and
is present in other real-parameter EAs, such as evolution strategy (ES) [42], differential evolution [44] and
particle swarm optimization [27].
On a 20-variable non-linear unconstrained minimization problem
f (x) =

n1
X
i=1

100(x2i xi+1 )2 + (xi 1)2 ,

(5)

a GA with the PCX operator [9] is reported to find

the optimum solution with an error of 1020 in function value in the smallest number of function evaluations compared to a number of other real-parameter
EAs ((, )-ES, CMA-ES [23] and differential evolution (DE)) and the classical quasi-Newton (BFGS)
approach (which got stuck to a solution having f =
6.077(1017)).

10

Method
GA+PCX
(1, 10)-ES
CMA-ES
DE
BFGS

Best
16,508
591,400
29,208
243,800
26,000

Median
21,452
803,800
33,048
587,920

Worst
25,520
997,500
41,076
942,040

In the following sections, we discuss a few practical

optimization problems and how their (approximate)
solution can be made easier with an evolutionary optimization strategy.

6.1

MIXED (REAL AND DISCRETE)

OPTIMIZATION
PROBLEMS

In an EO, decision variables can be coded in either

binary strings or directly depending on the type of
6 NON-SMOOTH
OPTI- the variable. A zero-one variable can be coded using a single bit (0 or 1). A discrete variable can be
MIZATION PROBLEMS
coded in either a binary string or directly (if the total
number of permissible choices for the variable is
Many practical problems are non-smooth in nature,
not
2k , where k is an integer). A continuous variable
introducing non-differentiabilities and discontinuities
in the objective and constraint functions. In such can be coded directly. This coding allows a natuspecial points of non-smoothness, the exact deriva- ral way to code different variables, as depicted in the
tives do not usually exist. Although numerical gra- following solution representing a complete design of
dients can be computed, they can often be meaning- a cantilever beam having four design variables:
less, particularly if such non-smoothness occurs on or
((1) 15 23.457 (1011))
near optimum points. Thus, it becomes difficult for
gradient-based optimization methods to work well in
The first variable represents the shape of the crosssuch scenarios. In a GA, such non-smoothness is not
section of the beam. There are two optionscircular
a matter, as a GA does not use any gradient infor(a 1) or square (a 0). Thus, its a zero-one variable.
mation during its search towards the optimum. As
The second variable represents the diameter of the
shown in Figure 5, as long as a descent information
circular section if the first variable is a 1 or the side
is provided by the function values (by the way of
of the square if the first variable is a 0. This variable
comparing function values of two solutions), a GA is
takes only one of a few pre-specified values. Thus,
ready to work its way towards the optimum.
this variable is a discrete variable coded directly. The
third variable represents the length of the cantilever
f(x)
f(x)
beam,
which can take any real value. Thus, it is a
Discontinuity
Discrete
continuous variable. The fourth variable is a discrete
variable representing the material of the cantilever
beam. This material takes one of 16 pre-specified
materials. Thus, a four-bit substring is required to
code this variable. The above solution represent the
x
x
12th
material from a pre-specified list. Thus, the
000 001
010 011 100 101 110 111
above string represents a cantilever beam made of
Figure 5: Discontinuous Figure 6: Discrete search the 12th material from a prescribed list of 16 materials having a circular cross-section with a diameter
search space.
space.
15 mm and having a length of 23.457 mm. With
the
above coding, any combination of cross sectional
In many problems, variables take discrete values,
shape
and size, material specifications, and length of
thereby providing no meaningful way of computthe
cantilever
beam can be represented. This flexiing gradients and using the gradient-based classical
bility
in
the
representation
of a design solution is not
methods. Once again, the sole purpose of arriving
possible
with
traditional
optimization
methods. This
at descent information using function value comparflexibility
makes
an
EO
efficient
to
be
used in many
isons allows a GA to be used in such discrete search
engineering
design
problems
[11,
16],
including
CFD
space as well (Figure 6).
problems involving shapes etc.

11

CONSTRAINT OPTIMIZATION

The terms (~x), (~x), Pc (~x), and (~x) are given below:
 0
(~x) =
( (~x))2 + ( 00 (~x))2 +
Constraints are inevitable in real-world (practical)
i
p
problems. The classical penalty function approach
` 0 (~x) 00 (~x)/ 0.25(`2 + (h + t)2 ) ,
of penalizing an infeasible solution is sensitive to
504, 000
a penalty parameter associated with the technique (~x) =
,
t2 b
and often requires a trial-and-error method. In the
3
penalty function method for handling inequality con- Pc (~x) = 64, 746.022(1 0.0282346t)tb ,
2.1952
straints in minimization problems, the fitness func(~x) =
,
t3 b
tion F (~x) is defined as the sum of the objective function f (~x) and a penalty term which depends on the where
constraint violation hgj (~x)i:
6, 000
,
0 (~x) =
J
X
2h`
2
p
Rj hgj (~x)i ,
(6)
F (~x) = f (~x) +
6, 000(14 + 0.5`) 0.25(`2 + (h + t)2 )
00
j=1
.
(~x) =
2 {0.707h`(`2/12 + 0.25(h + t)2 )}
where h i denotes the absolute value of the operand,
if the operand is negative and returns a value zero, The optimized solution reported in the literature

otherwise. The parameter Rj is the penalty param- [38] is h = 0.2444, ` = 6.2187, t = 8.2915,

eter of the j-th inequality constraint. The purpose and b = 0.2444 with a function value equal to

of a penalty parameter Rj is to make the constraint f = 2.38116. Binary GAs are applied on this probviolation gj (~x) of the same order of magnitude as the lem in an earlier study [4] and the solution ~x =
objective function value f (~x). Equality constraints (0.2489, 6.1730, 8.1789, 0.2533) with f = 2.43 (within
are usually handled by converting them into inequal- 2% of the above best solution) was obtained with
a population size of 100. However, it was observed
ity constraints as follows:
that the performance of GAs largely dependent on
the chosen penalty parameter values, as shown in Tagk+J (~x) |hk (~x)| 0,
ble 1. With R = 1 (small values of R), although 12
where is a small positive value.
out of 50 runs have found a solution within 150% of
In order to investigate the effect of the penalty pa- the best-known solution, 13 EO runs have not been
rameter Rj (or R) on the performance of GAs, we able to find a single feasible solution in 40,080 funcconsider a well-studied welded beam design problem tion evaluations. This happens because with small R
[38]. The resulting optimization problem has four de- there is not much pressure for the solutions to become
sign variables ~x = (h, `, t, b) and five inequality con- feasible. With larger penalty parameters, the presstraints:
sure for solutions to become feasible is more and all
50 runs found feasible solutions. However, because of
2
Min. fw (~x) = 1.10471h ` + 0.04811tb(14.0 + `),
larger emphasis of solutions to become feasible, when
s.t. g1 (~x) 13, 600 (~x) 0,
a particular solution becomes feasible it has a large
g2 (~x) 30, 000 (~x) 0,
selective advantage over other solutions (which are
g3 (~x) b h 0,
infeasible) in the population and the EO is unable to
g4 (~x) Pc (~x) 6, 000 0,
reach near to the true optimal solution.
g5 (~x) 0.25 (~x) 0,
In the recent past, a parameter-less penalty ap0.125 h 10,
proach is suggested by the author [7], which uses the
0.1 `, t, b 10.
following fitness function derived from the objective
(7)
12

ROBUST

AND

f(x)

Table 1: Number of runs (out of 50 runs) converged

RELIABILITY-BASED OPwithin % of the best-known solution using an EO
with different penalty parameter values and using the
TIMIZATION
proposed approach on the welded beam design problem.
For practical optimization studies, robust and
reliability-based techniques are commonplace and are
getting increasingly popular. Often in practice, the

Optimized fw (~x)
mathematical
optimum solution is not desired, due to

>
Inits
sensitivity
to
the parameter fluctuations and inacR
150% 150% fes.
Best
Med.
Worst
100
12
25
13
2.413 7.625 483.502 curacy in the formulation of the problem. Consider
101
12
38
0
3.142 4.335
7.455 Figure 7, in which although the global minimum is at
103
1
49
0
3.382 5.971
10.659 B, this solution is very sensitive to parameter fluctu106
0
50
0
3.729 5.877
9.424 ations. A small error in implementing solution B will
Prop.
50
0
0
2.381 2.383
2.384 result in a large deterioration in the function value.
On the other hand, solution A is less sensitive and
more suitable as the desired solution to the problem.
The dashed line is an average of the function around
and constraint values:

2
f (x), if x is feasible;
PJ
PK
F (x) =
fmax + j=1 hgj (x)i + k=1 |hk (x)|, else.
1.8
(8)
In a tournament between two feasible solutions, the
1.6
first clause ensures that the one with a better function value wins. The quantity fmax is the objective
1.4
function value of the worst feasible solution in the
population. The addition of these quantity to the
1.2
A
constraint violation ensures that a feasible solution
is always better than any infeasible solution. More1
Original
Average
B
over, since this quantity is constant in any generation, between two infeasible solutions the one with
0.8
0
0.2
0.4
0.6
0.8
1
smaller constraint violation is judged better. Since
x
the objective function value is never compared with
a constraint violation amount, there is no need of any
penalty parameter with such an approach.
Figure 7: Although solution B is the global minimum,
When the proposed constraint handling EO is ap- it is a robust solution. Solution A is the robust soluplied to the welded beam design problem, all 50 sim- tion.
ulations (out of 50 runs) are able to find a solution
very close to the true optimal solution, as shown in a small region near a solution. If the function shown
the last row of Table 1. This means that with the pro- in dashed line is optimized the robust solution can be
posed GAs, one run is enough to find a satisfactory achieved [2, 26, 12].
For a canonical deterministic optimization task,
solution close to the true optimal solution.
the optimum solution usually lies on a constraint surface or at the intersection of more than one constraint
surfaces. However, if the design variables or some
system parameters cannot be achieved exactly and
13

are uncertain with a known probability distribution

of variation, the so-called deterministic optimum (lying on one or more constraint surface) will fail to remain feasible in many occasions (Figure 8). In such

Relationship

x_2
99.99%





 




























 

 

99%






















































































95%
































 






















































































 90%






































































































































































































































80%






























 

























































































 

























































































 









x_1

























































 

 

 

 

 
 
 







Deterministic
optimum

Figure 8: The constrained minimum is not reliable.

With higher desired reliability, the corresponding solutions moves inside the feasible region.
scenarios, a stochastic optimization problem is usually formed and solved, in which the constraints are
converted into probabilistic constraints meaning that
probability of failures (of being a feasible solution)
is limited to a pre-specified value (say ) [36, 17].
The advantage of using an EO is that a global robust
solution can be obtained and the method can be extended for finding multi-objective reliable solutions
easily [28].

OPTIMIZATION
WITH
DISTRIBUTED COMPUTING

One way to beat the difficulties with problems having

computationally expensive evaluation procedures is

a distributed computing environment. This way the

evaluation procedure can be parallelized and overall computational time will reduce. There is an additional advantage with using an EO. Since an EO
deals with a population of solutions in every generation and their evaluations are independent to each
other, each population member can be evaluated on a
different processor independently and parallely. Homogeneous or heterogeneous processors on a masterslave configuration can be used for such a purpose
equally well to any other arrangements. Since the
computational time for solution evaluation is comparatively much more than the time taken by the
genetic operators, a fast input-output processing is
also not a requirement. Thus, a hand-made Beowulf
cluster formed with a number of fast processors can
be used efficiently to run an EO. This feature makes
an EO suitable for parallel processors compared to
their classical counterparts.
Ideally, the evaluation of each solution must also be
parallelized, if deemed necessary. For this purpose,
several clusters can be combined together and a GA
population member can be sent to each cluster for its
efficient evaluation, as shown in Figure 9.

10

COMPUTATIONALLY
DIFFICULT OPTIMIZATION PROBLEMS

Many engineering optimization problems involve objectives and constraints which require an enormous
computational time. For example, in CFD control
problems involving complicated shapes and geometries, every solution requires mesh generation, automated node-numbering, and solution of the finite difference equations. The computational time in such
problems can easily consume a few minutes to days
for evaluating a single solution on a parallel computer, depending the rigor of the problem. Clearly,
optimization of such problems is still out of scope
in its traditional sense and often innovations in using the optimization methodology must be devised.
Here, we suggest a couple of approaches:
1. Approximate models for evaluation instead of
14

f
e f
e fe l
k l
k lk

c
c
d
d
cc d
d
cc ddcc j
j
ii j
j
ii j
j
ii jjii

b
b
aab
ab
a b
b
aab
ab
a bbaababa h
h
ggh
h
gg h
h
ggh
h
gg hhgghhgg

a
a
a g
g
g w w w
`
_ `
_ `_
`_
x
x
x
uu vvuu
^
^
]] ^^]]

^^]]
v
u v
u v

r
q r
q rq v
u v
u v

\
\
[[[ \
\
[[[ \\[[[ p
p
oo p
p
oo p
p
oo ppoo
t
t
sss t
t
sss ttsss
\
\
[[
\
\
[[
\\[[ m
m
m t
t
ss
t
t
ss
ttss
n
n
mmm n
n
mmm nnmmm
n
n
m
m n
n
m
m nnmm

) *

) *) /
0

/ 0

/ 0/
*

'' (

'' (

'' (('' -

-- .

-- ..-(

-
.

(

(

.

.

(

%% &

%% &&%% ,

++ ,

++ ,,++
&

&

%% &

%% &%&%% ,

+++ ,

+++ ,,+++
%&

%&

&

&

,

,

&
& ,

,
,

,
, <

# $

# $#
$# &

; <

; <;
$

!! ""!!

9
:

99 :

99 ::99
"

""!!
"



:

5
6
5 6
5 65 9
:



  3

33 4

33 433
77 8

77 8877


8

3
4







778

778





8

8



  4
1 4
1 4 1 8

7 8

7 878787




8

8

2

2
1 2

12

1 2

1
2



2
2
1
1
2
11 2

11 2211
2

2

2

practical problems are solved. A saving of 30 to 80%

savings in the computational time has been recorded
by using the proposed approach.
G H

G HG N

M N

M NM
H

E

EE F

EE FFEE K

KK L

KK LLKK
E
F

K
L

F

F

L

L

CC D

CC DDCC J

II J

II JJII
D

D

CC
D

CC
DCDCC J

III
J

III
JJIII
J

J

CD

CD

D

D

J

J

A
B

A BA
BA
Y Z
Y ZY
Z

?? @@??

WW X

WW X

WW XXWW
@?
X

@

X

@


@? T

X

S T

S TS X

QQ R

QQ R

QQ RRQQ V

== >

== >>== R

UU V

UU VVUU


R


R


R
>



>


V
>

==

==

UV

UU

>

V

O V

= >

= >==>= O

U V

U VUUVU
>

>

V

OO O

OO PPOO U

P



P
P



P
OP

OP

O
P

P
OO
P

OO
POO

10.2

Approximate Optimization Principle

In practical optimal control problems involving timevarying decision variables, the objective and constraint functions need to be computed when the
whole process is simulated for a time period. In CFD
 

  

 ||{{ 

 















problems, such control problems often arise in which













 ||{{|{|{ 

 
|{||{{ 





 





















control parameters such as velocities and shapes need







{
|




 


||{{ 











to be changed over a time period in order to obtain


 zzyyzy 
 
  

 


 zzyy 

 



  

 




a minimum drag or a maximum lift or a maximally
 

zyyzy 



 ~~}}~} 
  


z




 ~~}} 

 
  

 




 
























zzyyzzyy 

stable system. The computation of such objectives























}


~




















}


~



~} 

 








is a non-linear process of solving a system of gov
~~}}~~}} 







erning partial differential equations sequentially from
the initial time to a final time. When such a series
Figure 9: A parallel computing environment for an of finite difference equations are solved for a particular solution describing the system, one evaluation
ideal EO.
is over. Often, such an exact evaluation procedure
may take a few hours to a few days. Even if only
exact objective and constraint functions can be a few hundred solutions are needed to be evaluated
used.
to come any where closer to the optimum, this may
take a few months on even a moderately fast par2. For time-varying control problems, a fast yet apallel computer. For such problems, an approximate
proximate optimization principle can be used.
optimization procedure can be used as follows [43].
The optimal control strategy for the entire time
10.1 Approximate Models
span [ti , tf ] is divided into K time intervals [tk , tk +
t], such that t0 = ti and tK = tK1 + t = tf . At
In this approach, an approximate model of the obthe k-th time span, the GA is initialized by mutatjective and constraint functions are first developed
ing the best solution found in (k 1)-th time span.
using a handful of solutions evaluated exactly. In
Thereafter, the GA is run for generations and the
this venture, a response surface methodology (RSM),
best solution is recorded. The control strategy correa kriging methodology or an ANN approach can all
sponding to this best solution is assigned as the overbe used. Since the model is at best assumed to be
all optimized control strategy for this time span. This
an approximate one, it must be used for a while and
is how the optimized control strategy gets formed as
discarded when solutions begins to crowd around the
the time span increases from t0 to tf . This probest region of the approximate model. At this stage,
posed genetic search procedure is fast (O(K) comanother approximate model can be developed near
pared to O(K 2 )) and allows an approximate way to
the reduced search region dictated by the current
handle computationally expensive time-varying optipopulation. One such coarse-to-fine grained approxmization problems, such as the CFD problem solved
imate modeling technique (with an ANN approach)
elsewhere [43]. Although the procedure is fast, on the
has been recently developed for multi-objective optiflip side, the proposed search procedure constitutes
mization [31] and a number of test problems and two
an approximate search and the resulting optimized
15

solution need not be the true optimum of the optimal

control problem. But with the computing resources
available today, demanding CFD simulations prohibit
the use of a standard optimization algorithm in practice. The above approximate optimization procedure
allows a viable way to apply optimization algorithms
in computationally demanding CFD problems.

MULTI-MODAL
MIZATION

0.8
0.7
0.6
f(x)

11

0.9

OPTI-

Many practical optimization problems possess a multitude of optima some global and some local. In
such problems, it may be desirable to find as many
optima as possible to have a better understanding of
the problem. Due to the population approach, GAs
can be used to find multiple optimal solutions in one
simulation of a single GA run. In one implementation [10], the fitness of a solution is degraded with its
niche count, an estimate of the number of neighboring solutions. It has been shown that if the reproduction operator is performed with the degraded fitness
values, stable subpopulations can be maintained at
various optima of the objective function. Figure 10
shows that a niched-GA can find all five optima (albeit a mix of local and global optima) in one single
simulation run.

0.5
0.4
0.3
0.2
0.1
0

0.2

0.4

0.6

0.8

Figure 10: A niched-GA finds multiple optimal solutions.

depends on the chosen scalarization scheme. Once

again, due to the population approach of a GA, multiple Pareto-optimal solutions are found simultaneously in a single simulation run, making it an unique
way to handle multi-objective optimization problems.
To convert a single-objective GA to a multiobjective optimizer, three aspects are kept in mind:
(i) non-dominated solutions are emphasized for progressing towards the optimal front, (ii) less-crowded
12 MULTI-OBJECTIVE EVO- solutions are emphasized to maintain a diverse set of
LUTIONARY OPTIMIZA- solutions, and (iii) elite or best solutions in a population are emphasized for a quicker convergence near
TION
the true Pareto-optimal front. In one such implemenMost real-world search and optimization problems tation elitist non-dominated sorting GA or NSGAinvolve multiple conflicting objectives, of which the II (which received the ESI Web of Sciences recoguser is unable to establish a relative preference. Such nition as the Fast Breaking Paper in Engineering
considerations give rise to a set of multiple optimal in February 2004), parent and offspring populations
solutions, commonly known as the Pareto-optimal are combined together and non-dominated solutions
solutions [8]. Classical approaches to solve these are hierarchically selected starting from the best soproblems concentrate mainly in developing a single lutions. A crowding principle is used to select the
composite objective function from multiple objectives remaining solutions from the last front which could
and in using a single-objective optimizer to find a not be selected in total. These operations follow the
particular optimum solution [30]. Such procedures above three aspects and the algorithm has been sucare subjective to the user, as the optimized solution cessful in solving a wide variety of problems. A sketch
16

of the algorithm is shown in Figure 11.

Nondominated
sorting

Crowding
distance
sorting

4000
3500
Pt+1

NSGAII (5% error)

3000

F1
Pt

2500
Volume (cc)

F2
F3

Qt

Rejected

2000
JA (1990)

1500
1000

Rt

NSGAII (10% error)

500
0

Figure 11: An iteration of the NSGA-II procedure.

10

12

14

Power (kW)

On a 18-speed gearbox design problem having 28

mixed decision variables, three objectives, and 101
nonlinear constraints, the NSGA-II is able to find
as many as 300 different trade-off solutions including
individual minimum solutions (Figure 12). It is clear
from the figure that if a large error () in the output
shaft speeds from the ideal is permitted, better nondominated solutions are expected. Because of the
mixed nature of variables and multiple objectives, a
previous study [25] based on classical methods had
to divide the problem in two subproblems and even
then it failed to find a wide spectrum of solutions
as found by NSGA-II. Since the number of teeth in
gears are also kept as variables here, a more flexible
search is permitted, thereby obtaining a better set of
non-dominated solutions.

Figure 12: NSGA-II solutions are shown to outperform a classical approach with two limiting errors in
output speeds.

For example, for the gearbox design problem described above, when we analyze all 300 solutions, we
find that the only way these optimal solutions vary
from each other is by having a drastically different
value in only one of the variables (module of the
gear) and all other variables (thickness and number
of teeth of each gear) remain more or less the same
for all solutions. Figure 13 shows how module (m) is
changing with one of the objectives (power delivered
(p) by the gearbox). When a curve is fitted through
these points, the following mathematical relationship

12.1 Innovization:
Understanding emerges: m = p. This is an useful information
for a designer to have. What this means is that if
the Optimization Problem Bet- a gearbox is designed today for p = 1 kW power reter
quirement and tomorrow if a gearbox is needed to
Besides finding the trade-off optimal solutions, evo- be designed for another application requiring p = 4
lutionary multi-objective optimization (EMO) stud- kW the only change needed in the gearbox is an inies are getting useful for another reason. When a crease of module by a factor of two. A two-fold inset of trade-off solutions are found, they can be ana- crease in module will increase the diameter of gears
lyzed to discover any useful similarities and dissimi- and hence the size of the gearbox. Although a comlarities among them. Such information, particularly plete re-optimization can be performed from scratch
the common principles, if any, should provide useful and there may be other ways the original gearbox
can be modified for the new requirement, but such a
insight to a designer, user, or a decision-maker.
17

0.4

Upper limit

0.35

Module (cm)

0.3
0.25
0.2
0.15
Lower limit

0.1
1 2

10

12

14

16

18

20

Power (kW)

Figure 13: An optimal relationship between module

and power is discovered by the NSGA-II.
simple change in module alone will mean a redesign
of the gearbox for the new requirement in an optimal
manner and importantly not requiring any further
computation [14]. Such a task brings out a recipe
or blue-print of solving the problem optimally. It is
not clear how such useful information can be achieved
by any other means.

13

from classical optimization methods in a number of

ways: (i) it does not use gradient information, (ii)
it works with a set of solutions instead of one solution in each iteration, (iii) it is a stochastic search
and optimization procedure and (iv) it is highly parallelizable.
Besides GAs, there exist a number of other implementations of the evolutionary idea, such as evolution strategy [42, 1], evolutionary programming [19],
genetic programming [29], differential evolution [44],
particle swarm optimization [27] and others. Due to
the flexibilities in their search, these evolutionary algorithms are found to be quite useful as a search tool
in studies on artificial neural networks, fuzzy logic
computing, data mining, and other machine learning
activities.

Acknowledgment
The author acknowledges the efforts of all his collaborators during the past 13 years of research and
application in developing efficient evolutionary optimization methods for practical problem solving.

References
[1] H.-G. Beyer. The theory of evolution strategies.
Berlin, Germany: Springer, 2001.
[2] J. Branke. Creating robust solutions by means of
an evolutionary algorithm. In Proceedings of the
Parallel Problem Solving from Nature (PPSNV), pages 119128, 1998.

CONCLUSIONS

In this paper, we have presented a number of

commonly-used practical problems which are, in
essence, optimization problems. A closer look at
these problems has revealed that such optimization problems are complex, non-linear, and multidimensional, thereby making the classical optimization procedures inadequate to be used with any reliability and confidence. We have also presented the
evolutionary optimization procedure a populationbased iterative procedure mimicking natural evolution and genetics, which has demonstrated its suitability and immense applicability in solving various
types of optimization problems. A particular implementation genetic algorithm (GA) is different

[3] R. Dawkins. The Selfish Gene. New York: Oxford University Press, 1976.
[4] K. Deb. Optimal design of a welded beam
structure via genetic algorithms. AIAA Journal,
29(11):20132015, 1991.
[5] K. Deb. Genetic algorithms in optimal optical
filter design. In Proceedings of the International
Conference on Computing Congress, pages 29
36, 1993.

18

[6] K. Deb. Optimization for Engineering Design: [15] K. Deb and A. R. Reddy. Classification of twoAlgorithms and Examples. New Delhi: Prenticeclass cancer data reliably using evolutionary alHall, 1995.
gorithms. BioSystems, 72(1-2):111129, 2003.
[7] K. Deb. An efficient constraint handling method [16] K. Deb and A. Srinivasan. Innovization: Innovation through optimization. Technical Report
for genetic algorithms.
Computer Methods
KanGAL Report Number 2005007, Kanpur Gein Applied Mechanics and Engineering, 186(2
netic Algorithms Laboratory, IIT Kanpur, PIN
4):311338, 2000.
208016, 2005.
[8] K. Deb. Multi-objective optimization using evolutionary algorithms. Chichester, UK: Wiley, [17] O. Ditlevsen and H. O. Madsen. Structural Reliability Methods. New York: Wiley, 1996.
2001.
[9] K. Deb, A. Anand, and D. Joshi. A computationally efficient evolutionary algorithm for realparameter optimization. Evolutionary Computation Journal, 10(4):371395, 2002.
[10] K. Deb and D. E. Goldberg. An investigation of
niche and species formation in genetic function
optimization. In Proceedings of the Third International Conference on Genetic Algorithms,
pages 4250, 1989.
[11] K. Deb and M. Goyal. A robust optimization procedure for mechanical component design
based on genetic adaptive search. Transactions
of the ASME: Journal of Mechanical Design,
120(2):162164, 1998.

[18] N. Eldredge. Macro-Evolutionary Dynamics:

Species, Niches, and Adaptive Peaks. New York:
McGraw-Hill, 1989.
[19] D. B. Fogel. Evolutionary Computation. Piscataway, NY: IEEE Press, 1995.
[20] L. J. Fogel. Autonomous automata. Industrial
Research, 4(1):1419, 1962.
[21] D. E. Goldberg. Genetic Algorithms for Search,
Optimization, and Machine Learning. Reading,
MA: Addison-Wesley, 1989.
[22] D. E. Goldberg. The design of innovation:
Lessons from and for Competent genetic algorithms. Kluwer Academic Publishers, 2002.

[12] K. Deb and H. Gupta. Searching for robust

Pareto-optimal solutions in multi-objective optimization. In Proceedings of the Third Evolutionary Multi-Criteria Optimization (EMO-05)
Conference (Also Lecture Notes on Computer
Science 3410), pages 150164, 2005.

[23] N. Hansen and A. Ostermeier. Adapting arbitrary normal mutation distributions in evolution
strageties: The covariance matrix adaptation. In
Proceedings of the IEEE International Conference on Evolutionary Computation, pages 312
317, 1996.

[13] K. Deb, P. Jain, N. Gupta, and H. Maji.

Multi-objective placement of VLSI components
using evolutionary algorithms. To appear in
IEEE Transactions on Components and Packaging Technologies, Also KanGAL Report No.
2002006, 2002.

[24] J. H. Holland. Adaptation in Natural and Artificial Systems. Ann Arbor, MI: MIT Press, 1975.
[25] P. Jain and A. M. Agogino. Theory of design: An
optimization perspective. Mech. Mach. Theory,
25(3):287303, 1990.

[14] K. Deb and S. Jain. Multi-speed gearbox [26] Yaochu Jin and Bernhard Sendhoff. Trade-off
between performance and robustness: An evodesign using multi-objective evolutionary algolutionary multiobjective approach. In Proceedrithms. ASME Transactions on Mechanical Deings of the Evolutionary Multi-Criterion Optisign, 125(3):609619, 2003.
mization (EMO-2003), pages 237251, 2003.
19

[27] James Kennedy and Russell C. Eberhart. Swarm

intelligence. Morgan Kaufmann, 2001.
[28] R. T. F. King, H. C. S. Rughooputh, and
K. Deb. Evolutionary multi-objective environmental/economic dispatch: Stochastic versus
deterministic approaches. In Proceedings of the
Third International Conference on Evolutionary Multi-Criterion Optimization (EMO-2005),
pages 677691. Lecture Notes on Computer Science 3410, 2005.
[29] J. R. Koza. Genetic Programming : On the Programming of Computers by Means of Natural Selection. Cambridge, MA: MIT Press, 1992.
[30] K. Miettinen. Nonlinear Multiobjective Optimization. Kluwer, Boston, 1999.
[31] P. K. S. Nain and K. Deb. Computationally effective search and optimization procedure using
coarse to fine approximations. In Proceedings
of the Congress on Evolutionary Computation
(CEC-2003), pages 20812088, 2003.
[32] D. Pratihar, K. Deb, and A. Ghosh. Fuzzygenetic algorithms and time-optimal obstaclefree path generation for mobile robots. Engineering Optimization, 32:117142, 1999.
[33]

[34]

[35]
[36]

[37] I. Rechenberg. Cybernetic solution path of an

experimental problem, 1965. Royal Aircraft Establishment, Library Translation Number 1122,
Farnborough, UK.
[38] G. V. Reklaitis, A. Ravindran, and K. M. Ragsdell. Engineering Optimization Methods and Applications. New York : Wiley, 1983.
[39] A. Rogers and A. Pr
ugel-Bennett. Modelling the
dynamics of steady-state genetic algorithms. In
Foundations of Genetic Algorithms 5 (FOGA5), pages 5768, 1998.
[40] G. Rudolph. Convergence analysis of canonical
genetic algorithms. IEEE Transactions on Neural Network, 5(1):96101, 1994.
[41] H.-P. Schwefel. Projekt MHD-Staustrahlrohr:
Experimentelle
optimierung
einer
zweiphasend
use, teil I.
Technical Report
11.034/68, 35, AEG Forschungsinstitut, Berlin,
1968.
[42] H.-P. Schwefel. Numerical Optimization of Computer Models. Chichester, UK: Wiley, 1981.

[43] T. K. Sengupta, K. Deb, and S. B. Talla.

Drag optimization for a circular cylinder at high
reynolds number by rotary oscillation using geD. K. Pratihar, K. Deb, and A. Ghosh. Opnetic algorithms. Technical Report KanGAL Retimal path and gait generations simultaneously
port No. 2004018, Mechanical Engineering Deof a six-legged robot using a ga-fuzzy approach.
partment, IIT Kanpur, India, 2004.
Robotics and Autonomous Systems, 41:121,
[44] R. Storn and K. Price. Differential evolution
2002.
A fast and efficient heuristic for global optimizaA. Pr
ugel-Bennett and J. L. Shapiro. An analtion over continuous spaces. Journal of Global
ysis of genetic algorithms using statistical meOptimization, 11:341359, 1997.
chanics. Physics Review Letters, 72(9):1305
[45] M. D. Vose. Simple Genetic Algorithm: Foun1309, 1994.
dation and Theory. Ann Arbor, MI: MIT Press,
S. S. Rao. Optimization: Theory and Applica1999.
tions. Wiley, New York, 1984.
[46] M. D. Vose and J. E. Rowe. Random heuristic
S. S. Rao. Genetic algorithmic approach for mulsearch: Applications to gas and functions of unitiobjective optimization of structures. In Protation. Computer Methods and Applied Mechanceedings of the ASME Annual Winter Meeting
ics and Engineering, 186(24):195220, 2000.
on Structures and Controls Optimization, volume 38, pages 2938, 1993.
20