122 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL.

SMC-16, NO, 1, JANUARY/FEBRUARY 1986

Optimization of Control Parameters for

Genetic Algorithms
JOHN J. GREFENSTETTE, MEMBER, IEEE

Abstract-The task of optimizing a complex system presents at least

two levels of problems for the system designer. First, a class of optimiza- Environment E Performance Measure u
tion algorithms must be chosen that is suitable for application to the
system. Second, various parameters of the optimization algorithm need to
be tuned for efficiency. A class of adaptive search procedures called genetic
Control Feedback f
algorithms (GA) has been used to optimize a wide variety of complex Input c
systems. GA's are applied to the second level task of identifying efficient
Adaptive
GA's for a set of numerical optimization problems. The results are vali- Strategy
dated on an image registration problem. GA's are shown to be effective for
both levels of the systems optimization problem. Fig. 1 One-level adaptive system model.

I. INTRODUCTION
a parameterized space of GA's in order to identify efficient
T HE PROBLEM of dynamically controlling a complex GA's for the task of optimizing a set of numerical func-
process often reduces to a numerical function optimi- tions. This search is performed by a metalevel GA. Thus
zation problem. Each task environment for the process GA's are shown to be suitable for both levels of the system
defines a performance response surface which must be optimization problem.
explored in general by direct search techniques in order to The remainder of this paper is organized as follows:
locate high performance control inputs (see Fig. 1). Section II contains a brief overview of GA's and a summary
If the response surface is fairly simple, conventional of previous work. Section III describes the design of ex-
nonlinear optimization or control theory techniques may periments which test the performance of GA's as meta-level
be suitable. However, for many processes of interest, e.g. optimization strategies. The experimental results appear in
computer operating systems or system simulation pro- Section IV. A validation study is presented in Section V.
grams, the response surface is difficult to search, e.g., a The conclusions are summarized in Section VI.
high-dimensional, multimodal, discontinuous, or noisy
function of the control inputs. In such cases, the choice of II. OVERVIEW OF GENETIC ALGORITHMS
optimization technique may not be obvious. Even when an
appropriate class of optimization algorithms is available, Suppose we desire to optimize a process having a re-
there are usually various parameters that must be tuned, sponse surface u, which depends on some input vector x. It
e.g., the step size in a variable metric technique. Often the is assumed that no initial information is available concern-
choice of parameters can have significant impact on the ing the surface u, but that a black box evaluation proce-
effectiveness of the optimization algorithm [8]. The prob- dure can be invoked to compute the scalar function u(x).
lem of tuning the primary algorithm represents a sec- The state of the art in such situations is to perform some
ondary, or metalevel, optimization problem (see Fig. 2). sort of random search, perhaps combined with local hill-
This work attempts to determine the optimal control climbing procedures [5], [9]. Genetic algorithms are global
parameters for a class of global optimization procedures optimization techniques that avoid many of the shortcom-
called genetic algorithms (GA's). The class of GA's is ings exhibited by local search techniques on difficult search
distinguished from other optimization techniques by the spaces.
use of concepts from population genetics to guide the A GA is an iterative procedure which maintains a con-
search. However, like other classes of algorithms, GA's stant-size population P(t) of candidate solutions. During
differ from one another with respect to several parameters each iteration step, called a generation, the structures in
and strategies. This paper describes experiments that search the current population are evaluated, and, on the basis of
those evaluations, a new population of candidate solutions
Manuscript received March 21, 1984; revised August 28, 1985. This
is formed (see Fig. 3.)
work was supported in part by a Fellowship from the Vanderbilt Univer- The initial population P(O) can be chosen heuristically
sity Research Council and by the National Science Foundation under or at random. The structures of the population P(t + 1)
Grant MCS-8305693. are chosen from P(t) by a randomized selection procedure
The author is with the Computer Science Department, Vanderbilt
University, Nashville, TN 37235, USA. that ensures that the expected number of times a structure
IEEE Log Number 8406073. is chosen is approximately proportional to that structure's

0018-9472/86/0100-0122$01.00 (©1986 IEEE

Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
GREFENSTETTE: GENETIC ALGORITHMS FOR MULTILEVEL ADAPTIVE SYSTEMS 123

Parameters
x _--------------

St

Fig. 2 Two-levvel adaptive system model.

t <- 0;
initialize P(t); -- P(t) is the population at time t
by fixing the total number of structure evaluations, or
evaluate P(t); some other application dependent criterion. For a more
while (termination condition not satisfied) do
begin thorough introduction to GA's [7], [17].
t <- t+l;
select P(t);
As stated above, GA's are essentially unconstrained
recombine P(t); search procedures within the given representation space.
evaluate P(t);
end; Constraints may be handled indirectly through penalty
Fig. 3 Skeleton of a genetic algorithm. functions. A more direct way to incorporate constraints has
been proposed by Fourman [12], who treats the structures
in the population as lists of consistent constraints for VLSI
performance relative to the rest of the population. In order layout problems.
to search other points in the search space, some variation is Genetic algorithms have not enjoyed wide recognition,
introduced into the new population by means of idealized possibly due to a misconception that GA's are similar to
genetic recombination operators. The most important re- early "evolutionary programming" techniques [11], which
combination operator is called crossover. Under the cross- rely on random mutation and local hill-climbing. The basic
over operator, two structures in the new population concepts of GA's were developed by Holland [17] and his
exchange portions of their internal representation. For students [1], [2], [4], [6], [13], [15], [19]. These studies have
example, if the structures are represented as binary strings, produced the beginnings of a theory of genetic adaptive
crossover can be implemented by choosing a point at search. For example, an application of gambler's ruin
random, called the crossover point, and exchanging the theory to the allocation of trials to the hyperplanes of the
segments to the right of this point. Let xl = 100:01010 and search space shows that genetic techniques provide a near-
x2= 010:10100, and suppose that the crossover point has optimal heuristic for information-gathering in complex
been chosen as indicated. The resulting structures would be search spaces [6], [17]. Bethke [1] provides theoretical char-
Yi = 100:10100 and Y2 = 010:01010. acterizations of problems which may be especially well-
Crossover serves two complementary search functions. suited or especially difficult for GA's [1]. In addition, a
First, it provides new points for further testing within the number of experimental studies show that GA's exhibit
hyperplanes already represented in the population. In the impressive efficiency in practice. While classical gradient
above example, both xl and Yi are representatives of the search techniques are more efficient for problems which
hyperplane 100# # # # #, where the # is a "don't care" satisfy tight constraints, GA's consistently outperform both
symbol. Thus, by evaluating Yl, the GA gathers further gradient techniques and various forms of random search
knowledge about this hyperplane. Second, crossover intro- on more difficult (and more common) problems, such as
duces representatives of new hyperplanes into the popula- optimizations involving discontinuous, noisy, high-dimen-
tion. In the previous example, Y2 is a representative of the sional, and multimodal objective functions. GA's have
hyperplane #1001.# ##, which is not represented by been applied to various domains, including combinatorial
either parent structure. If this hyperplane is a high-perfor- optimization [12], [16], image processing [10], pipeline con-
mance area of the search space, the evaluation of Y2 will trol systems [15], and machine learning [2], [18], [25].
lead to further exploration in this subspace. Each evalua-
tion of a structure of length L contributes knowledge III. EXPERIMENTAL DESIGN
about the performance of the 2L hyperplanes represented
by that structure. The power of GA's derives largely from We now describe experiments which attempted to opti-
their ability to exploit efficiently this vast amount of mize the performance of GA's on a given set of function
accumulating knowledge by means of relatively simple optimization problems. These experiments were designed
selection mechanisms [17]. Termination of the GA may be to search the space of GA's defined by six control parame-
triggered by finding an acceptable approximate solution, ters, and to identify the optimal parameter settings with
Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
124 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-16, NO. 1, JANUARY/FEBRUARY 1986

respect to two different performance measures. The searches generation. The current experiments allowed G to vary
for the optimal GA's were performed by GA's, which between 0.30 and 1.00, in increments of 0.10.
demonstrates the efficiency and power of GA's as metalev- 5) Scaling Window (W): When maximizing a numerical
el optimization techniques. A metalevel GA could similarly function f(x) with a GA, it is common to define the
search any other space of parameterized optimization pro- performance value u(x) of a structure x as u(x) = f(x) -
cedures. fmin, where fmin is the minimum value that f(x) can
assume in the given search space. This transformation
guarantees that the performance u(x) is positive, regard-
A. The Space of Genetic Algorithms less of the characteristics of f(x). Often, fmin is not
Holland [17] describes a fairly general framework for the available a priori, in which case it is reasonable to define
class of GA's. There are many possible elaborations of u(x) = f (x) - f(xmin), where f(xmin) is the minimum
GA's involving variations such as other genetic operators, value of any structure evaluated so far. Either definition of
variable sized populations, etc. This study is limited to a u(x) has the unfortunate effect of making good values of x
particular subclass of GA's characterized by the following hard to distinguish. For example, suppose fmin = 0. After
six parameters. several generations, the current population might contain
1) Population Size (N): The population size affects both only structures x for which 105 <f(x) < 110. At this
the ultimate performance and the efficiency of GA's. GA s point, no structure in the population has a performance
generally do poorly with very small populations [22], be- which deviates much from the average. This reduces the
cause the population provides an insufficient sample size selection pressure toward the better structures, and the
for most hyperplanes. A large population is more likely to search stagnates. One solution is to define a new parameter
contain representatives from a large number of hyper- fm,1in with a value of say, 100, and rate each structure
planes. Hence, the GA's can perform a more informed against this standard. For example, if f(xi)= 110 and
search. As a result, a large population discourages prema- f(x;) - 105, then U(Xi) = f(xi) - fmin = 10, and u(xj) =
ture convergence to suboptimal solutions. On the other f (xj) - fmin 5; the performance of xi now appears to be
hand, a large population requires more evaluations per twice as good as the performance of xi.
generation, possibly resulting in an unacceptably slow rate Our experiments investigated three scaling modes, based
of convergence. In the current experiments, the population on a parameter called the scaling window W. If W = 0,
size ranged from 10 to 160 in increments of 10. then scaling was performed as follows: fmin was set to the
2) Crossover Rate (C): The crossover rate controls the minimum f(x) in the first generation. For each succeeding
frequency with which the crossover operator is applied. In generation, those structures whose evaluations were less
each new population, C * N structures undergo crossover. than fmin were ignored in the selection procedure. The fmin
The higher the crossover rate, the more quickly new struc- was updated whenever all the structures in a given popula-
tures are introduced into the population. If the crossover tion had evaluations greater than fm. If 0 < W < 7, then
rate is too high, high-performance structures are discarded we set fmin to the least value of f(x) which occurred in the
faster than selection can produce improvements. If the last W generations. A value of W = 7 indicated an infinite
crossover rate is too low, the search may stagnate due to window (i.e., no scaling was performed).
the lower exploration rate. The current experiments al- 6) Selection Strategy (S): The experiments compared two
lowed 16 different crossover rates, varying from 0.25 to selection strategies. If S = P, a pure selection procedure
1.00 in increments of 0.05. was used, in which each structure in the current population
3) Mutation Rate (M): Mutation is a secondary search is reproduced a number of times proportional to that
operator which increases the variability of the population. structure's performance. If S = E, an elitist strategy was
After selection, each bit position of each structure in the employed. First, pure selection is performed. In addition,
new population undergoes a random change with a prob- the elitist strategy stipulates that the structure with the best
ability equal to the mutation rate M. Consequently, ap- performance always survives intact into the next genera-
proximately M * N * L mutations occur per generation. A tion. In the absence of such a strategy, it is possible for the
low level of mutation serves to prevent any given bit best structure to disappear, due to sampling error, cross-
position from remaining forever converged to a single over, or mutation.
value in- the entire population. A high level of mutation We denote a particular GA by indicating its respective
yields an essentially random search. The current experi- values for the parameters N, C, M, G, W, and S. Early
ments allowed eight values for the mutation rate, increas- work by De Jong [6] suggests parameter settings which
ing exponentially from 0.0 to 1.0. have been used in a number of implementations of genetic
4) Generation Gap (G): The generation gap controls the algorithms. Based on De Jong's work, we define the stan-
percentage of the population to be replaced during each dard GA as GAS = GA(50, 0.6,, 0.001, 1.0, 7, E). The
generation. That is N * (1 G) structures of P(t) are Cartesian product of the indicated ranges for the six
chosen (at random) to survive intact in P(t + 1). A value parameters (N, C, M, G, W, S) defines a space of 218
of G = 1.0 means that the entire population is replaced GA's. In some cases, it is possible to predict how variations
during each generation. A value of G = 0.5 means that half of a single parameter will affect the performance of the
of the structures in each population survive into the next GA's, assuming that all other parameters are kept fixed [6].
Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
GREFENSTETTE: GENETIC ALGORITHMS FOR MULTILEVEL ADAPTIVE SYSTEMS 125

However, it is difficult to predict how the various parame- TABLE I

ters interact. For example, what is the effect of increasing FUNCTIONS COMPRISING THE TEST ENVIRONMENT
the population size, while lowering the crossover rate? The Function Dimensions Size of Space Description
analytic -optimization of this space is well beyond our fl 3 1.0 x 109 parabola
current understanding of GA's. It is also clear that an f2 2 1.7 x 106 Rosenbrock's saddle [23]
exhaustive enumeration of the space is infeasible. Our f3 5 1.0 X 1015 step function
f4 30 1.0 X 1072 quartic with noise
approach was to apply a metalevel GA's to the problem of f5 2 1.6 X 1010 Shekel's foxholes [241
identifying high-performance GA's. Each structure in the
population of the meta-level GA consisted of an 18-bit
vector which identified a particular GA. The performance response surface e. As normalized, UE and UE for random
of each GA was measured during the performance of a search will be 100.0, while UE and UE* for more effective
series of function optimization tasks. The meta-level GA search strategies will be correspondingly lower (for minimi-
used this information to conduct a search for high-perfor- zation problems).
mance algorithms.
D. Experimental Procedures
B. Task Environment
Two experiments were performed, one to optimize on-line
Each GA was evaluated by using it to perform five performance and one to optimize offline performance. For
optimization tasks, one for each of five carefully selected each experiment, the procedure for obtaining the optimum
numerical test functions. As a results of these optimization GA was as follows.
tasks, the GA was assigned a value according to one of the 1) One thousand GA's were evaluated, using a metalevel
performance measures explained below. The functions GA to perform the search through the space of GA's
comprising the task environment have been studied in defined by the six GA parameters. Each evaluation com-
previous studies of GA's [3], [6] and included functions prised running one GA against each of the five test func-
with various characteristics, including discontinuous, multi- tions for 5000 function evaluations and normalizing the
dimensional, and noisy functions. Table I gives a brief result with respect to the performance of random search on
description of the test functions. the same function. The metalevel GA started with a popu-
lation of 50 randomly chosen GA's and used the standard
C. Performance Measures parameter settings, i.e., GA(50, 0.6, 0.001, 1.0, 7, E). Past
Two performance metrics for adaptive search strategies experience has shown that these parameters yield a fairly
were considered, online performance and offline perfor- good search for a variety of problems, and so this was the
mance [6]. The on-line performance of a search strategy s natural choice for the meta-level.
on a response surface e is defined as follows: 2) Since GA's are randomized algorithms, the perfor-
mance of a GA during a single trial in the metalevel
Ue(s, T) = avet(Ue(t)), t = 0,1, *, T experiment represents a sample from a distribution of
where ue(t) is the performance of the structure evaluated performances. Therefore, it was decided that the GA's
at time t. That is, online performance is the average showing the best performances during step 1 would be
performance of all tested structures over the course of the subjected to more extensive testing. Each of the 20 best
search. GA's in step 1 was again run against the task environment,
The offline performance of a search strategy s on a this time for five trials for each test function, using differ-
response surface e is defined as follows: ent random number seeds for each trial. The GA which
exhibited the best performance in this step was declared
Ue*(s, T) = avej(u*(t)), t = 0, 1, T
the winner of the experiment.
where u*(t) is the best performance achieved in the time
interval [0, t]. Offline performance is the relevant measure IV. RESULTS
when the search can- be performed offline (e.g., via a
simulation model), while the best structure so far is used to A. Experiment 1-Online Performance
control the online system. The first experiment was designed to search for the
In order to measure global robustness, corresponding optimal GA with respect to online performance on the task
performance measures are defined for the entire set of environment. Fig. 4 shows the average online performance
response surfaces E: for the 50 GA's in each of the 20 generations of experiment
UE(s, T) = 100.0*
avee(Ue(s, T)/Ue(rand, T)), 1. Recall that the overall scores for random search on the
e in E task environment is 100.0. From the initial data point in
Fig. 4, we can estimate that the average online performance
UE (S, T) = 1000* avee(Ue*(s, T)/Ue*(rand, T)), of all GA's in our search space is approximately 56.6, or
e in E about 43.4 percent better than random search. Fig. 4 shows
where U, (rand, T) and U,*(rand, T) are on-line and off- that the final population of GA's had significantly better
line performance, respectively, of pure random search on performance than the average GA.
Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
126 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-16, NO. 1, JANUARY/FEBRUARY 1986

so 250k

z
50 c: 200
uJ¢D a:
z 0
CC
a: U-
Cl;
z
a:
040
U-
06150
tr: z
hi
aL
-J
hi U-
Z30 IL100
-J
z
0
C:
hi a:
020
a:
i 5
a: at:

10 -0 4 e 12 16 20
GENERRTIONS
Fig. 5 Experiment 2.
-0 4 8 12 16 20
GENERAT I ONS
Fig. 4 Experiment 1. represented in the population. As a result, a relatively good
structure may overrun the entire population in a few
This experiment identified GA1 = GA(30, 0.95, 0.01, generations. Unless the mutation rate is high, the GA will
1.0, 1, E) as the optimal GA with respect to online quickly converge to a suboptimal solution. In contrast,
performance. In an extended comparison, GA1 showed a random search will usually locate at least one high perfor-
3.09 percent improvement (with respect to the baseline mance point within the first thousand trials, leading to
performance of random search) over GAs on the task relatively good offline performance. That is, random search
environment. This represents a small but statistically sig- is a fairly tough competitor for search strategies when the
nificant improvement over the expected online perfor- goal is good offline performance. It is encouraging that
mance of GAS. The performance improvement between many GA's perform significantly better than random search
with respect to the offline performance measure.
GAS and GA1 can be attributed to an interaction among a This experiment identified GA2= GA(80, 0.45, 0.01,
number of factors. First, GA1 uses a smaller population, 0.9, 1, P) as the optimal GA with respect to offline perfor-
which allows many more generations within a given num- mance. In an extended comparison, GA2 showed a 3.0
ber of trials. For example, on functions in the task environ- percent performance improvement over GAS on the task
ment, GA1 iterated through an average of twice as many environment. Because of the high variance shown by GA's
generations as GAS. Second, GAS uses an infinite window, with respect to offline performance, this does not represent
i.e., no scaling is performed. GA1 uses a small window (one a statistically significant difference between GA2 and GAS.
generation), which resulted in a more directed search. There are several interesting difference between GA2 and
These two factors are apparently balanced by the signifi-
cantly increased crossover rate and mutation rate in GA1. GA'. With a larger population and higher mutation rate,
A higher crossover rate tends to disrupt the structures the population will tend to contain more variety, thus
selected for reproduction at a high rate, which is important increasing the random aspects of the GA. The slightly
in a small population, since high performance individuals lower generation gap also tends to reduce the effects of
are more likely to quickly dominate the population. The selection, resulting in a less focused search. These aspects
are balanced by the lower crossover rate and the small
higher mutation rate also helps prevent premature conver- scaling window which tend to enhance the selective pres-
gence to local optima. sure.
B. Experiment 2-Offline Performance
The second experiment was designed to search for the C. General Observations
optimal GA with respect to offline performance on the task Besides suggesting optimal GA's, the above experiments
environment. Fig. 5 shows that the average offline perfor- also provide performance data for 2000 GA's with various
mance of all GA's (214.8) appears to be much worse than parameter settings. Given that these are not independent
the average offline performance of random search (100.0). samples from the space of GA's, it is rather difficult to
This finding verifies the experience of many practitioners make valid statistical inferences from this data. Neverthe-
that GA's can prematurely converge to suboptimal solu- less the data does suggest some regularities that might
tions when given the wrong control parameters. For exam- warrant further studies.
ple, if a very small population size is used (i.e., N = 10), The experimental data confirms several observations first
the number of representatives from any given hyperplane is made by De Jong on the basis of a relatively small number
so small that the selection procedure has insufficient infor- of experiments [6]. For example, mutation rates above 0.05
mation to properly apportion credit to the hyperplanes are generally harmful with respect to online performance,
Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
GREFENSTETTE: GENETIC ALGORITHMS FOR MULTILEVEL ADAPTIVE SYSTEMS 127

30~

- 20
-j

10
-j )DI,
0i 'IO,
-0. -
A.
n_ A 10-
-u 3u u0 so u 120 IS -0-.
POPULATION SIZE
Fig. 6 Average online performance of various population sizes accord-
a: GA1
ing to Experiment 1.

with performance approaching that of random search with 0 100oo

TRIALS
O-
2000 3000 4000 5000
rates above 0.1 regardless of the other parameter settings. Fig. 7 Online performance of GA1 and GAS on an image registration
The absence of mutation is also associated with poorer task.
performance, which suggests that mutation performs an
important service in refreshing lost values. Best on-line
performance can be obtained with a population size in the 8
range of 30-100 structures as shown by Fig. 6, which plots
the average online performance as a function of population z
Lu 6
size, ignoring those GA's with mutation rates above 0.05. CC
w
A similar graph can be obtained for offline performance
except that range for the best population size is 60-110
structures. A large generation gap generally improves per-
formance, as does the elitist selection strategy. u.i
- - G

GAS
The performance data also suggests other regularities c:
that have not been previously noted. First, the adoption of
a small scaling window (1-5 generations) is associated with 10 1000 2000 3000 4000 5000
a slight improvement in both online and offline perfor- TRIALS
mance. This is reasonable since scaling enhances the pres- Fig. 8 Offline performance of GA2 and GAs on an image registration
sure of selection in later stages of the search. In small task.
populations (20 to 40) structures, good online performance
is associated with either a high crossover rate combined
with a low mutation rate or a low crossover rate combined task environment. The validation task was the following
with a high mutation rate. For mid-sized populations (30 to image registration problem: In order to compare'two gray-
90 structures), the optimal crossover rate appears to de- level images of a scene taken at different times or from
crease as the population size increases. For example, among different vantage points, it is often necessary to determine
the best 10 percent of all GA's with population size 30, the a transformation, or registration, which will map one image,
average crossover rate was 0.88. The best crossover rate the original image, into another, the target image. This
decreases to 0.50 for population size 50 and to 0.30 for registration problem is important in such diverse fields as
population size 80. This is reasonable since, in smaller aerial photography and medical imaging [20], [21]. One
populations, crossover plays an important role in prevent- approach to the image registration problem [10] is to define
ing premature convergence. In summary, the performance a parameterized class of transformations and to apply a
of GA's appear to be a nonlinear function of the control GA to the task of searching this class for an optimal
parameters. However, the available data is too limited to transformation, i.e., a transformation which maps the origi-
confirm or disconfirm the existence of discontinuities or nal image into the target image. The response surface for a
multiple local optima in the performance space of GA's. It given pair of images is a function of the transformation
would be interesting to compare the performance of other and corresponds to the average gray-level differences be-
nonlinear search techniques in optimizing the performance tween corresponding pixels in the transformed original
of GA's. image and the target image. This class of problems appears
to be difficult for conventional nonlinear programming
algorithms due to the inevitable presence of many local
V. VALIDATION minima [14].
Experiments were conducted to compare the effective-
In order to validate the experimental results, the three ness of the algorithms GAs, GA1, and GA2 for a sample
algorithms GAS, GA1 and GA2 were applied to an optimi- registration problem consisting of a pair of carotid artery
zation problem which was not included in the experimental images in which patient motion produced significant mo-
Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.
128 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-16, NO. 1, JANUARY/FEBRUARY 1986

tion artifacts. Each experiment consisted of five runs of a REFERENCES

GA on the pair of images, each run evaluating 5000
candidate transformations. Fig. 7 compares the online per- [1] A. D. Bethke, "Genetic algorithms as function optimizers," Ph. D.
thesis, Dept. Computer and Communication Sciences, Univ. of
formance of GAS with GA1. GA1 shows a small but Michigan, 1981.
statistically significant improvement in online performance [2] L. B. Booker, "Intelligent behavior as an adaptation to the task
over GAS on this problem. Fig. 8 shows that GA2 pro- environment," Ph. D. thesis, Dept. Computer and Communication
Sciences, Univ. of Michigan, Feb. 1982.
duces no significant difference in offline performance over [3] A. Brindle, "Genetic algorithms for function optimization," Ph. D.
GAS. These results are consistent with the results in Ex- thesis, Computer Science Dept., Univ. of Alberta, 1981.
periments 1 and 2, and indicate that those results may be [4] D. J. Cavicchio, "Adaptive search using simulated evolution,"
Ph.D. thesis, Dept. Computer and Communication Sciences, Univ.
generally applicable to other optimization problems. of Michigan, 1970.
[5] L. Cooper and D. Steinberg, Methods of Optimization. Phila-
delphia: W. B. Saunders, 1970.
VI. CONCLUSION [6] K. A. DeJong, A nalysis of the behavior of a class of genetic adaptive
systems, Ph.D. thesis, Dept. Computer and Communication Sci-
Experiments were performed to search for the optimal ences, Univ. of Michigan, 1975.
GA's for a given set of numerical optimization problems. [7] , "Adaptive system design: A genetic approach," IEEE Trans.
Syst., Man, Cyber. vol. SMC-10, no. 9, pp. 566-574, Sept. 1980.
Previous experiments show that the standard GA, GAS, [8] L. C. W. Dixon, "The choice of step length, a crucial factor in the
outperforms several classical optimization techniques on performance of variable metric algorithms," in Numerical Methods
the task environment. Thus, one goal of our experiments for Nonlinear Optimization, F. A. Lootsma, Ed. New York:
Academic, 1972.
was to identify GA's which are at least as good as GAS. [9] W. Farrell, Optimization Techniques for Computerized Simulation.
Both experiments succeeded in identifying control parame- Los Angeles: CACI, 1975.
ters settings that optimize GA's with respect to the de- [10] J. M. Fitzpatrick, J. J. Grefenstette, and D. Van Gucht, "Image
registration by genetic search," in Proc. of IEEE Southeastcon '84,
scribed performance metrics. The experimental data also pp. 460-464, Apr. 1984.
suggests that, while it is possible to optimize GA control [11] L. J. Fogel, A. J. Owens, and M. J. Walsh, Artificial Intelligence
parameters, very good performance can be obtained with a Through Simulated Evolution. New York: Wiley and Sons, 1966.
[12] M. P. Fourman, "Compaction of symbolic layout using genetic
range of GA control parameter settings. algorithms," Proc. Intl. Conf. on Genetic Algorithms and their
The present approach is limited in several ways. First, it Applications, pp. 141-153, July 1985.
was necessary to choose a particular parameterized sub- [13] D). R. Frantz, Non-linearities in genetic adaptive search, Ph.D. thesis,
Dept. Computer and Communication Sciences, Univ. of Michigan,
class of GA's to explore. In particular, we have neglected 1972.
other recombination operators such as multipoint crossover [14] W. Frei, T. Shibata, and C. C. Chen, "Fast matching of non-sta-
and inversion and other strategies such as the inclusion of tionary images with false fix protection," Proc. 5th Intl. Conif. Patt.
Recog., vol. 1, pp. 208-212, IEEE, 1980.
a "crowding factor" [6]. Second, the GA's we considered [15] D. Goldberg, "Computer-aided gas pipeline operation using genetic
were essentially unconstrained optimization procedures. As algorithms and rule learning," Ph.D. thesis, Dept. Civil Eng., Univ.
of Michigan, 1983.
previously noted, there are ways to incorporate constraints [16] J. J. Grefenstette, R. Gopal, B. J. Rosmaita, and D. Van Gucht,
into GA's but it remains for future research to determine "Genetic algorithms for the traveling salesman problem," in Proc.
how the presence of constraints affects the optimal control Intl. Conf. Genetic Algorithms and their Applications, pp. 160-168,
parameters. Finally, the metalevel experiments represent a July 1985.
[17] J. H. Holland, Adaptation in Natural and Artificial Systems, Univ.
sizable number of CPU hours. It is encouraging that the Michigan, Ann Arbor, MI, 1975.
results appear to be applicable to a wide class of optimiza- [18] "Escaping brittleness," in Proc. Int. Machine Learning
tion problems. Workshop, pp. 92-95, June 1983.
[19] R. B. Hollstien, Artificial Genetic Adaptation in Computer Control
An alternative approach to the optimization of GA's Systems, Ph.D. Thesis, Dept., Computer and Communication Sci
would be to enable the GA to modify its own parameters ences, Univ. of Michigan, 1971.
dynamically during the search. However, for many optimi- [20] R. A. Kruger and S. J. Riederer, Basic Concepts of Digital Subtrac-
tion Angiography. Boston: G. K. Hall, 1984.
zation problems the number of evaluations which can be [21] James J. Little, "Automatic registration of landsat MSS images to
performed in a reasonable amount of time would not allow digital elevation models," in Proc. IEEE Workshop Computer Vi-
sion: Representation and Control, pp. 178-184, 1982.
the GA enoughb evaluations to modify its search techniques [22] E. Pettit and K. M. Swigger, "An analysis of genetic-based pattern
to any significant degree. Therefore, the experiments de- tracking,' in Proc. National Conf. on Al, AAAI 83, pp. 327-332,
scribed above are important in that they identify ap- 1983.
proximately optimal parameter settings for the two perfor- [23] H. H. Rosenbrock, "An automatic method for finding the greatest
or least value of a function," Computer J., vol. 3, pp. 175-184, Oct.
mance measures considered. The data also suggests several 1960.
new tradeoffs among the control parameters which may [24] J. Shekel, "Test functions for multimodal search techniques," in
lead to further theoretical insights concerning the behavior Fifth Ann. Princeton Conf. Inform. Sci. Syst., 1971.
[25] S. F. Smith, "Flexible learning of problem solving heuristics through
of genetic algorithms. adaptive search," in Proc. of 8th IJCAI 1983.

Authorized licensed use limited to: University of Central Florida. Downloaded on January 22,2024 at 15:12:46 UTC from IEEE Xplore. Restrictions apply.