Western Michigan University

ScholarWorks at WMU

Dissertations Graduate College

1-2011

Multi-Scale Optimization Using a Genetic Algorithm

Jumana Ali Alshawawreh
Western Michigan University

Follow this and additional works at: https://scholarworks.wmich.edu/dissertations

Part of the Computer Engineering Commons, and the Electrical and Computer Engineering Commons

Recommended Citation
Alshawawreh, Jumana Ali, "Multi-Scale Optimization Using a Genetic Algorithm" (2011). Dissertations.
337.
https://scholarworks.wmich.edu/dissertations/337

This Dissertation-Open Access is brought to you for free

and open access by the Graduate College at
ScholarWorks at WMU. It has been accepted for inclusion
in Dissertations by an authorized administrator of
ScholarWorks at WMU. For more information, please
contact wmu-scholarworks@wmich.edu.
 THE GRADUATE COLLEGE
WESTERN MICHIGAN UNIVERSITY
KALAMAZOO, MICHIGAN

Date 02/11/2011

WE HEREBY APPROVE THE DISSERTATION SUBMITTED BY

Jumana AN Alshawawreh

ENTITLED Multi-Scale Optimization using a Genetic Algorithm

AS PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE

DEGREE OF Doctor of Philosophy

Electrical and Computer Engineering

UM-
(Department) Dr. Frank Severance
Dissertation Review Committee Chair

Electrical and Computer Engineering

7)n.~. I M.JL
(Program) Dr. Damon Miller
Dissertation Review Committee Member

IfkjJU^L^
Dr. Melinda Koelling
Dissertation Review Committee Membet

APPROVED

Date J^lM
 MULI-SCALE OPTIMIZATION USING A GENETIC ALGORITHM

Jumana AH Al-Shawawreh, Ph.D.

Western Michigan University, 2011

The Multi-Scale Algorithm (MSA) is a new optimization procedure based on a

genetic approach. It can be considered as a stochastic optimization method requiring

only function evaluations and random searches to locate an optimal solution to an

optimization problem. The algorithm is called multi-scale since it has the ability to

use a large scale in the initial stages then to use a scale refinement near the optimal

point. This is done by using membership functions as a random generator in the

mutation operation. Polar coordinate variables are used to perform the MSA for

functions of two variables, where the efficiency in locating an optimal solution is

validated in several examples. The concept of the polar coordinate variables is

generalized for functions of N variables.

Two new adaptive penalty methods are proposed to solve constrained

optimization problems. The main idea in those methods is to have a balanced

weighting of both terms in an unconstrained objective function; i.e. the original

objective function and the penalty function in order to locate a feasible optimal

solution.

Several unconstrained benchmark functions with different features were used to

investigate the capability of Multi-Scale Algorithm to locate global optimal solutions.

 Both engineering and classical mathematical optimization benchmarks problems

were used to test the performance of the proposed penalty method. These problems

include the design of a welded beam, design of a pressure vessel, minimization of the

weight of a tension/compression spring, minimization of the weight of a speed

reducer and Himmelblau's nonlinear optimization problem.

Finally, the Multi-Scale Algorithm was used to find the optimal power flow in a

power system network. The aim is to minimize the total cost of generation power

plants subject to several constraints in order to maintain power system network

stability. The optimal power flow of the IEEE-26 bus power system network and

IEEE-30 bus power system network were successfully calculated using the Multi-

Scale Algorithm.
MULTI-SCALE OPTIMIZATION USING A GENETIC ALGORITHM

by

Jumana Ali Alshawawreh

Dissertation
Submitted to the
Faculty of The Graduate College
in partial fulfillment of the
requirements for the
Degree of Doctor of Philosophy in
Electrical and Computer Engineering
Advisor: Frank Severance, Ph.D.

Western Michigan University

Kalamazoo, Michigan
April 2011
 UMI Number: 3455090

All rights reserved

INFORMATION TO ALL USERS

The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

UMI
Dissertation Publishing

UMI 3455090
Copyright 2011 by ProQuest LLC.
All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.

uest A ®

ProQuest LLC
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, Ml 48106-1346
 Copyright by
Jumana Ali Al-Shawawreh
2011
 DEDICATION

To my father, the greatest person in my life, you have inspired me alive and dead, for

your soul I dedicate this dissertation, to

Ali Khleaf AlShawawreh

 ACKNOWLEDGMENTS

I would like to start with my advisor Dr. Severance for his directions and

support and, of course, for his patient, without his help this works will never

complete. I appreciate his understanding in the most of the difficult times through my

studying.

I would like to thank my committee members; Dr. Miller, Dr. Doung and Dr.

Koelling for their reviews and useful feedback.

I would like to thank my family for their support, especially my mother when

I hear her voice this gives me the power to continue, and when I look to my daughters

Shadin and Areen faces this was enough to forget all the difficult times. Great thanks

to my husband, Hisham, and my brother, Ahmad, for their endless support and help.

Finally, I would like to thank my professors and my colleagues and everyone

who works in the ECE department in Western Michigan University, they really make

me feel that I am with my family.

Jumana Ali Al-Shawawreh

n
 TABLE OF CONTENTS

ACKNOWLEDGMENTS ii

LIST OF TABLES viii

LIST OF FIGURES x

CHAPTER

1. INTRODUCTION 1

1.1 General 1

1.2 The Traditional Genetic Algorithm 3

1.2.1 Coding/Decoding Operation 5

1.2.2 Fitness Function Evaluation 7

1.2.3 Offspring Generation Operations 8

1.2.3.1 Crossover Operation 8

1.2.3.2 Mutation Operation 10

1.2.3.3 Reproduction Operation 10

1.2.4 Termination Criteria 11

1.3 Scope and Objective of Thesis 11

2. LITERATURE REVIEW 13

2.1 Historical Background of Optimization Methods 13

2.2 Genetic Algorithm 15

2.2.1 Small Alphabet Cardinality Genetic Algorithms 15

2.2.1.1 Mutation Operation 19

2.2.1.2 Crossover Operation 19

iii
 Table of Contents—continued

CHAPTER

2.2.2 Real Representation in Genetic Algorithms 19

2.2.2.1 Mutation Operation 23

2.3 Summary and Conclusion 28

3. MULTI-SCALE ALGORITHM 30

3.1 Multi-Scale Algorithm 30

3.2 Mutation Operation 30

3.3 Unconstrained Optimization Problems 36

3.3.1 Unconstrained Objective Function with Single

Control Variable 36
3.3.2 Search Direction of N Variables Objective
Functions 44

3.3.3 Unconstrained Objective Function with Two

Control Variables 49

3.3.3.1 Rosenbrock Function 49

3.3.3.2 Goldstein and Price Function I 52

3.3.3.3 Eason and Fenton's Gear Train Inertia

Function 53
3.3.3.4 Multiple Local and Single Global Minimum
Function 55

3.3.3.5 Ackley Function for Two Variables 55

3.3.4 Unconstrained Optimization Function with N

Control Variables 57

3.3.4.1 Powell Quadratic Function 57

3.3.4.2 Wood Function 58

iv
 Table of Contents—continued

CHAPTER

3.4 Summary and Conclusions 58

4. OPTIMIZATION WITH CONSTRAINTS 60

4.1 Introduction 60

4.2 Penalty Method 63

4.2.1 Death Penalty Function 66

4.2.2 Static Penalty Function 66

4.2.3 Dynamic Penalty Function 67

4.2.4 Adaptive Penalty Function 68

4.3 Proposed Method 68

4.4 Test Functions 76

4.5 Constraint Handling Methods by Genetic Algorithm 79

4.5.1 Method* 1 79

4.5.2 Method#2 80

4.5.3 Method* 3 81

4.5.4 Method#4 82

4.5.5 Method#5 82

4.6 Results and Comparison 82

4.7 Engineering Optimization Problems 88

4.7.1 Design of Welded Beam 88

4.7.2 Design of a Pressure Vessel 89

4.7.3 Minimization of the Weight of a Tension/

Compression Spring 90

v
 Table of Contents—continued

CHAPTER

4.7.4 Minimization of the Weight of a Speed Reducer 91

4.7.5 Himmelblau's Nonlinear Optimization Problem 93

4.7.6 Results of the Engineering Optimization Problems

Using MSA and Discussion 93

4.8 Summary and Conclusions 97

5. OPTIMAL POWER FLOW IN POWER SYSTEM NETWORK 100

5.1 Introduction 100

5.2 Power Flow in Power System 102

5.3 Newton Raphson Method for Solving Algebraic System

of Equations 108
5.4 Application of Newton Raphson Method to Solve
Power Flow Equations 109
5.5 Optimal Power Flow 115

5.6 Optimal Power Flow Solution Using Lagrange Multiplier

Function 120

5.7 Optimal Power Flow of IEEE-26-Bus Power System

Network 124

5.8 Optimal Power Flow of IEEE-30-Bus Power System

Network 127

5.9 Summary and Conclusions 132

6. FUTURE WORK AND CONCLUSIONS 134

6.1 Conclusions 134

6.2 Future Work 137

REFERENCES 138

vi
 Table of Contents—continued

APPENDICES

A. Specifications of the IEEE-26-Bus Power System Network 145

B. Specifications of the IEEE-30-Bus Power System Network 148

C. Pseudo-Code for the Optimal Power Flow of the IEEE-30-Bus Power

System Network Using MSA with the Proposed Penalty Method 151

VII
 LIST OF TABLES

1-1 Gene's representation of the control variables 6

2-1 Octal vs binary codes 17

3-1 The group that has the highest probability of being selected

for several iteration intervals 32

3-2 Pseudo code of the mutation function 34

3-3 Pseudo code of the required functions of the mutation operation 35

3-4 Pseudo code of Problem* 1 38

3-5 Ten independent runs of MSA for problem #2 for

several subintervals 40
3-6 Ten independent runs of the MSA for different initial points 41

3-7 The optimal solution of problem #2 for 100 independent runs

of genetic function in MATLAB® and MSA 42

3-8 Solutions of problem #3 for 10 independent runs of MSA for

different initial points 43

3-9 Pseudo code of the Rosenbrock function 51

3-10 Optimal solutions of Wood function using MSA 58

4-1 Minimum function values obtained using penalty function with

different values of c 65
4-2 Mean value of 10 trials of the genetic function in MATLAB®
for solution of a constrained optimization problem 70

4-3 Optimal solution location for c = 100 71

4-4 Pseudo code of MSA with the proposed penalty method 75

4-5 Solutions of tests functions using penalty methods [42] and MSA
with proposed penalty method 83

vin
 List of Tables—continued

4-6 Solutions of the second test function using the MSA with the
modified penalty method 85

4-7 Pseudo code of the MSA with the modified penalty proposed
method 87

4-8 Design of welded beam result using MSA 94

4-9 Design of a pressure vessel result using MSA 95

4-10 Minimization of the weight of a tension/compression

spring result using MSA 95

4-11 Minimization of the weight of a speed reducer using MSA 96

4-12 Minimization of the Himmelblau's nonlinear problem using MSA 97

5-1 Power flow of the three-bus power system network using

Newton Rraphson method [48] 114

5-2 Fuel cost of the generation buses of the five-bus network [48] 118

5-3 Given data of the five-bus power system network 118

5-4 Power flow of the five-bus power system network 119

5-5 Optimal power flow of the five-bus power system network 120

5-6 Result of optimal power flow of IEEE-26-bus power system network

using Lagrange multiplier method [48] 125
5-7 Result of optimal power flow of IEEE-26-bus power system
network using MSA 126

5-8 Results of optimal power flow of IEEE-30-bus using MSA 128

5-9 Results of optimal power flow of IEEE-30-bus using different

optimization methods [50] and MSA 129
5-10 The optimal power flow of IEEE-30-bus using several
optimization methods 132

ix
 LIST OF FIGURES

1.1 Flow chart of traditional genetic algorithm 4

1.2 Crossover operations, (a) one point crossover point, (b) two-point
crossover, (c) cut and splice crossover and (d) uniform
crossover operations [4] 9

3.1 Random generator of the digit location of the mutation operation 32

3.2 Flowchart of the random digit selector 33

3.3 Convergence of x versus f(x) = 3x4 + 4x2 - 12x2 for different

initial points 37

3.4 The convergence of the an arbitrary initial point to the global minimum
point of the function f(x) = -sin(— ) and (b) a zoomed
x +2x + 2
area around the global minimum point 39
3.5 The sequence of the search x-values of function / ( x ) = x2 + sin(lOOx)
starting from arbitrary initial point 44

3.6 Plot of the function f(x,y) = (x-2)2+(x-2y)2, (x*,y*)is the

location of the minimum solution of the objective function 46

3.7 Contour plot of the function f(x) = (x - 2)2 + (x - 2y)2 and the
sequence of the success points starting from (x 0 , yQ) to
minimum solution (x*,^*)using cyclic method 46

3.8 Contour plot of the function f{x) = (x - 2) 2 + (x - 2y)2 and the sequence
of the success points starting from (x 0 , y0) to the minimum solution
(x' ,y*) using the Hook and Jeeves method 47

3.9 Plot of Rosenbrock function f(x) = \00(y - x 2 ) 2 + ( x - l ) 2 50

3.10. Contour plot of the banana function for different initial points with
sequences of search points, (a) x0 = 0, y0 = 0 (b)x0=-l, 50

x
 List of Figures—continued

3.11 Contour plot of Goldstein and Price Function I 52

3.12 Plot of the sequence of points of Goldstein and Price Function I for
different initial points, (a) (1.2,0.8) (b) (1.8,0.2) (c) (-0.6,-0.4) 53

3.13 Contour plot of Eason and Fenton's gear train inertia function 54

3.14 Solutions of Eason and Fenton's gear train inertia function found
using MSA for different initial points 54

3.15 Surface plot of f(x,y) = 20 + x2 + y2 -10cos(2nx) -10cos(2ny) .. 55

3.16 Contour plot of the function f(x,y) = 20 + x2 + y2 -10cos(2nx)-

10cos(2ny) 56

3.17 Surface plot of the Ackley function 56

3.18 Contour plot of Ackley function including a point sequence provided

by the MSA to locate the global minimum of this function
starting from the point (1,1) 57

4.1 Plot of a Lagrange problem 62

4.2 Constraints plot 69

5.1 Optimal power flow diagram 100

5.2 Bus on the power system network [48] 104

5.3 Three-bus power system network [48] 105

5.4 Optimal power flow stages 117

5.5 Five-bus power system network [48] 118

5.6 One line diagram of IEEE-26-bus power system network [48] 124

5.7 One line diagram of IEEE-30-bus power system network [50] 127

xi
 CHAPTER 1

INTRODUCTION

1.1 General

The genetic algorithm (GA) is a heuristic algorithm widely used to solve

optimization problems. The main idea of the genetic algorithm is to produce and

exchange information based on biological gene function. The variables of the

optimization problem are presented as genes using a coding operation; then the genes

are connected together to create chromosomes. Binary coding is primarily used to

represent chromosomes, where it is required to define the variable limits and

resolution accurately. If the resolution is not defined very well this may cause the

failure of the GA to locate the optimal solution due to the absence of the chromosome

that represents the optimal solution in the possible domain. To overcome this problem

a higher resolution will be necessary; however, increasing the resolution will cause a

tremendous increase of the length of chromosomes. To avoid coding problems, an un-

coded (real) presentation of the variables can be used. In this dissertation a new

Multi-Scale Algorithm (MSA) based on the genetic algorithm is proposed. The Multi-

Scale Algorithm uses a real presentation of the variables and the mutation operation

in the Multi-Scale Algorithm allows investigating a large space at the beginning and a

smooth search near the optimal solution.

Most practical optimization problems are constrained, the optimal search is

restricted inside a subspace described by a set of constraints and that is called a

1
feasible space. One of the methods to solve a constrained optimization problem is to

convert it to an equivalent unconstrained problem and use a penalty method. In the

penalty methods the unconstrained objective function consists of two terms; one of

them is the objective function and the other term is called the penalty function. The

penalty function is a positive function of constraint violations, where the penalty

function evaluation equals zero within the feasible space and is proportional to the

constraint violations outside the feasible space to maintain the solution within. Near

the boundaries of the feasible space the constraint violations may be insufficient to

prevent the search from going outward from the feasible space. For that reason

penalty functions are usually multiplied by a factor called the penalty parameter.

Based on the penalty parameter, the penalty method can be classified as death, static,

dynamic and adaptive penalty functions. In this dissertation two adaptive penalty

methods are proposed to solve constrained optimization problems. The performance

of the Multi-Scale Algorithm was investigated by solving several mathematical and

engineering optimization benchmarks with different features.

The optimal power flow is one of the important optimization problems in power

systems, where it is required to minimize the fuel cost of the generation units and

maintain the stability of the power system network. Usually the power system

network has several generation plants with different fuel costs. The total generation

power must meet the demand power of the customers and the power losses in the

transmission lines. The power flow through the power system network is usually

calculated by solving a set of nonlinear algebraic equations to determine the bus

2
voltage and the power losses in transmission lines. To solve the optimal power flow,

it is required to solve the power flow equations numerically through an optimal

search, to determine the power losses and to investigate the power system stability.

The Multi-Scale Algorithm was used to solve the optimal power flow of the IEEE-30

bus and the IEEE-26 bus power system network.

1.2 The Traditional Genetic Algorithm

The primary procedures in a simple Genetic Algorithm (GA) are coding

operations, fitness function evaluation and production operations [1]. The first step in

simple GA is to convert the control variables (optimization problem variables) to

binary string called genes. Then the genes are concatenated together to construct

chromosomes. The GA usually starts with an initial chromosome population of

sizeP . The size of the population depends on the nature of the optimization problem.

The next step is to evaluate the fitness function for each chromosome in the

population. Based on the fitness function the chromosomes will be selected to

generate offspring, where the chromosome that has the highest fitness function

evaluation will have the most useful information to locate the optimal solution. The

offspring can be generated by using three operations; mutation, crossover and

reproduction. The generation of offspring will be terminated when converge criteria

are met. The flow chart of the traditional GA is shown in Figure 1.1.

3
 Select initial population of size
P randomly

+
Create the chromosomes using
coding operation

-t"
Evaluate the fitness function of
each chromosome

Generate offspring from current

population using production
operations; crossover, mutation
and reproduction operations.
1
Evaluate the fitness function of
each chromosome

Terminate the algorithm

Figure 1.1. Flow chart of traditional genetic algorithm

4
1.2.1 Coding/Decoding Operation

The binary representation of the genes is usually done by a coding operation.

The numbers and the length of genes depend on the control variables' ranges and

their resolutions.

The length of the gene that represents the control variable x, depends on the

control variable range [a, ,b, ] and its resolution rl. The length of the gene Lt is

L. lOg2 ~ r ~ (1-1)
. I > )

The length of the gene is proportional to the control variable range and

inversely proportional to its resolution, where the length of the gene increases as the

control variable range is increased. If higher resolution is required then the length of

the gene is increased. For example, the lengths of the gene for the variable

x, e [-1,4] for different resolutions 0.1, 0.01, 0.001 and 0.0001 are 6, 9, 13 and 16

respectively.

The number of possible genes that represents the control variable x, is equal

b —a
to — '-. For example, if the control variable x, e [-1,4] and its resolution is 0.1,

then all the possible genes of the control variable will be as shown in Table 1-1.

5
 Table 1-1 Gene's representation of the control variables
Control variable Gene of control variable

-1 000000
-0.9 000001

-0.8 000010

4 110010

The genes in a simple GA are connected together to create the chromosomes.

For example, consider the optimization problem with four variables:

x,e[-l,4], r, =0.1
x 2 e[2,5], r 2 =0.01
x3 e [-2,2], r3 =0.1
x 4 e[0,10], r 4 =0.01

Suppose that the values of the control variables at the /"" generation

are xy = 2, x2 = 2.2, x3 - -2, x4 = 3 and x4 = 3, then the genes that represent the

control variables are G, =011110, G2 =00001010, G3 =000000 and

G4 = 0100101100. The chromosome that represents the genes at this generation will

be 0111100000101000000000100101100. The decoding operation is the operation

that converts the genes in chromosomes to real values of the control variables. The

decoding operation is necessary to evaluate the fitness function of the chromosomes.

6
1.2.2 Fitness Function Evaluation

The simple GA starts from an initial population of chromosomes of size N.

The chromosomes in the initial population can be selected randomly. Each

chromosome has the possibility to be chosen as a parent of the next generation. The

probability of selecting the chromosome depends on the fitness function evaluation

fi of the chromosome. The fitness function is a function that measures the usefulness

of the chromosome to direct the search toward the global optimal solution. For

example, if it is required to minimize the objective function, then the chromosome

that yields the least objective function evaluation will have the highest fitness

function evaluation. The fitness function is a positive function since it will be used to

determine the probability of the chromosomes to be chosen to generate the next

offspring.

The selection of the chromosomes to generate the next offspring depends on

the fitness function evaluations of chromosomes. Roulette wheel selection is used

widely in GAs [2]. The probability of the chromosome to be selected in a size k

population is:

P , = ^ - d-2)

The chromosomes will be selected randomly as in a roulette wheel. Each

chromosome will be considered as a packet and the size of the packet is proportional

7
to the probability of the chromosome. In order to select TV chromosomes from the

current population it will be required to play N games of the roulette wheel.

1.2.3 Offspring Generation Operations

The next offspring is usually generated based on the current population using

three operations: crossover, mutation and reproduction operations. The generation

operations can be classified into primary and secondary operations. The primary

operation will be considered as the main operation in generating the offspring, where

most of offspring will be generated using this operation. On the other hand, the

secondary operation has a lower rate than the primary operation to generate offspring.

1.2.3.1 Crossover Operation

A crossover operation is considered as the primary operation to generate

offspring in a simple GA, where most offspring are generated using the crossover

operation. In this operation two chromosomes are selected based on the fitness

function evaluation to be the parents of the next chromosomes. There are several

kinds of crossover operations: the one point, two-point, cut and splice and uniform

crossover operations [3]. The simple single point crossover operation is to choose two

chromosomes from the current population to be the parents of the offspring, then

divide them into two random segments. The offspring are generated by exchanging

the segments as shown in Figure 1.2-a. The single point crossover operation is

mostly used in the simple genetic algorithm. The two-point crossover operation

selects a random segment with same length of each chromosome. Then the segments

8
between the parents are exchanged to generate the offspring as shown in Figure 1.2-b.

In the cut and splice cross over operation each chromosome is divided into two

segments, but it is not necessary that both chromosomes are divided into equal

segment lengths; the population will have chromosomes with different lengths as

shown in Figure 1.2-c. The offspring in the cut and splice crossover operation will

have different lengths from their parents. Figure 1.2-d shows the uniform crossover

operation. In the uniform crossover operation the bits are chosen randomly from each

parent to be combined to generate offspring. Regardless of the crossover method, the

result is that two parents produce two offspring with a mixture of characteristics

passed to the next generation

Parent # 1 I I I I I I I I I I I I I Parent * 1 I I 1 I I I I I I I TTl

Parent # 2 Parent * 2

Offspring # 1 I I I I Offspring fr 1

Offspring # 2 • • • • I I Offspring # 2

(a) O)

Parent £ 1 I T~ I I I I I I I I
Parent * 1 I I I I I I I I I I I I I I I I

Parent h 2 Parent & 2

Offspring rt 1 I I I Offsprmg * 1 I I Ml

Offspring U 2 • • • I I Offspring £ 2 I • I
(d>
(<0

Figure 1.2. Crossover operations, (a) one point crossover point, (b) two-point
crossover, (c) cut and splice crossover and (d) uniform crossover operations [4]

9
1.2.3.2 Mutation Operation

The mutation operation is a secondary generation operation in a simple GA

[5]. The main aim of the mutation operation is to decrease the probability of the GA

staying at local optimal solution. The mutation operation increases the randomness in

the GA to increase the probability of the GA to test a large feasible region for an

optimal solution. A simple mutation operation selects one chromosome from the

current population based on the fitness function evaluation to be the parent, and

selects the location of the mutated bit in the parent randomly. The next offspring will

be the same as its parent with a complemented bit value in the mutated bit location.

For example, if the current chromosome is 011 land the mutated bit location is 3, then

the bits of the next chromosome will be the same as the current chromosome except

the 3th bit will be changed from one to zero. The next chromosome will be 0101.

1.2.3.3 Reproduction Operation

A reproduction operation selects the chromosomes with highest fitness

function evaluation and makes copies of these chromosomes. If the reproduction

operation is used to generate the offspring, then the next offspring will be a copy of

its parent. The reproduction operation is used to increase the numbers of the

chromosomes that have a high fitness function evaluation in the current population.

This will increase the probability that the chromosomes with the highest fitness

function evaluation will be selected to generate the next offspring [5].

10
1.2.4 Termination Criteria

The termination criterion can be classified based on two possible criterions;

converge and diverge criteria. The convergence criteria terminate a GA, and the

algorithm in this case finds a potential solution. The divergence criteria terminate the

GA when the algorithm fails to locate optimal solution.

A GA converges to an optimal solution when the chromosomes converge to a

single solution. The chromosomes converge to a single solution when the sum of the

standard deviation of the individuals (chromosome values) is less than a threshold

value [3].

GA terminates the search when the number of the generation exceeds the

maximum number of generations and when there is no progress in a certain number

of iterations [3].

1.3 Scope and Objective of Thesis

This dissertation contains 6 chapters; Chapter 1 has discussed the general

background of this dissertation and the simple genetic algorithm.

In chapter 2, a literature review of the binary coded and the real coded genetic

algorithms are studied.

In Chapter 3, the Multi-Scale Algorithm (MSA) based on the genetic

algorithm is presented. Offspring are generated mainly using a new mutation

operation, where it gives the possibility to have multi-scaled searches toward an

11
optimal solution. Several unconstrained optimization problem with different features

were solved using the Multi-Scale Algorithm successfully.

Chapter 4 focuses on solving constrained optimization problems. Two penalty

methods are proposed to solve constrained optimization problems. The first proposed

method is used to solve a set of mathematical benchmark optimization problems. The

second proposed method was used to solve a set of engineering optimization

problems.

In Chapter 5, the power flow and optimal power flow of a power system

network are introduced. The Multi-Scale Algorithm was used to solve the optimal

power flow of the IEEE-26 bus and the IEEE-30 bus power system networks.

In Chapter 6, Conclusions and future works are discussed.

12
 CHAPTER 2

LITERATURE REVIEW

2.1 Historical Background of Optimization Methods

The first optimization methods were related to Newton, Lagrange, and Cauchy

[6]. Lagrange derived an optimization method to solve constrained optimization

problems by adding unknown multipliers (Lagrange multipliers) to incorporate

constraints in the objective function [6]. For example, if it is required to

minxf(x),xeR" (2.1)

subject to a set of equality and inequality constraints:

C,(JC)<0

C2(x)<0
(2.2)

C m (x)<0

then using the Lagrange method the optimization problem yields

mmXAL(X,A)
(2.3)
L(X,Z) = f(X) + A]Cl+A2C2+... + AmCm

where x = [xx,x2,---,xn] and A = [A],A1,---,Am].

The steepest descent method was used to solve unconstrained optimization

problems by Cauchy [6]. Many numerical methods of unconstrained optimization

13
problems were developed because of the existence of high-speed digital computers

[6]. The numerical optimization methods can be divided into linear, nonlinear,

geometric and stochastic methods [6].

• Linear programming methods are used to solve a linear objective function

with linear equality and inequality constraints. The simplex method by Danzig

in 1947 is the most efficient and most famous linear programming method [6].

• Nonlinear programming was founded because of the development of the

principle of optimality for dynamic programming problems by Bellman in

1957. The sufficient and necessary conditions of nonlinear programming

methods were developed by Kuhn and Tucker in 1951. The work of

Zoutendijk, Rosen, Carroll and Fiacco in nonlinear programming was very

important [6].

• Geometric programming (integer programming) was developed in the 1960s

by Duffin, Zener, and Peterson [2]. Geometric programming is an

optimization problem of the form:

Minimize /„ (X) and subject to ft(X) <\,i = \,...,Iand gk(X) = \,k = l,...,K.

• Stochastic programming methods that use random search for solving

optimization problems were developed by Dantzig, Charnes and Cooper [6].

Many real world optimization problems cannot be solved using deterministic

methods because of the large search space, large number of constraints, multiple local

14
optima and the difficulty of finding explicit formula for the objective function [6]. For

these reasons many heuristic methods such as simulated annealing by by S.

Kirkpatrick, C. Gelatt and M. Vecchi [7], cross-entropy method by Rubinstein and

Kroese [8], random search by Zhigljavsky [9] , basin hopping technique (a.k.a. Monte

Carlo) with minimization [10,11], stochastic tunneling [12] and genetic algorithm

[12,13] were developed to solve optimization problems. The random searches in

heuristic methods increase the probability for finding the global maxima/minima of

optimization problems.

2.2 Genetic Algorithm

The control variables of the optimization problem are usually called genes in

genetic algorithms (GA), where these genes are joined linearly together to create the

chromosomes. The representation of the control variables as genes is usually done by

using a coding operation. Based on the coding operation the genetic algorithms can be

classified into two types; small alphabet cardinality genetic algorithms and higher

cardinality genetic algorithms [14].

2.2.1 Small Alphabet Cardinality Genetic Algorithms

The binary representation of chromosomes was used in most of the early work

of the genetic algorithm based on the Schema Theorem. The Schema Theorem was

created by Holland [15] and it recommended the use of small alphabet cardinality.

The Schema theorem is "short schema with better than average costs occur

15
exponentially more frequently in the next generation. Schema with worse than

average costs occur less frequently in the next generation" [16, page 534].

The term schema is used to describe a subset of similar strings. For example,

for the binary string of length 4, the schema IddO is the set of binary strings with

length 4 that have Is at position 1 and Os at position 4. The drepresents a don't care

case, which means positions 2 and 3 can be either 0 or 1. The schema IddOis

{1000,1010,1100,1110}.

The defining length of the schema is the distance between the first and last

non d elements in the schema. The order of the schema is defined as the total number

of non d elements in the schema (i.e the total numbers of zeros and ones in the

schema). The length of the schema IddOldOd is 6 (i.e 7-1=6) and the order is 4 [14].

Small alphabet cardinality, such as a binary has the largest number of

schemata available for the genetic procedure; based on that the small alphabet

cardinality will have the greatest information[14]. The information is used to decide

the direction of the search. The Schema theorem states that the schema above the

average fitness increases their frequencies in the population at each generation at an

exponential rate [14]. Goldberg [14] used a simple example to show that a small

alphabet has the greatest information towards the optimal, suppose that the binary and

octal coding was used to minimize a function and the function evaluation of certain

chromosome is listed in Table 1.1.

16
Table 2-1 Octal vs binary codes

Binary Code Octal Code Function value

101 5 1

Oil 3 10

000 0 12

111 7 2

Based on Table 2.1, the binary numbers 101 and 111 have the least function

values, which yield that the schema \d\ will be a useful schema to locate the

minimum value. While in the octal code, it is difficult to determine the useful schema

to locate the minimum solution, since each number in the octal coding represents

itself [14].

Altenberg [17] illustrated that it is a mistake to conclude that the quality of the

search is related completely to the growth of schema in GA [17]. He showed that

using an example of the needle-in-a-haystack fitness function," For example, consider

a population with a needle-in-a-haystack fitness function, where exactly one genotype

(the "needle") has a high fitness, and all the other genotypes in the search space (the

"hay") have the same low fitness. Consider a population in which the "needle" has

already been found. The needle will tend to increase in frequency by selection, while

recombination will most likely generate more "hay". The Schema Theorem will still

be seen to operate, in that short schemata with above average fitness (those schemata

containing the needle) will increase in frequency, even though the fitness of new

17
instances of the schemata (more hay) will not be any more likely to have the high

fitness of the needle." [17, page 24].

Simple binary coding needs to define the length of the gene that represents the

control variable, where the length of the gene depends on the range of the control

variables and their resolution. The length of chromosomes increases as the number of

control variables increases or when a higher resolution is required, where the length

of the chromosome becomes tremendously large. One of the drawbacks of the simple

binary coding is the large lengths of chromosome, which slows the speed of the

algorithm [18]. The other drawback of the simple binary representation is the

Hamming cliff problem; it is a term that refers to the problem of representation of two

adjacent numbers in binary, where it is required to change all the bits to get the

adjacent number [18]. For example, if the binary code of the number 63 is 0111111

and the binary representation of 64 is 1000000 then the Hamming distance is 7 [18].

To overcome the Hamming cliff, gray coding is widely used in genetic

algorithms where in gray coding any two successive numbers are differing only by

one bit [19]. Several empirical comparisons between binary coding and gray coding

proved statistically the advantage of using the gray coding over the binary coding

[19]. The other advantage of using Gray coding is that the mutation operation in this

code allows for mostly incremental changes. A single bit-change using mutation

operation can cause a big change and lead to new properties [19].

18
2.2.1.1 Mutation Operation

The mutation operation is considered as a secondary operation compared to

the crossover operation to produce offspring. Even so there is a still strong argument

as to which is more important to generate offspring, crossover or mutation operations.

The mutation operation increases the capability of the genetic algorithm to locate the

global solution of the optimization problem because the randomness in mutation

operations decreases the probability of the genetic algorithm from getting stuck at a

local optimal solution [1]. It gives non-zero probability for all points in the search

space to be chosen. For binary genetic algorithms the rate of mutation operation is

very small because the mutation operation causes small changes in the chromosomes

and especially in the case of the Hamming cliff. In Gray coding, the mutation

operation is more effective, where changing one bit causes a change in the entire

chromosome, so the rate of the mutation operation is increased [19].

2.2.1.2 Crossover Operation

The crossover operation is considered to be the primary operation of small

cardinality genetic algorithms, where it usually has the highest rate to generate the

offspring. There are several kinds of crossover operations: the one point, two-point,

cut-and-splice and uniform crossover operation ( See section 1.2.3.1) [3].

2.2.2 Real Representation in Genetic Algorithms

A brief history of real coded genes in evolutionary optimization methods was

given by Goldberg [14], where he mentions that the first real coded genes in genetic

19
algorithms was suggested by Weinberg [20] in his dissertation, where Weinberg gave

brief details of the real coded genes but without any simulation results.

Goldberg [14] has a good explanation of the benefits and limitations of using

either the small alphabet or large alphabet coded genetic algorithms by listing the

advantages and disadvantages of both of them. He summarized the advantages of

using real coded genetic algorithm as follows:

The mutation operations in real coded genetic algorithms perform little

perturbation on the current genes, which allow the genetic algorithm to do a

hill climbing in the underlying decision space.

The speed of convergence near to optimal solution was increased.

It reduces the dimensionality of the problem.

One of the advantages of real code representation is that the population statistics

(such as the mean, standard deviation of the fitness function, etc.) can be used as

convergence criterion of the genetic algorithm.

A critique of the schema theorem was offered by Lee Altenberg [17], where he

illustrated why the Schema Theorem cannot distinguish between a good genetic

algorithm and a bad genetic algorithm. He mentions three drawbacks of the Schema

Theorem: " 1 . The Schema Theorem holds even with random representations, which

cannot be expected to perform better than random search, whereas it has been used to

claim that GAs perform better than random search; 2. The Schema Theorem holds

20
even when the schemata defined by a representation may not capture the properties

that determine fitness; and 3. The Schema Theorem extends to arbitrary subsets of the

search space regardless of the kind of genetic operators, not merely the subsets

defined by Holland schemata" [17, page 24].

Collins and Eaton [21] devised an experimental basis of testing four types of

coding schemes: binary code, gray code, E-code and the real code. They concluded

that statistically the real code representation has the best performance.

Tsutsui, Ghosh, Corne and Fujimoto [22] developed two new slightly different

schemes of bi-population. The new schemes perform a balance between exploration

and exploitation at different stages of the search process. This solves so-called highly

epistemic multimodal problems and deceptive functions. They chose these problems

because they are considered as difficult problems for the previous real coded genetic

algorithms. The novel approach is to use separate populations for exploration to avoid

being trapped in local optima. It restarts mechanism and exploitation to focus on the

neighborhood of the so far best solution. They use an adaptive load balancing

mechanism which allocates time between the explorer and exploiter. The difference

between the two population methods is related to as the problem type. The second

method (exploiter) was developed for the problem that has a known optimal value,

where it is required to locate the optimal solution.

Srikanth and Kamala [23] applied the real genetic algorithm to optimize "the

cutting parameters in turning problem" [23, page 189]. They compared their result

with another optimization method which was used to optimize the problem including

21
a binary genetic algorithm. They concluded that the real coded genetic algorithm is

necessary for fast solving the same constrained optimization problem of the cutting

parameters in turning problem.

Real-coded adaptive range genetic algorithms (ARGAs) present a new approach

using dynamic coding proposed by Arakawa and Hagiwara for binary-coded GAs

[24]. It is proposed to solve continuous design spaces [24]. The length of the

chromosome increases rapidly as the number of variables increases or if a high

precision of the control variables is required. The adaptive range genetic algorithm is

used to overcome this disadvantage of the binary coded genetic algorithms. It allows

small length chromosomes and the ranges of the control variables do not have to be

predefined [24]. The ranges of the control variables will move adaptively in each

generation by using the mean value and the standard deviation of the previous

generation.

Oyama, Obayashi and Nakahashi [25] developed the adaptive range genetic

algorithm [ARGA] for floating point presentation. The floating point representation

of the genes was proposed to enhance the adaptive range of a genetic algorithm in

three ways: "The first one is how to select the system parameters and on which

robustness and efficiency of ARGAs largely depend. The second one is the use of

constant intervals even near the center of the normal distributions. The last one is that

since genes represent relative locations, the offsprings become constantly away from

the centers of the normal distributions when the distributions are updated" [25, page

714].

22
The developed scheme of the adaptive genetic algorithm was applied to a

aerodynamic airfoil shape optimization problem, and it was concluded that the real

coded adaptive range genetic algorithm have two advantage compared to the other

genetic algorithm; first it located a better solution and second it ensures the feasibility

of the optimal solution [25].

Self-adaptation "is a phenomenon which makes search algorithms flexible and

close to natural evolution. Any good search algorithm must explore a large search

space in the beginning then narrows when it converges to a solution" [26, page 159].

Based on M. Raghuwanshi and O. Kakde paper [26], it is possible to achieve self-

adaptive behavior in the real coded genetic algorithm with specialized recombination

operators in which the children solutions is proportional to the parent solutions [26].

2.2.2.1 Mutation Operation

Goldberg [14] distinguished between the aims of mutation operation in binary

string genes and real coded genes. In the binary coded genes a mutation operation is a

bitwise complement operator. The mutation operator in real coded genes is used to

perturb the selected control variables to locate a best solution. The mutation operation

in real coded genetic algorithm can easily do hill climbing in the decision space (see

section 2.2.1); while this is difficult for mutation in the binary coded genetic

algorithm, this can cause the genetic algorithm to experience Hamming cliffs. The

importance of mutation operations to generate offspring is increased for higher

cardinality representation genetic algorithms. For that reason the higher cardinality

23
genetic operation has a high mutation rate compared with the binary genetic

algorithm [27].

The population in a real representation of control variables is usually

presented in matrix form. The number of columns is equal to the number of genes that

represent the control variables. The numbers of rows equal to the number of

chromosomes. The population matrix of TV control variables and M chromosomes is

given:

£11 £12 "" SIN

g 21 g
Population= . >2 [" g N
] .

_SM\ &M2 '" SMN _

The simple mutation operation for a real representation of control variables

can be summarized as follows [28]: first determine the number of offspring that will

be generated using a mutation operation (the number of mutated control variables is

equal to the mutation rate multiplied by the population size). Secondly, the location of

the mutated control variables is selected randomly. Finally, the selected control

variables will be changed to values selected randomly from the control variables

ranges.

For example, if the population size of 2 variables optimization problem is 14,

the control variables range are 0 < xx < 3, 2 < x2 < 6 and the mutation rate is 0.2.

The size of the population matrix will be ( 7 x 2 ) . The number of offspring that will

be generated using a mutation operation at each generation will be equal to

24
0.2x14 « 3. If the selected locations of the mutated control variable are (2,1), (3,2)

and (5,2), this means that the first control variable for the mutation operation is

located at the second row and the first column. The second one is located in the third

row and the second column. The third mutated variable is located in the fifth row and

the second column in the population matrix, refer to equation (2.4). The selected

control variables are g2], gn and g52.

'gll gn'
gll gn
831 #32
population matrix £41 #42 (2.4)
£51 #52

#61 #62

.gll £72 _

After the mutation operation, the mutation matrix will be:

gll gn
g'll gn
#31 g\i
population matrix = <?41 gn (2.5)
gsi g\i

gei gei

gn gn

where g'2x= random number e [0,3] and g'32,g'52 = random numbers e [2,6].

Janikow and Michalewicz [29] created an experimental comparison between

the binary and the floating point representations in genetic algorithms. From their

25
experimental results they concluded that the floating point representation is faster and

provides a higher precision. In this instance the high precision of the length of the

binary coded chromosome will be very long especially for a large number of control

variables. They also proposed a dynamic mutation operation for the floating point

representation. The digit location is selected uniform randomly while the value of the

selected digit is chosen in such a way that there will be a uniform search at the

beginning of the search and it will be close to zero at the last stage of the search to

have a local search.

The mutation operation suggested by Wright [2] is based on mutating the

individual parameters. He suggested a new way to deal with two problems in the real

representation mutation operation: the distribution size of the mutation operation and

how to keep the mutated variable within the specified range. The maximum mutation

rate is defined before starting the algorithm where he chose three maximum mutation

rates 0.1, 0.2 and 0.3 through his experiment. In order to prevent the mutated

parameter to go outside its range, the mutated parameters are chosen using a uniform

distribution between the parameter and the parameter plus or minus the maximum

mutation rate. If the parameter is closer to the edge of the range than the maximum

mutation size, then the new value of the parameter is chosen using a uniform

distribution over the interval between the parameter and the edge of the range.

J. Leite et al. used a real coded representation of the control variables with an

efficient scheme for crossover and mutation as proposed in [30]. The mutation and

crossover properties are dynamically adaptive through the optimal search based on

26
the population behavior. In this experiment the control variable ranges for the

optimization problem are specified by trial and error. The optimization procedure is

repeated several times to reduce the variable ranges which increase the results

precision and facilitates the search toward the global minimum. The final solution can

only be considered optimal after using the algorithm several times and obtaining the

same results each time.

Collins and Eaton [21] used two types of mutation operations: step and

stochastic mutation operation. They defined the mutation operations as follows

If the range of the selected control variable to be mutated is [-a,b], where

a and b are positive real numbers, then

x 2x 3x 5x 6x Ix 8x
step(x) = x' e and
4 4 4 4 4 4 4

\x'e{0X...,b} if x>0l
stochastic(x)= < >.
[x'e {a,...,0} if x<0J

They tested each mutation operation separately and they found that the stochastic

operation has a "poorer" result compared to the step mutation operations.

The single point random mutation was used by Srikanthand and Kamala [23]

is defined as follows: If the parent chromosome is G(t) = [gi,g2,—,g,,—,g„] and the

gene gt € [at, bt ] is the selected genes to be mutated, then the mutated value of the

genes g\ is replaced by a random value from range [a,, bt ] and the new offspring will

beG(t + \) = [g],g2,...,g'l,...,gn].

27
2.3 Summary and Conclusion

Early genetic algorithms used small alphabet coding based on the Schema

Theorem. A small alphabet will have the largest number of schema. Recent

researches showed that large alphabet genetic algorithms have better performance in

solving many real optimization problems [22-30].

Mutation operations in the small alphabet genetic algorithm are considered a

secondary operation as compared to the crossover operation. The main aim of

mutation operation in a small alphabet genetic algorithm is to increase the

randomness of the search in order to decrease the probability of the GA getting stuck

in the local optimal solution.

The importance of the mutation operation is increased in real coded genetic

algorithms. The mutation operator in real coded genes is used to perturb the selected

control variables to locate a best solution. The mutation operation in real coded

genetic algorithm can easily do hill climbing in the decision space, while this is

difficult for mutation in the binary coded genetic algorithm, which can cause the

genetic algorithm to experience Hamming cliffs.

Before illustrating the proposed Multi-Scale Algorithm, it is important to

mention that there are existing algorithms called Multiscale Genetic Algorithms. The

principle of the Multiscale Genetic Algorithm is to use fine and coarse numerical

grids during the simulation [1,2]. The advantage of using both models is to improve

the solution quality and to provide a faster convergence rate than the simple Genetic

Algorithm [1,2]. The multi-scale here is applied in the population grid (size), where it

28
can be changed during the search, so instead of having a large population and taking

extra computing time, the size of the population can be changed through the search by

starting from a small population, then increasing the population size to find a good

quality solution [1,2,3]. The Multi-Scale Algorithm in this dissertation is focused on

having multi-scale information in each chromosome by using an adaptive mutation

operation as will be described in Chapter 3.

29
 CHAPTER 3

MULTI-SCALE ALGORITHM

3.1 Multi-Scale Algorithm

The Multi-Scale Algorithm (MSA) is an optimization algorithm based on a

genetic approach. It is a real coded genetic algorithm, where it is not required to use a

coding operation to represent the control variables in the population. Therefore, the

algorithm doesn't require defining the control variable limits or their resolutions. The

algorithm is called the Multi Scale Algorithm because it uses different scales during

the search. In the beginning of the search, it uses a large scale to cover a large feasible

region; then the scale is reduced to have a finally small scale around a potential

solution. Since the control variables in the MSA have a real representation, the multi-

scale search can be performed by changing the most significant digits of the control

variables randomly during the first iterations; then as the number of iterations is

increased the next digits will have more probability of being changed. Finally, at the

end of the search, the last digits of least significance will have the most probability to

be changed.

3.2 Mutation Operation

A mutation operation will be used in MSA to generate offspring. There are

two parameters in the mutation operation; the first one is the digit location and the

second one is the value of the selected digit. For example, if the parameters of the

mutation operation are (5,7) this means to select the fifth digit on the current string

30
and modify it to 7. So, if the current string is "123.4567" then the next string will be

"123.4767".

The first parameter of the mutation operation (digit location) can be selected

using the random generator that is shown in Figure 3.1. The selection of the bit

location can be divided into several stages, where at the first stage the digits of the

control variable are equally divided into two groups and they are labeled as group 1

and group 2. Then the selected group from the previous stage is divided into two

groups and they also are labeled as group 1 and group 2. These stages will be

continued until each group has only one digit. Figure 3.1 shows the probability of

each group to be chosen versus the number of iterations for three stages, where the

number of iterations is chosen arbitrary to be 1000. Figure 3.2 shows the sequence of

digit location selection for an eight digits control variable.

Table 3-1 shows the group that has a higher probability of being selected for

several iteration intervals (the total number of iteration is set to be 1000). The first bit

location has a higher probability to be chosen at the first 125 iterations. The second

digit has a higher probability in the second 125 iterations, and the third digit on the

third 125 iterations; this is will continue until the last 125 iterations are finished.

Although there is a single digit that has the highest probability to be selected at each

iteration, the other digits still have a non-zero probability to be chosen through all the

iterations. This feature of the MSA increases the probability that this algorithm will

locate the global optimal solution and does not get stuck at a local optimum solution.

For example, if the first digit will be changed only during the first 125 iterations, then

31
it will be fixed. This may cause the algorithm to get stuck at a local optimum area and

the algorithm will not be able to leave this local area.

The second output of the mutation operation is the bit value. The bit value is

an integer number that is selected randomly from the interval [0,9].

1
group 1
o5

0 100 200 300 400 500 GOO 700 800 300 1000
number of iterat ton (k)
(a)

group 1 ~~—
05
_.''''" group!
0 100 200 300 400 500 600 700 800 900 1000
number of iteration (k)

\ ^ grdup 1
0.5
.-'group! **• \ 1 - L- i i "- ^ ' i
1 " ^ i
0 100 200 300 400 500 600 700 800 900 1000
number of iteration (k)
<c)

Figure 3.1. Random generator of the digit location of the mutation operation

Table 3-1 The group that has the highest probability of being selected for several
iteration intervals

iteration Stage #1 Stage#2 Stage#3

001-125 Group 1 "1234" Group 1 "12" Group 1 " 1 "

125-250 Group 1 "1234" Group 1 "12" Group 2"2"
250 -375 Group 1 "1234" Group 2"34" Group 2 " 3 "
375-500 Group 1 "1234" Group 2"34" Group 1 "4"
500-625 Group 2"5678" Group2 "56" Group 1 "5"
625-750 Group 2"5678" Group 2"56" Group2"6"
750-875 Group 2"5678" Group 1"78" Group 2"7"
875-1000 Group 2"5678" Group 1 "78" Group 1 "8"

32
 Stage 1
Group 1 ^ = " 1 2 3 4 "
Group2 x^="5678"

Select one of the

groups randomly as it
shown in Fig. (3.1 a)

Yes

Stage 2 Stage 2
Groupl JC^J ="12" Groupl jt^|="78"
Group2 JC^J = "34" Group2 3^2 = "56"

Select one of the

Select one of the groups randomly as it
groups randomly as it shown in Fig. (3.1-b)
shown in Fig.(3.1-b)

Stage 3 Stage 3 Stage 3 Stage 3

Groupl JC^I= " 1 " Groupl J C ^ = "4" Groupl J ^ , = "5" Groupl jc^= "8"
Group2 x^ = "2" Group2 o^2 = " 3 " Group2 Xn = "6" Group2 xn='l"

Select one of the groups randomly as shown in Figure 3.1-c

Figure 3.2. Flowchart of the random digit selector

33
 A population in the MSA can be created in two ways: the first one is to start

from a single chromosome, then the first population will be the possible outputs of

applying the MSA to the initial point. Thus the initial population will be multi-scaled

strings of the initial point. Based on the function evaluation of the candidates of the

population, the best string will be chosen. For example, if it is required to minimize a

function, then the best string will be the string that has the least function evaluation.

The next population will be generated by applying the Multi-Scale Algorithm on the

selected string from the previous population. The other way is to start from an initial

population, which is selected randomly, then perform a large scale of the current

population at the first iterations to create the next population. As the number of

iterations increases the small scales will be applied to the all strings in the population.

The pseudo code of the mutation operation as shown in Table 3-2. The required

functions of the Mutation function are shown in Table 3-3.

Table 3-2 Pseudo code of the mutation function

Mutation Function
y =Mutation(x, k) % x is the selected string to be mutated
% k is the number of iteration.
[xl, dotloc]=Removedot(x) % Removedot is a function created to
% remove the dot from the selected string
% xl is the same as x but without the dot
% dotloc is the location of the ',' in the
% string x
Loc=Randgen(k) % Randgen is a created function to select
% the digit location based on the number
% of iteration (k).
xl(Loc) = integer(9*rand(l,l)); % the value of the selected location is
% selected randomly
y = Returndot(xl,dotloc) % Returndot is a created function to return
% the dot back into the string

34
Table 3-3 Pseudo code of the required functions of the mutation operation

Randgen Function
Loc =Randgen (k) % Randgen is a function created to
% select the digit location
% k is the number of iteration
% Loc is the digit location to be mutated
Rand = random(l,l) % Rand is a random value in [0,1]
Randv=Rand*[l;l;l] % Randv is a vector of length 3.
PG = [ 1- (k/kmax)
1 - (2k/kmax)
Minimum (|l - (4k/kmax)| 3-(4k/kmax)|];
% PG is a vector of length 3, it
% represents the equations of
% the random generator in the
% MSA.
V = 0.5* (sign( PG-Randv)+1) % V is vector of length 3to present the
% selected group of the three stages
xx = [ 6 5 7 8 3 4 2 1] % xx is the possible digital location
% the digits locations are arranged to be
% compatible with the group labeled.
Value = 4V(1) + 2V(2)+V(3) +1 % Value is used to determine the digit
location from the vector xx.
Loc= xx (Value);

Returndot Function
Y =Returndot(X,dotloc) % Returndot is a function created to
% return the dot to the selected
% string X in the location dotlocation.
For 1=1 To(dotloc-l)
Y( I) = X (I)
End
Y(I+1)=7
For I = locdot+1 To length(X) + 1
Y(I) = X(I-1)
END

Removedot Function
[Y, dotloc]=Removedot(X)
J=i;
For 1=1 tolength(X)
If(X(I)~=0)
Y(J) = X(I)
J = J+1
Else
dotloc = I
Endif
End

35
3.3 Unconstrained Optimization Problems

Optimization problems can be classified based on the existence of constraints:

unconstrained and constrained optimization problems. Unconstrained optimization

problems are problems that need to optimize a given objective function without

constraints, where the search space is the entire space. Constrained optimization

problems require optimizing the objective function within a feasible region, where the

feasible region is usually defined by a set of inequality and equality constraints.

Although most practical optimization problems are considered as constrained

optimization problems, the MSA will be used in this chapter to solve unconstrained

optimization problems for two reasons. First, we will be able to demonstrate the

capability of this algorithm to locate the optimal solution for optimization problems

with different features. Second, there are methods that handle constrained

optimization problems by converting them to equivalent unconstrained problems such

as the Lagrange method and the Penalty method. Constrained optimization problems

will be solved using MSA in Chapter 4.

3.3.1 Unconstrained Objective Function with Single Control Variable

In the case of a single variable function, there is only one direction used to

search for the optimal solution. The MSA will be used to locate the optimal solution

of several unconstrained optimization problems:

- Problem #1 Minimize / ( * ) = 3x4 +4x 3 -12x 2 (3.1)

36
 1
Problem #2 Maximize f(x) = sin 3
(3.2)
x +2x + 2

Problem #3 Minimize f{x) = x2 + sin(lOOx) (3.3)

The first problem is an example of a polynomial problem that has one global

minimum point located at x = -2, and one local minimum point located at x = 1.

Figure 3.3 shows a sequence of the points that is founded by the MSA starting from

an initial point. The algorithm was tested in its ability to locate the global minimum

point starting from different initial points, where four starting points are chosen; one

near the local minimum point, one near the global minimum point, one between

global and local minimum points and one located at the local minimum point. These

starting points are shown in Figure 3.3-a, Figure 3.3-b, Figure 3.3-c and Figure 3.3-d

respectively. As it is shown in Figure 3.3 the algorithm converges to the global

minimum point for all the selected initial points.

Figure 3.3. Convergence of x versus / ( x ) = 3x4 + 4x2 -12x 2 for different initial
points

37
 The pseudo-code of the MSA for locating the optimal solution of Problem#l

is shown in Table 3-4.

Table 3-4 Pseudo code of Problem # 1

pseudo-code of Problem #1
x='+00.000000' % Starting value of the string.
xi=x;
f=3x A 4+4x A 3-12x A 2 % The objective function
err= 100 % initial value of the error
k=l % initial value of the iteration number
j=l
kmax=1000 % maximum number of iterations
tolx=le-6; % desired tolerance of the variable
tolf=le-6; % desired tolerance of the function value
while ( err > 0 & k < kmax)
xtr=Mutation(x,k);
xl=x+xtr;
fl=3xl A 4+4xl A 3-12xl A 2
If (fl < f )
x=xtr
xminmum=xl
fminimum=fl
errx=absolute (xi-xl)-tolx
erry=absolute (f-fl)-toly
err=minimum(errx ,erry)
xi=xl
f=fl
End if
k=k+l
End

Problem #2 required maximizing the objective function/(x) and this is

equivalent to minimizing the objective function - f{x). The objective function in this

problem has a time-varying frequency as shown in Figure 3.4, where this function has

multiple global minimum points. Most of the time the MSA locates the global

minimum value at x =-0.582744. Figure 3.4 shows the sequence of the search

points. Table 3-5 and Table 3-6 show the results of the MSA for locating the global

38
minimum solution of optimization problem #2 for different initial points. The global

minimum points of problem #2 are located in the interval [-1,1], so to focus in this

region the interval [-1,1] is divided into subintervals of a length of 0.2. The MSA was

run 10 times for a different initial point that is selected randomly from the

subinterval. After this, the value found by MSA is recorded in Table 3-5. Table 3-6

shows the result of running the MSA 10 independent times for the same initial point,

and it records the value that the MSA converges at each run

O)
Figure 3.4. (a) The convergence of the an arbitrary initial point to the global
minimum point of the function / ( x ) =-sin(— ) and (b) a zoomed area
x +2x + 2
around the global minimum point

39
Table 3-5 Ten independent runs of MSA for problem #2 for several subintervals

Interval Convergence Function Number of times the method converged

Point evaluation

[-1 -0.8] -0.5828 -1.0000 7

-0.8251 -1.0000 2
-0.8246 -0.9985 1

[-0.8 -0.6] -0.5827 -1.0000 7

-0.7365 -1.0000 2
-0.8252 -1.0000 1

[-0.6 -0.4] -0.5827 -1.0000 10

[-0.4 -0.2] -0.5827 -1.0000 8

-0.7946 -1.0000 1
-0.7365 -1.0000 1

[-0.2 0] -0.7366 -1.0000 1

-0.5827 -1.0000 7
-0.7797 -1.0000 1
-0.7946 -1.0000 1

[0.0 0.2] -0.5827 -1.0000 8

-0.8252 -1.0000 2

[0.2 0.4] -0.5827 -1.0000 8

-0.7771 -0.9996 1
-0.7365 -1.0000 1

[0.4 0.6] -0.5827 -1.0000 8

-0.8252 -1.0000 2

[0.6 0.8] -0.5827 -1.0000 7

-0.5761 -0.9989 1
-0.8252 -1.0000 1
-0.7651 -0.9984 1

[0.8 1] -0.5827 -1.0000 9

-0.8252 -1.0000 1

40
Table 3-6 Ten independent runs of the MSA for different initial points

X0 =-10 x 0 =-1 x 0 =-0.1 X0 = 0 x 0 =10

* * * *
X* /' /' X /* /' /'
X X X

-0.5827 -1.0 -0.5827 -1.0 -0.5827 -1.0 -0.5827 -1.000 -0.6015 -0.9880
-0.5827 -1.0 -0.5827 -1.0 -0.5828 -1.0 -0.5823 -1.000 -0.8252 -1.0000
-0.7366 -1.0 -0.5827 -1.0 -0.5814 -1.0 -0.5826 -1.000 -0.5827 -1.0000
-0.7629 -1.0 -0.5827 -1.0 -0.7642 -1.0 -0.5827 -1.000 -0.7946 -1.0000
-0.5827 -1.0 -0.7365 -1.0 -0.5827 -1.0 -0.8252 -1.000 -0.5827 -1.0000
-0.5826 -.58 -0.5827 -1.0 -0.7861 -1.0 -0.5827 -1.000 -0.5827 -1.0000
-0.1910 -1.0 -0.5827 -1.0 -0.7946 -1.0 -0.5827 -1.000 -0.5827 -1.0000
-0.5827 -1.0 -0.8252 -1.0 -0.7821 -1.0 -0.5827 -1.000 -0.5827 -1.0000
-0.5827 -1.0 -0.7946 -1.0 -0.5820 -1.0 -0.5827 -0.999 -0.5827 -1.0000
-0.5828 -1.0 -0.5827 -1.0 -0.5827 -1.0 -0.5827 -1.000 -0.5846 -0.9999

As it is shown in Figure 3.4 the global minimum points lie either on the

interval [-1 -0.7] or at the point -0.5827. The genetic algorithm in MATLAB® was

used to locate the minimum value of problem #2. The search interval of the genetic

function is [-2 2], then the genetic function is run 100 independent times to locate the

minimum value of this problem. The results can be classified in to three regions; the

first one, the global minimum points lie in the interval [-1 -0.7], in the second, the

minimum point is at -0.5827 and the third one, there is no correct solutions, where the

results are not optimal. In each run of the MATLAB® genetic algorithm, the optimal

solution is recorded if there are no warning messages for the divergence. Table 3-7

shows the results of genetic function to locate the minimum value of problem #2 in

the interval [-2 2]. It shows the number of times that the genetic function locates the

optimal solution in the interval [-1 -0.7], -0.5827 and a non-optimal solution.

41
Table 3-7 The optimal solution of problem #2 for 100 independent runs of genetic
function in MATLAB® and MSA

Number of times Number of times Number of times

was located the were located the were located
optimal point at optimal solutions in Non optimal
-0.5827 the interval |"-1 -0.7] solution
Genetic 0 43 57
[MATLAB®1
MSA 87 11 2

The function of problem #3 has multiple local minimum points and one global

minimum point located atx = 0.01571111. The function in this problem has two

features:

1. The function has two terms; one of them is considered as a high frequency

term with respect to the other.

2. Function evaluations of many of local minimum points are close to the global

minimum value. Figure 3.5 shows the sequence of the search points toward

the global minimum point.

Table 3-8 shows the results of the global minimum points of problem #3 using

the MSA for different initial points. The MSA was run ten independent times for

different random points, which are selected randomly from each interval listed in

Table 3-8.

42
Table 3-8 Solutions of problem #3 for 10 independent runs of MSA for different
initial points
Interval Convergence points
Point Function Number of times the method
evaluation converged
[-110,-90] -0.0157 -0.9998 6
0.1099 -0.9879 1
0.0471 -0.9978 1
-0.016 -0.9993 2
[-90,-70] -0.0160 -0.9993 2
-0.0157 -0.9998 5
0.0471 -0.9978 2
0.1099 -0.9879 1
[-70,-50] -0.0160 -0.9993 1
-0.0157 -0.9998 7
0.1100 -0.9879 2
[-50,-30] -0.0157 -0.9998 6
0.1098 -0.9878 1
-0.0160 -0.9993 2
0.0470 -0.9978 1
[-30,-10] -0.0157 -0.9998 8
-0.0160 -0.9993 1
0.0478 -0.9954 1
[-10,10] -0.0157 -0.9998 10
[10,30] -0.0157 -0.9998 8
0.0471 -0.9978 1
-0.0160 -0.9993 1
[30,50] -0.0157 -0.9998 9
-0.0160 -0.9993 1
[50,70] -0.0157 -0.9998 10
[70,90] 0.1100 -0.9879 1
-0.0157 -0.9998 8
0.0471 -0.9978 1
[90,110] -0.0157 -0.9998 10

43
 1.5

X 0.5

-0.5

Figure 3.5. The sequence of the search x-values of function f(x) = x2 + sin(lOOx)
starting from arbitrary initial point

3.3.2 Search Direction of TV Variables Objective Functions

Before applying the MSA, it is first necessary to determine the search

direction. Several search techniques can be used such as Cyclic coordinate,

Rosenbrock, and Hook and Jeeves.. .etc.

The cyclic coordinate method [34] performs a sequential search along the

coordinate axes of the control variables, where the direction is parallel to one of the

coordinate axes, while the other variables are kept constant until a success point is

founded. Then the search direction is changed to the direction of the second control

variable axis. The cyclic method for minimizing unconstrained objective function of

N control variables can be summarized as follows:

44
 1. Start from initial values of the control variable (XQ = [x10,x20,...,x„0])

and set / = 1 and k = 0, where i is the control variable index and k is

the current success iteration number.

2. The direction of the search will parallel to the ilh control variable axis.

3. Perform a line search on that direction until a success point is found

II Xk+X - f(Xk) < e, where e is the tolerance value.

4. If the converge criteria are met, or if the number of iterations exceeds a

certain limit, then terminate the search method, otherwise go to step 5.

5. i = i + \ ,if i > n then i = 1.

6. Go to step 2.

The direction of the search in the cyclic coordinate method can be used in

MSA, where instead of using a step size in line search, the MSA will use multi-scale

steps to locate an optimal solution. The plot of the objective function

f(x,y) = ( x - 2 ) 2 +(x-2_y) 2 is shown in Figure 3.6. Figure 3.7 shows an expletory

search of the MSA to locate the minimum solution of the objective function.

45
Figure 3.6. Plot of the function f(x,y) = (x-2)2 +(x-2y)2, (x*,y*)is the location
of the minimum solution of the objective function

;^>£>-^-^7--"' --, " -'"" --''"''" —-'"' ~^*^^~ ^^~~^~~~ ^—^~-^~~~

•<i77<r^c"-'" •*""* - " ^ ~^'S*^'~~^L^*~^^~ 1^-"^"—~"~"
- „,'^^^~Z~^^^--^^Z~*^~^\^^^—
- -
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^
"" _, *~
^ ^ ^ \ ^ " ^ ^ ^ ^ ^~^^ O^-jO
- ~ s - ^ ~ ^

' "* ^"" ^^ - ^ ^ / ^ / ^ ' **

s' s / •
^ / C-i^o)
i/ f f [\
( V
V ~ ^ — — ^ - _ ^ _ — — — ~ _ — — ~ — — - — " - _ ^ . — • " __ —-

-^^ ^_____ ~—~"~" ~_ .-——"~"~ _ -*~~"^~~ _- -

"^ -—-_____ „ _ _ — — — " " " l - — - "~~S- ~-~
— • ' - - • , . - _ _ _

- ~_ _ — — - ~ ~ " ~ _ - - — ~ " "•

! \ i 1 • — _ 1 1 1
.

Figure 3.7. Contour plot of the function / ( x ) = ( x - 2 ) 2 +(x-2> , ) 2 and the sequence
of the success points starting from (x 0 , y0) to minimum solution (x*, y*) using cyclic
method

The drawback of the cyclic method is that it is very slow and inefficient to

locate the optimal solution within a flat area like the Rosenbrock function (as will be

discussed later).

46
 In the method of Hooke and Jeeves [35], there are two types of searches,

exploratory and pattern search. The exploratory search is the same as the search in the

cyclic coordinate method and the pattern search is along the direction of the line that

passes through the last two successive points. Figure 3.8 shows the point sequence to

locate the minimum value of the function f{x,y) = ( x - 2 ) 2 + (x-2y)2 using the

search topology in the Hook and Jeeves method with MSA. The search method is still

insufficient to locate the optimal solution of Rosenbrock function.

-1

-2
3 -i i . -• ~ i.. J i
- 3 - 2 - 1 0 1 2 3
x

Figure 3.8. Contour plot of the function f(x) = (x - 2)2 + (x - 2y)2 and the sequence
of the success points starting from (x 0 , yQ) to the minimum solution (x*, y ) using the
Hook and Jeeves method

When these methods were used in MSA to locate the optimal solution of

functions of two variables, satisfactory results were obtained for some objective

functions, but unfortunately, not for all of them, especially those with a flat area near

47
the global optimum solution. In order to increase the capability of MSA to find a

suitable path toward the optimal solution, the search direction in the MSA will be

based on polar coordinate axes. The control variables in the polar coordinates are

radius (r) and angles (6>,), the multi-scale strategy will be applied to the radius while

the angles 0, will be randomly uniformly distributed in [0,2;r]. Using the polar

coordinate axes to specify search directions will allow the MSA to find the suitable

path from the current point to the optimal solution. Using (3.4), it becomes easy to

transform the polar coordinate control variables to x-y control variables as follows:

x = r cos(8)
(3.4)
y = r sin(#)

Polar coordinates variables can be extended to N variables as follows:

x„ =rsin(0„)flcos(0,_,) for n= 1,2,3,..., N-\

1=2

xN=rf[cos(0,) (3.5)
1=1

where, r,6x,62,...,0N_^ are the random search variables.

The search control variables r,6^,92,...,9N_x can be calculated in term of the original

control variablesx ] ,x 2 ,...,x N as follows:

r = *\x\ +x] +... + x2N (3.6)

48
 tan(0„)= \-^— forn=l,2,...,N-l (3.7)

I i=n+\

Now the MSA will be used to locate the optimal solution for unconstrained

objective functions of JV variables using the polar coordinate as a search direction.

3.3.3 Unconstrained Objective Function with Two Control Variables

The MSA will be used to locate the optimal solution for several unconstrained

objective functions with different features: a flat area around the optimal point,

multiple local minima, high nonlinearities, and multiple local optimal points.

3.3.3.1 Rosenbrock Function

The Rosenbrock function/(x, .y) = \00(y-x2) + (x-l)2 or as it is sometimes

called the Banana function, it is one of the best test functions because of the way that

its curvature bends around the origin. The difficulty of locating the minimum solution

of this function is due to "the minimum point being located in a long narrow curved

valley [36 page 3915]. The plot of the Rosenbrock function is shown in Figure 3.9.

The minimum point of the Rosenbrock function is zero and it is located at (1,1).

Sequences of the search points toward the minimum value of this function are shown

in Figure 3.10 for different initial points.

49
 4000 v - -

-2 -2

Figure 3.9. Plot of Rosenbrock function f(x) = \00(y-x1) +(x-l)

>. o >> o

> o > o

Figure 3.10. Contour plot of the banana function for different initial points with
sequences of search points, (a) x0 = 0,^ 0 = 0 (b) x0 = -l,yQ = - 1 ,
(c) x0=l,y0=3and (d) x0 = 3,y0 = 0

50
 The pseudo code of the MSA for locating the minimum value of the

Rosenbrock function is shown in Table 3-9.

Table 3-9 Pseudo code of the Rosenbrock function

Pseudo code of the Rosenbrock function

r='00.000000' % Starting value of the radius.

th = '0.0000000' % Starting value of the angle theta.
xi=0 % strating from any initial value of x.
yi=0 % starting from any initial value of y.
f = 100((y-xA2)A2)+(x-l)A2 % The objective function
err= 100 % initial value of the error
k=l % initial value of the iteration number
j=l
kmax=1000 % maximum number of iteration
tolx=le-6; % desired tolerance of the variable
tolf=le-6; % desired tolerance of the function value
while ( err > 0 & k < kmax)
rtr=Mutation(r,k) % mutate the selected string
thtr=2*pi*rand(l,l) % theta is selected randomly in [0 2n ]
x 1 =xi+rtr* cos(thtr) % The new string is multi scaled from the
y 1 =yi+rtr * sin(thtr) % current string
A A A
f1=100((y 1 -x 1 2) 2)+(x 1 -1 ) 2
If (fl < f )
r=rtr
th=thtr;
xminmum=xl
yminimum=yl
fminimum=fl
errx=maximum(absolute ( xi-xl)-tolx, absolute (yi-yl)-tolx)
errf=absolute (f--fl)-toly
err=minimum(errx,erry)
xi=xl
yi=yl
f=fl
End if
k=k+l
End

51
3.3.3.2 Goldstein and Price Function I

The Goldstein and Price Function I [36] has four local minimum points

located at (1.2,0.8), (1.8,0.2), (-0.6,-0.4) and (0,-1). The last point (0,-1) is the global

minimum point. The function equation is:

f(x,y) = {l + (x + y + l)2(19-14x + 3x2-14y + 6xy + 3y2)}

x {30 + (2x - 3y)2 (18 - 32 x + 12x2 + 48 y - 36 xy + 27 y2)}

The contour plot of this function is shown in Figure 3.11. Figure 3.12. shows the

sequence of the search points starting from one of the other local minimum points.

countour plot of Goldstein and Price function I

Figure 3.11. Contour plot of Goldstein and Price Function I

52
 1.2

0 0.1 0.2 0.3

Figure 3.12. Plot of the sequence of points of Goldstein and Price Function I for
different initial points, (a) (1.2,0.8) (b) (1.8,0.2) (c) (-0.6,-0.4)

3.3.3.3 Eason and Fenton's Gear Train Inertia Function

1 + y2 x2y2+100
The function Jf(x,y) = — 12 + x2 + •+ [36] has four
" 10 x y

global minimum point located at X* =(1.7435, 2.0297), (1.7435, 2.0297), (1.7435,

2.0297) and (1.7435, 2.0297), where f(X*) = 1.74 . Figure 3.13 shows a contour plot

of this function. Figure 3.14 shows the result of the solution using the MSA for

different initial points. The square area bounded by -3.25 < x < 3.25 and

-3.25 <y < 3.25 is divided into 169 sub-squares. In each sub-square, a point is

selected as an initial point for the MSA to locate the optimal solution, then the colors

are used to distinguish between the global minimum points that the MSA converges

53
for each sub-square. The blue color is related to the point (1.7435, 2.0297), the green

color for (1.7435, -2.0297) point, yellow color is for the (-1.7435, 2.0297) and the red

color is for (1.7435, 2.0297). As it is shown in Figure 3.14 even when the starting

point is near one of the global minimum points, the MSA still has the capability to

locate the other global solutions.

Figure 3.13. Contour plot of Eason and Fenton's gear train inertia function

», o

-1

-2 • * • ! ireer

-3

-1

Figure 3.14. Solutions of Eason and Fenton's gear train inertia function found
using MSA for different initial points

54
3.3.3.4. Multiple Local and Single Global Minimum Function

The function/(x, jy) = 20 + x2 + y2 -10cos(2^x)-10cos(2^y) has multiple

local minimum points as shown in Figure 3.15 and one global minimum point is

located at (0,0). Figure 3.16 shows a contour plot for this function and the sequence

of the points as found by MSA starting from different initial points. The MSA

converges to the global minimum for all the initial points.

Figure 3.15. Surface plot of f(x,y) = 20 + x2 + y2 -10cos(2^c)-10cos(2^y)

3.3.3.5 Ackley Function for Two Variables [36]

(i '•> 7 I I
f(x,y) = 20 + e- 20exp(-0.2 - ( x 2 +y ))- exp(-cos(2nx) + -cos(2ny))

The function has multiple global minimum points and a single global minimum point

at (0,0) as shown in Figure 3.17. Figure 3.18 shows the contour plot of this function

includes the sequence of the success string toward the global minimum point.

55
 (b)
2 -V^J s'O^'
-"' '^'
0> '«$> CQ> 'o> c^> . o
1 1^ '@j k€2> ; ' ^ ) c cc
><<^> KO> 'm .-a '0 CI Co;>
o> r
„ ' CO C-t } &1 -ro)
>, 0 >, 0 ^ ' C^5, 1
—^S^ c^
<CS ^o ciJ ^ Co; '(p). ,. CO c.-), CoC
-1 P> v<^> ^ ^ '^ -1 >/ '@> « C<g)> c;
0 > ' to) (o5 " ^
CQ) cO> fS.f 'Co) ct3>
-2 -7*>i-. rrs rrT^ -2 7T^ f^, ,r->,-
-1 0 1 -2 0
x x
(c) (d)
2 2 FT ^ 3 ^ •i~^LJ.- -S^U • ^
CO) "'Co>' Co) cc$>
1 1 ?:; ®- ^ -<§o :Q
v
o " <{_>, C»,> " CoV 'cod co „- co COD
> 0 ^>; ^ C C@>.x ' & '5 > 0
0> ^O-> v. » ' & <£
;co> ~' <.'-? T <c-t cot Co) C-C roC
1 &' v<gS$> ' ' W ' '*H>'~ <£ -1 ^ ^ ^ <^' £
COC ;o> £ ? f 'coc ^ 5 Xo> , Co> CQt
-rTv, £2i: £ ^ - -2Sij< ^ i , ,'r^,. -rix- ""
-2 -1 0 1 0
X x

Figure 3.16. Contour plot of the function f(x,y) - 20 + x2 + y2 - 10cos(2mc) -

10cos(2/ry). For different initial points, (a) x0 = 1.5,y0 =1.5 (b) x0 = -l,y0 = - 1 ,
(c) x0 =0,y0= -1 and (d) x0 = -2,yQ = -2

A c k e l y function

8 -,

s; 4
x

-2 -2

Figure 3.17. Surface plot of the Ackley function

56
Figure 3.18. Contour plot of Ackley function including a point sequence provided by
the MSA to locate the global minimum of this function starting from the point (1,1)

3.3.4 Unconstrained Optimization Function with N Control Variables

In this section the MSA will be used to locate the minimum solution of two

unconstrained objective functions; the Powell quartic function and the Wood

function. In the next chapter constrained optimization problems are solved by

converting them to unconstrained objective functions using the penalty function

method, so many unconstrained N variable optimization problems will be solved in

Chapter 4.

3.3.4.1 Powell Quadratic Function

The function f(X) = (x] +10x 2 ) 2 +5(x3 - x 4 ) 2 +(x2 -2x3)4 +10(x, - x 4 ) 4

The function has a minimum point located at (0,0,0,0). The minimum point of

this function is difficult to locate since the function has a four dimensional flat area

57
around the optimal point [36]. The minimum point that was found by the MSA is

located at (0,0,0,0) for several starting points.

3.3.4.2 Wood Function

f(X) = \00(x2 -x, 2 ) 2 + ( l - x , ) 2 + 90(x4 -x 3 2 ) + ( l - x 3 ) 2 + 10.1[(x2 - l ) 2 +

( x 4 - l ) 2 ] + 19.8(x 2 -l)(x 4 -l)

The Wood function is a four variable polynomial [36], where the function has

a minimum point located at X* = (1,1,1,1). This is a good function for testing

convergence criteria of optimization algorithms, since this function has a flat area that

causes the optimization algorithm to get stuck far away from the optimal solution

[36]. Results found using the MSA for 1000 iteration is shown in Table 3-10.

Table 3-10 Optimal solutions of Wood function using MSA

Trial # X' /(A")

1 (1.0467,1.0957,0.9521,0.9066) 0.008110033
2 (1.0156,1.0323,0.9848,0.9695) 0.9711413e-3
3 (0.9926, 0.9853,1.0072,1.0144) 0.1906355e-3
4 (1.0264,1.0538, 0.9732,0.9469) 0.2572039e-2
5 (1.0305,1.0616,0.9701,0.9402) 0.3413652e-2
6 (1.0142,1.0272,0.9875,0.9752) 0.8593082e-3
7 (1.0393,1.0805,0.9618,0.9247) 0.5730240e-2
8 (0.9617,0.9252,1.0356,1.0727) 0.4963659e-2
9 (1.0152,1.0308,0.9850,0.9702) 0.8347264e-3
10 (1.0099,1.0198,0.9904, 0.9807) 0.3476778e-3

3.4 Summary and Conclusions

The main idea of the MSA algorithm was explained in this chapter. The

mutation operation in the MSA makes the MSA more likely to locate the global

solution of optimization problems. The MSA was used to solve different

58
unconstrained optimization problems to investigate the capability of the MSA

algorithm for locating the optimal solution. The MSA was used to solve three single

variable functions with different features such as multiple local optimum solutions

and time varying frequency. For two variable functions, search directions using the

polar coordinate axes allowed the MSA to search a large area in all directions to find

a suitable path to locate the optimal solution. Several two variable functions with

different features were solved successfully by the MSA. The idea of transforming the

control variables to the polar coordinates was generalized for N variables functions. It

can be concluded based on the results of optimal solution using MSA for the selected

optimization functions that the MSA has the capability to locate the global solution

for many famous benchmark optimization functions.

59
 CHAPTER 4

OPTIMIZATION WITH CONSTRAINTS

4.1 Introduction

Most practical optimization problems are subject to a set of constraints, where

it is required to optimize an objective function / o v e r a specific regionV c 9?". The

search region is called the feasible region and it is usually defined by a set of

inequality and equality constraints involving the control variables. Thus, the

constrained optimization problem can be formulated as follows:

Minimize the objective function f(xl,x2,...,xn) (4.1)

Subject to:

Quality constraints hXxx,x2,...,xn)-0 i-l,2,..,m (4.2)

Inequality constraints g,(x ] ,x 2 ,...,x n ) < 0 i = \,2,..,k (4.3)

In the case of a maximization problem, an equivalent minimization problem can be

defined by replacing the objective function / b y - / .

There are several optimization methods used to solve constrained optimization

problems by converting them to equivalent unconstrained optimization problems in

y{"+m+k space. These primarily include the Lagrange and Penalty methods.

The unconstrained objective function of Lagrange method L(x) consists of the

constrained objective function and the constraints as follows [35]:

m k

L(x,A,v) = Ax) + ^Jh,(x) + J£M,gJ(x) (4-4)

7=1 7=1

60
Where A ; ,// 7 are called Lagrange multipliers, which are also referred to as adjacent

variables. This reduces the constrained problem to an unconstrained problem with

n + m + k control variables. Formally, the optimal solution is found by differentiating

the Lagrange objective function L(x,A,ju) with respect to the control and adjacent

variables and then equating them to zero as follows:

8L df A , dhj * dg
= ^-+>A— +\M,— = 0
> i = \,2,...,n
dxt dx, 7^ dx, ~( dx, (4.5)

—— = h(x) = 0, j = l,2,...,m
(4.6)
o r
- — = g,(x) = 0, j = l,2,...,k
(4.7)

For example, suppose that it is required to:

Minimize f(x,y) = xy2 (4.8)

Subject to h{x, y) = x2 + y2 -1 = 0 (4.9)

The Lagrange function of this problem is L(x,y, A) = xy2 + A(x2 + y2 - 1 ) . The partial

derivatives of the Lagrange function are:

— = y2+2Ax = 0
dx
— = 2xy + 2Ay = 0 (4.10)
dy
^ = x2+y2-l =0
dA

61
The solutions of the above equations are
(\ 2] 9
r i -2^ f - 1 2^
lV3 ' V3 J LV3 ' V3 J'
-1-2^
and . The minimum value of this problem is —1= and it is located at the
3V3

~\ f
•1 2 -1 - 2 ) 2
points and
V3 ' V3 J I V3 ' V3 . The maximum value of this problem is —j=
3V3 and it
f
1 l\ ,( 1 -2
is located at the points . A plot of this function is shown in

Figure 4.1.

Figure 4.1 shows the plots of the constraint h(x,y) (solid line) and function

f(x,y) = c (dashed lines) for different values of the constant c. The two functions

2 -2
h{x, y) and fix, y) = c are tangent when the value of the c is equal to —-= and —j= .
3V3 3v3

Figure 4.1. Plot of a Lagrange problem

62
4.2 Penalty Method

The penalty method is one of the famous methods that replace a constrained

optimization problem by an equivalent set of unconstrained problems [37]. The

objective function of the unconstrained optimization problem will then have two

terms; one is the constrained objective function and the other term is called the

penalty function. The penalty functions are appropriate functions to measure the

violation of the constraints [34, 37,38].

The penalty function evaluation is nonzero when the constraints are violated

and is zero in the feasible region, where constraints are not violated. The constrained

optimization problem that was defined by Equations (4.1-4.3) is reduced to an

unconstrained optimization problem as follows:

Minimize u(X) = f(X) + a(X), X = [xl,x2,...,xn]e R" (4.11)

where a(X) is a suitable penalty function and it is defined as follows:

m k

1=1 /=i

Here <f> and (p are continuous functions, where the main characteristics of them are as

follows [34]

>(;;) = 0 if y < 0 '

My) > 0 if y > 0
- ' \. (4.13)
p(y) = 0 if y = 0 I

<p<y)>0 ify*0

Typical functions of ^and <p are [34]:

<P(y) = [max{0,y}Y (4.14)

63
 cp{y) = \y\P (4.15)

where p is a positive integer constant.

The (p{y) function is a positive function used to present the equality

constraints h(X). The value of this function is zero when there is no violation in the

quality constraints, and it is positive and proportional to constraint violations outside

the feasible region. The function <f>(y) is used to present the inequality

constraint g(X), the value of this function equals zero when the solution is inside the

feasible region (i.e y < 0); otherwise, the (p{y) value is positive and proportional to

the inequality constraint violation. A typical function of ^(.y)is given in Equ.(4.14). If

the solution is feasible (i.e y < 0), then the value of max{0, y) is zero, and if the

solution is unfeasible (i.e y > 0), then the value of max{0,y} is>>.

Usually the penalty function term is multiplied by a constant called the

penalty multiplier. The purpose of the penalty multiplier is to increase the effective

weight of the constraint violation to prevent the search from going outside the

feasible region. The penalty multiplier represents the weight of the penalty functions

in the new unconstrained objective function. The importance of the penalty parameter

is to keep the search within the feasible region. Within the boundaries of the feasible

region, violations of the constraints are very small and will not be enough to keep the

solution within the feasible region. If the penalty function is multiplied by a large

number, then the small violation of the constraints will be more effective to prevent

the search from locating unfeasible solutions. The constraint h(x, y) is zero at feasible

64
optimal solution which means that the optimal solution of the objective function

f(x,y) is the same as the optimal solution of the penalty objective function U(x,y).

Again, consider the optimization problem described by Equations (4.8) and

(4.9). The corresponding penalty objective function can be written as:

U(x,y) = f(x,y) + ch2(x,y) = xy2+c(x2+y2-\)2 (4.16)

The optimal solution for several values of c is shown in Table 4-1. These results were

obtained using the genetic function in MATLAB®. The genetic function in

MATLAB® is an optimization function based on the genetic algorithm implemented

to solve unconstrained optimization problems and constrained optimization problems

of a certain format. In this problem the MATLAB® genetic function will be used to

solve the unconstrained objective function as described in Equ.(4.16). For each value

of c, the MATLAB® genetic function was run 100 times and the average of the

minimum values of the 100 tests are listed in Table 4-1.

Table 4-1 Minimum function values obtained using penalty function with different
values of c

Value of Average of the minimum Standard deviation of the Mean value of

c function value f(x*, y*) Minimum value the constraint
violations
h{x\y)
0.1 -4.0730 0.0045 9.2256
1 -0.4708 0.0677 0.3295
5 -0.3784 0.0729 0.0569
10 -0.3373 0.1129 0.0270
100 -0.1029 0.1997 9.3658e-004
1000 0.1011 0.2057 8.2703e-004

65
 The effect of the value of the penalty parameter is shown in Table 4-1, where

it is noticed that the constraints violation is decreased as the penalty parameter is

increased. At the same time the minimum function value is increased as the penalty

parameter is increased. In Table 4-1 the penalty parameter is kept constant through

the search sequence towards the optimal solution.

Penalty parameter, the penalty functions can be classified as: Death Penalty,

Static Penalty, Dynamic penalty Function and Adaptive Penalty Function [38,39].

4.2.1 Death Penalty Function

In the Death Penalty Function, the unfeasible solution is rejected through the

search, in order to maintain the search inside the feasible region. For some feasible

regions such as convex feasible regions the method is supposed to have good

performance [38]. For many of the constrained optimization problems, it is difficult to

find feasible solutions especially for the optimization problems that have a lot of

constraints [38]. In this case the Death Penalty Function will spend too much time to

locate feasible solutions by investigating all constraints [38]. Besides that, sometimes

unfeasible solutions may be more useful to locate the optimal solution more than

feasible solutions, especially when the unfeasible solution is closer to the optimal

feasible solution than the feasible solution.

4.2.2 Static Penalty Function

A simple method to force the search toward a feasible solution is to apply a

constant penalty in the objective function. The penalized objective function would

66
then be the objective function plus a penalty function. The static penalty function of

the optimization problem described by Equations (4.1-4.3) is as follows [38]:

m+k
Minimize U(X) = f(X) + YdC,al"(X) (4.17)

\ K(X)| if/ = l,2,---,m , „ _

Where: a,(X) = { (4-18)
[max(0,g,(X)) if i = m + \,...,m + k

The value ofp is often 1 or 2, and C, is a constant penalty imposed for the violation

of constraint i.

4.2.3 Dynamic Penalty Function

The penalty parameter in a dynamic penalty function varies with the number

of iterations. The penalty parameter V{t) is increased as the number of iterations t is

increased [38, 40, 41, 42]. The advantage of increasing the penalty parameter over the

search is that the violation of the constraints becomes very small most of the time

near the optimal solution. By increasing the penalty parameter, the weight of the

constraints in the unconstrained objective function is still effective to prevent the

search from locating unfeasible optimal solutions. A typical example of the dynamic

penalty function [42] can be given as:

m+k
U(X) = f(X) + r (0£ < (X) (4.19)
1=1

where V{t) is an increasing function of the iteration.

An advantage of this method is that it requires few numbers of penalty

parameters, where there is only a single penalty parameter V(t)fox all constraints.

67
One of the difficulties of the dynamic penalty function is to choose these parameters.

Method #2 in section 4.5.2 provides an example of dynamic penalty function.

4.2.4 Adaptive Penalty Function

In the adaptive penalty function, the penalty parameter is activated at each

iteration based on the information of the population [39].

4.3 Proposed Method

Before discussing the proposed method it is worthwhile to illustrate the effect

of increasing the penalty parameter by a simple optimization problem. The

optimization problem is:

Minimize f(x, y) = x2 + y1 (4.20)

Subject to /z(x,>0 = ( x - 8 ) 2 + O / - 6 ) 2 - 2 5 = 0 (4.21)

g(x,y) = y + 2x-\2>0 (4.22)

The constraints are intersected in two points (3,6) and (5,2) as shown in Figure 4.2;

one of them is the minimum point (5,1) and the other is the maximum point (3,6)

68
 O 2 4 O 8 lO 12 14
x

Figure 4.2. Constraints plot

The minimum solution of the problem is 29 (/ min =29) and it is located at (5,2),

while the maximum solution is 45 (/max = 45) located at (3,6).

The constrained optimization problem can be solved by using the penalty

function method, where the new unconstrained objective function will be:

f/(x,^) = x 2 + > - 2 + c [ ( ( x - 8 ) 2 + ( > ; - 6 ) 2 - 2 5 ) 2 + ( m a x ( - ^ - 2 x + 12,0))2] (4.23)

The optimal solution of the unconstrained optimization objective function

U(x,y) for several values of penalty parameter cis shown in Table 4-2. The genetic

algorithm in MATLAB® was used to calculate the optimal solution of U(x, y),

where for each value of c the program was run 10 independent times to calculate a

solution. Then the mean value of the referred values and the mean value of

constraints violation are calculated and listed in Table 4-2.

69
Table 4-2 Mean value of 10 trials of the genetic function in MATLAB® for
solution of a constrained optimization problem

Penalty Minimum solution Minimum Value h(x,y) max(g(x*,/),0)

parameter c O* ,y)
1 (4.1276,2.7936) 24.8450 0.2808 0.9513
10 (4.6352,2.3418) 27.2400 -0.0247 0.3878
100 (4.5179,2.9718) 32.9155 -0.0324 0.1848
1000 (4.4440, 3.8450) 42.2300 -0.0132 0.0429
10000 (4.1512,4.2045) 41.8661 -0.0077 0.0276
100000 (5.5454, 2.0887) 38.7861 -0.0051 0.0364
106 (4.4283,4.0940) 44.9813 -0.0004 0.0034

Table 4-2 shows that, when the value of c increases, the constraint violations

decreases, since the increasing of the value of the penalty parameter will increase the

effect of constraint violations in the unconstrained objective function. Even though

for a small violation of constraints, the constraints will multiply by a large penalty

parameter to prevent the search from going outside the feasible region. For the mean

value of the optimal solution, it is not improved as the value of c is increased.

To illustrate the effect of increasing the value of penalty parameter on the

optimal solution of f{x,y), let us examine the location of solutions for c = 100 from

Table 4-3. The location of the minimum solution converges to the point (3,6) at the

first, fifth and eighth trial, while in the other trials it converges to (5,2). Recall that the

minimum solution of the unconstrained objective function U(x,y) is located at (5,2),

and the maximum solution is located at (3,6). Also refer to Figure 4.2. Both points

(4,2) and (3,6) satisfy the constraints. By examining these results it is possible to

conclude that when the value of the penalty parameter is increased, then the weight of

70
the penalty function in the unconstrained optimization objective function will

increase. This will cause the search to lead to points that do not violate the constraints

more than focusing on searching the optimal solution of the unconstrained objective

function U(x,y).

If the value of the penalty parameter is increased to a very large value, then

the location of the solution of the constrained objective function U(x,y) will

converge to points that satisfy both constraints. Notice that the values of constraints

near the feasible optimal solution are very small and go toward zero, for that reason

the value of C( in Equ.(4.17) must be increased to prevent the search from going out

of the feasible region.

Table 4-3 Optimal solution location for c = 100

Number of run Optimal solution

1 (2.9984, 5.9972)
2 (5.0148,1.9890)
3 (4.9964, 2.0028)
4 (5.0225 , 1.9836)
5 (3.0702 ,6.8312)
6 (5.5352 , 1.6496)
7 (5.0443 , 1.9675)
8 (3.0982 ,6.9840)
9 (5.3420 , 1.7642)
10 (4.9919 ,2.0079)

Clearly, the penalty method is affected strongly by the value of C,. If the value

of C, is too small, final infeasible solutions may result, and if C, is too large, it may

converge to non-optimal feasible solutions.

71
 Most of the penalty function methods change the value of C, based on the

number of violated constraints and the constraint violations. Sometimes, it is difficult

to make sure that the changed value of the factor C, is sufficient to prevent the search

from going outside the feasible region without evaluation of the objective function/.

For example, let the objective function be the total cost of a factory, which is

measured by millions of dollars, and one of the constraints is related to the number of

workers, which can be measured by hundreds. In this case the initial value and the

increased rate of C, related to this constraint must be large. Otherwise the opposite

case where the constraints have higher measuring units than the objective function

will lead to choosing a low initial value and low increasing rate of C,.

In summary, the initial value and the changing rate of the coefficient C, is

strongly affected by the constrained objective function.

In the proposed method, adding another factor with the constrained objective

function will be helpful in making a balanced emphasis between the constrained

objective function and the constraints to locate optimal solutions. The penalty

function can be written as follows:

m k

U(X) = CJ(X) + C2[YJ(hX^)) + t,(max(Sl^m2)] (4.24)

1=1 1=1

The constrained optimization problem now is divided to two problems:

1. To find the global optimal solution for the unconstrained optimization

function U(X).

72
 2. To determine suitable values of the factors C, and C2 that lead the search to

optimal solutions.

The values of C, and C2will be updated in every iteration based on the relative

change of the constrained objective function and the relative change of the

constraints. The relative change of the objective function and the relative change on

the constraints is given by (4.12) and (4.13) respectively.

A/„ = ^ / ^ (4.25)
Ji-\

h
A A-K-1 l,-h2m,,-x max(g 1 ,,0) 2 -max(g 1 ,_ 1 ,0) 2
Aa n = m a x [ ^ ,...,
h i,,-i h m,i-\ max(g, ,_,,0)
(4.26)
max(gk, ,0)2 - max(gyt,_, ,0)2

™ax(g iM ,0) 2

If the relative change in the constrained objective function Afn is greater than the

relative change in the constraints Aa„, then the value of C 2 will be increased;

otherwise, the value of C, is increased.

A suggested initial value of C, is 1, while the initial value of C2 is chosen as follows:

C2=e ^—^ (4.27)

max(/z 2 ,/2 2 ,...,/z 2 ,max(g 1 ,0) 2 ,max(g 2 ,0) 2 ,...,max(g i ,0) 2

where e is a real number on the interval [0,1].

The value of s represents the proportion of the initial weight that will be

given to the constraints. The changed rate of the penalty parameters is given by:

73
 A C (4 28)
>.<=T^ -

In many constrained optimization problems the control variables have upper and

lower limits as given in Equ.(4.30). In this method, if the control variable exceeds its

limits then value of the control variable will be adjusted to the nearest limit value as

given in Equ.(4.31).

x <x <x (4.30)

imin i (max V"-'"/

x, =min(x, max ,max(x,,x, min )) (4.31)

The pseudo-code of the MSA with the proposed penalty method starting from

single string is shown in Table 4-4.

74
Table 4-4 Pseudo code of MSA with the proposed penalty method

Pseudo code of the MSA with the proposed penalty method

r='00.000000' % Starting value of the radius.

thl = '0.0000000' % Starting value of the angle theta.
th2 = '0.0000000' % Starting value of the angle theta.
xi= 0 ; yi=0 % strating from any initial value of x and y.
cl=l; c2=l % initial value of the penalty parameter
f =xA2+yA2 % The objective function
h=(x-8)A2+(y-6)A2-25 % The equality constraint
g=-y-2x+12 % The inequality constraint
viol = maximum(absolute(h,g)) % the maximum violation of constarints
fcon=cl*f+c2*(hA2+max(g,0)A2) % The constraint objective function
err= 100 % initial value of the error
k=l % initial value of the iteration number
kmax= 1000 % maximum number of iteration
tolx=le-6; tolf=le-6; % desired tolerances
while (err > 0 & k < kmax)
rtr=Mutation(r,k) % mutate the radius r
thtr=2*pi*rand(l,l) % theta is uniformly distributed from [0 2 7T ]
xl=xi+rtr*cos(thtr); % xl is the multi scaled string from xi
yl=yi+rtr*sin(thtr) % yl is the multi scaled string from yi
fl =xlA2+yA2 % the function evaluation of the new strings
hl=(xl-8) A 2+(yl-6) A 2-25 % the quality constraint violation
g 1 =-y 1-2x1+12 %the inequality constraint violation
fconl=cl*fl+c2*(hl A 2+max(gl,0) A 2) % the penalty objective function
If ( fconl < fcon ) % is the next string better than the previous one
r=rtr;
th=thtr;
xminmum=xl
yminimum=yl
fminimum=fconl;
errx=maximum(absolute ( xi-xl)-tolx, absolute ( yi-yl)-tolx)
errf=absolute (fcon -fconl)-tolf
err=minimum(errx,errf)
xi=xl; yi=yl
violl= maximum(absolute(hl,gl))
If viol 1> le-6
violrel=(viol 1 -viol)/viol;
frel=(fcon 1 -fcon)/fcon
If frel > violl
c2=c2+frel/violrel
Else
cl=cl+violrel/frel
End if
End if
fcon=fconl
viol=viol 1
End if
k=k+l
End

75
4.4 Test Functions

In order to investigate the capability of the MSA with the proposed penalty

method to locate the optimal solution of constrained optimization problems, solution

of a set of benchmarks is performed, so the results can be compared. The optimal

solution of five mathematical bench mark constrained optimization problems is

provided [42]. The five optimization problems are called benchmark and it are

usually used to test the optimization methods.

Problem#l

4 13

Minimize f(X) = 5^ xt (l - x,) - ^ x,

i=l (=5

Subject to:

2x, + 2x2 + xl0 +xn < 10

2x, + 2x3 + xl0 + xl2 < 10
2x2 + 2x3 + x,, + x,2 ^ 10
- 8xj +xw <0
-8x2 + xn <0
- 8x3 + xI2 <0
- 2x4 - x5 + xl0 < 0

- 2x4 - x5 + x10 <0

- 2x6 - x7 + Xj, <0

0<x,<l, i = l,...,9
0<x,<100, i = 10,11,12
0<xn <1

76
The global minimum is X* = (1,1,1,1,1,1,1,1,1,3,3,3,1). The minimum is

/ ( * ' ) =-15

Problem#2

Minimize f{X) = x, + x2 + x3

Subject to:

l-0.0025(x4+x6)>0
1 -0.0025(x 5 + x7 - x 4 ) > 0
l-0.01(x8-x5)>0
x,x 6 -833.33252x 4 -100x, + 83333.333 > 0
x2x7 -1250x 5 - x 2 x 4 +1250x4 > 0
x3x8 -1250.000 - x3x5 + 2500x5 > 0

100 <x, < 10000

1000 < x, < 10000, / = 2,3
10 <x, <1000, i=4 8.

The global minimum is X' = (579.3167, 1359.943, 5110.071, 182.0174, 295.5985,

217.9799, 286.4162, 395.5979). The minimum value is f(X*) =7049.330923.

Problem#3

Minimize:

/ ( * ) = (*, -10) 2 +5(x 2 -12) 2 +x34 +3(x 4 - l l ) 2 +10x56 +7x62 +x74 -4x 6 x 7 -10x 6 -8x 7

Subject to:

77
 127 - 2xj - 3x42 -x3- 4x] -5x5>0
282 - 7xj - 3x2 - 10x] - x4 + xs > 0
196- 23x, - x\ - 6x26 +8x7>0
- 4x] - x] + 3x,x2 - 2x\ - 5x6 + llx7 > 0

-10.0 <x, < 10.0, i = l,2 7.

The global minimum is f(X') =0.0539498478 and it located at X' =(-1.717143,

1.595709, 1.827247, -0.7636413, -0.7636450).

Problem#4

Minimize:

f(X) = ex,XlX*x,Xs

Subject to:

x] + x] + x\ + x] + x] =10

x) +x32 = -1

-23 <x, <2.3, i = l,2

-3.2 <x, <3.2, / = 3,4,5.

The global minimum value is f(X*)= 0.0539498478 and it located at X* =(-

1.717143, 1.595709, 1.827247,-0.7636413,-0.7636450).

ProbIem#5

Minimize:

/(JT) = x2 +x22 + x,x2 -14x, -16* 2 + (x3 -10) 2 +4(x 4 - 5 ) 2 +(x 5 - 3 ) 2 +

2(x6 - l ) 2 +5x72 +7(x 8 - l l ) 2 +2(x 9 -10) 2 +(x 10 - 7 ) 2 +45

Subject to:

78
 105 - 4 x , - 5 x 2 + 3x7 - 9 x 8 > 0
-10x, + 8x2 +17x7 - 2 x 8 > 0
8x, - 2x2 - 5x9 + 2x10 +12 > 0
- 3(xl - 2) 2 - 4(x2 - 3)2 - 2x32 + 7x4 +120 > 0
-5x2-8x2-(x3-6)2+2x4+40>0
- x2 - 2(x2 - 2)2 + 2x,x2 - 14x5 + 6x6 > 0
- 0.5(x, - 8)2 - 2(x2 - 4)2 - 3x2 + x6 + 30 > 0
3x, - 6x2 - 12(x9 - 8)2 + 7x10 > 0
-10.0 < x, < 10.0, / = 1,...,10

4.5 Constraint Handling Methods by Genetic Algorithm

The results of the optimal solution using the Multi-Scale Algorithm with the

proposed penalty method will be compared to the optimal results of five constraint

handling methods. The constrained methods used the penalty function to handle the

constraints of the optimization problem as follows [41]:

f hAX) if \<i<m
a,(X) = < (4.32)
|max(g,(X),0) if m + l<i<m + k

4.5.1 Method #1

The method of Homaifar, Lai and Qi [41] is an example of static penalty

function. They divide the constraint violations into / levels, for each violation level

they defined penalty coefficients. The value of penalty coefficient is increased as the

search reaches a higher level of violation. The unconstrained objective function of

this method can be written as follows [41]:

U(X) = f(X) + £ [ i ? y a 2 ( X ) ] , j = 1,2,...,/ (4.33)

79
 For example, if there are 3 constraints and it is decided to have 5 levels of

violation of each constraint, this means that there will be 3x5= 15 values of the

penalty parameter R.

The weakness of this method is the large number of parameters. The total

number of penalty parameters R is (k + m)l and the boundary interval of the

violation of the constraints must also be determined, which means that it needs

(m + k)(2l +1) parameters [41]. It is also difficult to define the set of parameters that

returns feasible solutions near the optimal [41].

4.5.2 Method #2

The method was proposed by Joines and Houk [41,42,43], they proposed a

single dynamic penalty function. The value of the penalty coefficient is increased as

the number of iteration increases. The penalty function of this method is defined as

follows [41]

U{X) = /CD + CO/X < (X) (4.34)

1=1

where t is the current iteration. Here C,/?and vare constants [41].

The advantage of this method is that the number of parameters in this method is

independent of the number of constraints. A reasonable choice of C,/?and v are 0.5,

2 and 2 respectively [41]. One of the drawbacks [41] of this method is that for a large

number of iterations the value of the penalty coefficient becomes very large. Also it is

found that the method converges to an unfeasible solution or to unoptimal feasible

solutions most of the time [44].

80
4.5.3 Method #3

The method of Schoenauer and Xanthakis [41] can be summarized as follows:

1. Select initial population contains feasible and unfeasible chromosomes.

2. Start with the first constraint and set ( j = 1 ).

3. Evaluate the value a ^X) for the initial population to determine the feasible

population percentage until the feasible population percentage exceeds a

threshold limit, where it is called the flip threshold.

4. Set ; = ; + l

5. Starting from the population of the previous constraint (i.e. j - \ constraint),

compute the value of a} (X) until the percentage of the feasible population

exceeds the flip threshold value.

6. Repeat step 4 and 5 for all constraints until j = m + k.

7. Optimize the objective function f(X).

In summary the method can be divided into two stages. The first stage is to

determine the feasible population by investigating the constraints in order. Then in

the second stage the objective function f(X) is optimized without need of penalty

functions [43]. The drawback of this method is that the effect of constraint orders is

unclear. Zbigniew [39,40] found experimentally that for different constraint orders

this method will yield different results based on the running time and the precision.

The method depends on the size of the feasible search space, where it has a low

performance for a very small feasible search space [42].

81
4.5.4 Method #4

The method proposed by Michalewicz and Attia [44] starts with a single point

that satisfies the linear quality and linear inequality constraints. The initial population

is just copied from the starting point. The penalty function of this method is [41]

I m+k
U(X) = f(X) + — Xaf(X) (4.35)

where r is temperature parameter. Starting from an initial value of the temperature,

then the temperature is decreased until it reaches the freezing point. This method

requires starting from a feasible point of the equality constraints[41]. In order to

maintain feasibility of the linear constraints a set of closed operators is used [43]. The

closed operators are "operators that convert the feasible solution to another feasible

solution" [41 page 153].

4.5.5 Method #5

The method of Powell and Skolnick [41] uses a penalty function

m+k
U(X) = f(X) + r['£a,(X)] + W,X) (4.36)

where r is constant and t is the iteration number. The term A(t,X) is used to

maintain easible solution. The value of A(t,X) is higher for an unfeasible solution

which a guarantees that any feasible solution is better than any unfeasible solution

[41].

4.6 Results and Comparison

Michalewicz [41] had an experiment which tested the five penalty methods

for locating the optimal solution of the five test functions. The comparison between

82
methods is done based on the best value (b), the mean value (m), the worst value (w)

of the solutions and the number of violated constraints (c) at the median value of the

discovered solutions that are found from 10 independent runs of each method. Here c

is sequence of three numbers, they indicate how many constraints are violated with

value more than 1, more than 0.1 and more than 0.001 respectively [41]. The symbol

'*' means that method can not be applied to this test problem and the symbol ' - '

means that the result is meaningless. The results of his experiment are listed in Table

4-5 [41].

Table 4-5 Solutions of tests functions using penalty methods [41] and MSA with
proposed penalty method

Exact Method Method Method Method Method MSA with

# optimal #1 #2 #3 #4 #5 proposed
value penalty
Method
b -15.001 -15.000 -15.000 -15.000 -15.000 -15.000
m -15.002 -15.000 -15.000 -15.000 -15.000 -15.000
1 -15.000 w -15.002 -14.999 -14.998 -15.000 -14.999 -15.000
c 0,0,4 0,0,0 0,0,0 0,0,0 0,0,0 0,0,0
b 2282.723 3117.242 7485.667 7377.97 2101.367 3111.3
m 2449.798 4213.497 8271.292 8206.15 2101.411 4869.5
2 7049.33 w 2756.679 6056.211 8752.412 9652.90 2101.551 7503.6
c 0,3,0 0,3,0 0,0,0 0,0,0 1,2,1 0,1,1
b 680.771 680.787 680.836 680.642 680.805 680.9865
m 681.262 681.111 681.175 680.718 682.682 681.7250
3 680.630 w 689.660 682.798 685.640 680.955 685.738 682.3823
c 0,0,1 0,0,0 0,0,0 0,0,0 0,0,0 0,0,0
b 0.084 0.059 0.054 0.067 0.0489
m 0.955 0.812 0.064 0.091 0.0872
4 0.054 w 1.000 2.542 * 0.557 0.512 .153322
c 0,0,0 0,0,0 0,0,0 0,0,0 0,0,2
b 24.690 25.486 18.917 17.388 24.4945
m 29.258 26.905 24.418 22.932 31.0616
5 24.306 w 36.060 42.358 44.302 48.866 34.5194
c 0,1,1 0,0,0 0,1,0 1,0,0 0,1,0

83
 For the first test function, the MSA with the proposed penalty method

successfully located the feasible optimal solution, while for the second test function

the method located an unfeasible solution of small constraint violations. In this test

function, for a small violation of constraints, the method gives a large decrease in the

objective function. The feasible optimal solution is 7049, while the mean value of the

solutions found by the proposed method is 4869.5 with two violated constraints, one

of them is violated with 0.1 and the other is violated with 0.01.

In most cases, it is very important to obtained a feasible solution and to not

allow any violation of constraints. The proposed method can be modified to reject any

unfeasible solutions by replacing the unfeasible solution with a feasible solution

which lies on the border of the violated constraints. This will be the same as the way

that this method deals with the control variables limits, where at any iteration, if the

control variable x, is less than the minimum point, then x, = x, min , and if the control

variable x, is greater, than x (max thenx, = x,max by using Equ.(4.31). In this case, the

constraints can be divided as separable and inseparable constraints based on the

ability to rewrite the constraints in a way that each control variable in the constraint is

bounded by the others control variables.

The linear inequality constraints are always considered as separable

constraints, so Equation (4.31) will now be modified to include the linear inequality

constraints to prevent violations on these constraints. In this case, if there is any

violations of the linear inequality constrains the location of the solution is modified to

84
be located on the border of these constraints. For example, if

g, (X) = x, + x2 - 3x3 +12 < 0, then the feasible solution of this constraints will be:

x2 < - 1 2 - x , + 3x3

x3 > —(12 + x, + x2) .

These equations will then be combined with the limits range of the control variables

as follows:

x, =max([x lmax ,min(x 1 ,x lm , n )-12-x 1 +3x 3 ])

x2 =max([x 2max ,min(x 2 ,x 2ram ),-12-x 2 +3x 3 ])

x3 =max(x 3max ,min([x 3 ,x 3min ,-(12 + x1 +x 2 )]))

Inseparable constraints will be represented by their penalty functions in the

unconstrained objective function U(X). Returning to the second test function and

applying the modified method to find the optimal solution, the result is listed in Table

4-6.

Table 4-6 Solutions of the second test function using the MSA with the modified
penalty method

Best solution Mean solution Worst solution Constraints violation

7077.028105 7090.205150 7098.809647 None

85
 The MSA with modified penalty method converges in all the ten independent

runs to a feasible solution. The modified method converges quickly at the first

iterations to the optimal solution, while it is very slow near the optimal solution. The

modified penalty function will now be applied to find the optimal solution of

engineering optimization problems. Returning back to the optimization problem was

given in Equations (4.20 - 4.22), the pseudo code of the MSA with the modified

penalty method of this problem is shown in Table 4-7.

86
Table 4-7 Pseudo code of the MSA with the modified penalty proposed method

Pseudo code of the MSA with the modified penalty proposed method
r='00.000000' % Starting value of the radius.
thl = '0.0000000' % Starting value of the angle theta.
th2 = '0.0000000' % Starting value of the angle theta.
xi=0
yi=0 >
^1=1 • % strating from any initial value of x and y.
ci-i,
% initial value of the penalty parameter
c2=l
f=xA 2+yA2 % The objective function
h=(x- 8)A2+(y-6)A2-25 % The equality constraint
viol = absolute(h) % the maximum violation of constarints
fcon= cl*f+c2*(hA2) % The constraint objective function
err= 100 % initial value of the error
k=l % initial value of the iteration number
kmax =1000 % maximum number of iteration
tolx= le-6
tolf=l e-6 % desired tolerances
while ( err > 0 & k < kmax)
rtr=Mutation(r,k);
thtr=2*pi*rand(l,l)
xl=xi+rtr*cos(thtr);
yl=max(yi+rtr*sin(thtr), 2-2x1) % the inequality constraint
fl =xlA2+yA2 ; hl=(xl-8) A 2+(yl-6) A 2-25;
fconl=cl*fl+c2*(hl A 2)
If ( fconl < fcon )
r=rtr;
th=thtr
xminmum=xl;
yminimum=yl;
fminimum=fconl
errx=maximum(absolute (xi-xl)-tolx, absolute ( yi-yl)-tolx)
errf=absolute (fcon -fconl)-tolf
err=minimum(errx,errf)
xi=xl;
yi=yl;
violl=absolute(hl)
violrel=(viol 1 -viol)/viol
frel=(fcon 1 -fcon)/fcon
If violl> le-6
If frel > violrel
c2=c2+frel/violrel
Else
cl=cl+violrel/frel
Endif
Endif
fcon=fconl
viol=violl
Endif
k=k+l
End

87
4.7 Engineering Optimization Problems

Five engineering optimization problems are considered as benchmarks:

Design of a Welded Beam, Design of a Pressure Vessel, Minimization of the Weight

of a Tension/Compression Spring, Minimization of the Weight of a Speed Reducer

and Himmelblau's Nonlinear Optimization Problem.

4.7.1 Design of Welded Beam

The problem is minimizing the cost of the welded beam, subject to several

constraints of shear stress (T), bending stress in the beam (<r), buckling load on the

bar (Pc) and the end deflection of the beam ( 8 ) [ 46].

Minimize f(X) = 1.10471 x,2x2 + 0.0481 lx3x4 (14+ x 2 )

Subject to g, (X) = z{X) -1360 < 0

g 2 (X) = a ( X ) - 3 0 0 0 0 < 0

g 3 (X) = x , - x 4 < 0

g4(X) = 0.10471x,2 + 0.0481 lx3x4(14 + x 2 ) - 5 < 0

g 5 (X) = 0.125-x, < 0

g6(X) = S(X)-0.25<0

gl(X) = 6-Pc(X)<0

r(X) = ^T'2 (X) + 2T'(X)T"(X)^ + (T"(X))2

r W = 3^/2

88
 M(X)R(X)
r\X)
J(X)

M{X) = 6000(14 + ^ - )

R(X) = yx22+(x]+x2)2

X
2 \X\ ~*~ X3 )
J{X) = v2x,x 2
\ J

504000
a{X) = —
X^ X^

X3V5
PC(X) = 20.065x\06 yfxjx, 1 -
28V8

0.1 <x, <2, 0.1 <x 4 < 2

0.1 <x 2 <10, 0.1 <x 3 <10

It has been found [42] that the optimal solution is f(X*) = 1.724852 located at

X* =(0.205730, 3.470489, 9.03662,0.205729).

4.7.2. Design of a Pressure Vessel

This problem needs to minimize the total cost of design of cylindrical vessels.

The cylindrical vessel is capped at both ends by hemispherical heads where the

control variables of this problem are: thickness of shell (x,) , thickness of the head (

x 2 ), the inner radius ( x3) and the length of the cylindrical section of the vessel (x 4 )

[46,47].

89
Minimize f(X) = 0.6224x,x3x4 + 1.7781x2x2 + 3.1661x2x4 + 19.84x,2x3

Subject to g, (X) = -x, + 0.0193x3 < 0

g 2 (X) = -x 2 +0.00954x 3 < 0

g3 (X) = -m\xt —TVC] +1296000 < 0

g 4 (X) = x 4 - 2 4 0 < 0

10<x 3 <200,
10<x 4 <200

Where, x,and x2 are discrete integer multipliers of 0.0625 as follows:

0.0625x1 <x, <0.0625x99

0.0625x1 <x 2 < 0.0625x99

The optimal solution is X* = (0.205730,3.470489,9.03662,0.205729) and

f(X') = 6059.7143.

4.7.3 Minimization of the Weight of a Tension/Compression Spring

The problem requires minimizing the weight of a tension/compression spring

[46,47]. The control variables are the wire diameter (x,), the mean coil diameter,

(x 2 ), and the number of active coils (x 3 ). The optimization problem is:

Minimize: f(X) = (x3 + 2)x2x,2

Subjectto ,(D = l - ^ . 0

4
g2(X)= *2-*.*2 + _J_._ 1 < 0
12566(x 2 x, 3 -x ] 4 ) 5108x,2

90
 x2x3

0.05 < x, < 2

0.25 <x, <1.3

2 < x , <15

The optimal solution is X* = (0.051690,0.356750,11.287126) and

f{X') = 0.012665.

4.7.4 Minimization of the Weight of a Speed Reducer

It is required to minimize the weight of speed reducer subject to bending stress

of the gear teeth, surface stress, transverse deflections of the shafts and stresses in the

shaft. The control variables are the face width (x,), module of teeth (x 2 ), number of

teeth in the pinion (x 3 ), length of the first shaft between bearings (x 4 ), length of the

second shaft between bearings (x 5 ), diameter of the first shaft (x 6 ) and the diameter

of the second shaft (x 7 ) [46,47]. The optimization problem is:

f(X) = 0.7854x,x2(3.333x2 +14.9334x3 -43.0934)-1.508x, (x62 +x72)

Minimize
+ 7.4777(x63 + x73) + 0.7854(x4x2 + x5x72)

21
Subject to gx(X)= 2 -1<0

397 5
g2(X) = ^ T - \ < 0
x, x 2 x3

91
 1 93x3
XjXjXfr

1.93x 5 3
g*{X) = 4
1<0
9 "^ 7

( v ^
745' + 16.9xl0 e
x x
v 2 3y
& ( * ) = •
-1<0
l i Ox;

745' + 157.5xl0 6
X X
V 2 3 J
g6W = - -1<0
85x 7 3

61
40

gs(X) = ^-l<0

g9W 1<0
12x,

ftoW = l^±i?-l,0

l.lx 7 +1.9
6iW = •1<0

2.6<x,<3.6, 0.7<x 2 <0.8, 17<x3<28, 7.3<x 4 <8.3

7.8 < x5 < 8.3, 2.9 < x6 < 3.9, 5 < x7 < 5.5

The optimal solution is f(X') = 2996.348165 located at

X' =(3.50000,0.7, 17, 7.30000,7.80000,3.350214,5.286683)

92
4.7.5 Himmelblau's Nonlinear Optimization Problem [47]

Minimize f(X) = 5.3578547x] + 0.8356891x,x5 + 37.293239x, -40792.141

Subject to:

gx(X) = 85.334407 + 0.0056858x2x5 + 0.00026x,x4 -0.0022053x 3 x 5 - 9 2 < 0

g2(X) = -85.334407-0.0056858x 2 x 5 -0.00026x,x 4 +0.0022053x2x5 <0

g 3 (X) = 80.51249 + 0.0071317x2x5+0.0029955x,x2+0.0021813x32 -110<0

g 4 (X) = -80.51249-0.0071317x 2 x 5 -0.0029955x 1 x 2 -0.0021813x 3 +90<0

g5(X) = 9.300961 + 0.0047026x3x5 +0.0012547x,x3 +0.0019058x3x4 - 2 5 < 0

g6(X) = -9.30096l-0.0047026x 3 x 5 -0.0012547x,x 3 -0.0019058x 3 x 4 + 2 0 < 0

78<x,<102, 35<x2<45, 27 <x, < 45, i = 3,4,5

The optimal solution of this problem is not available and the best solution is found by

[46] at X* = {078, 33, 27.070997, 45, 44.969242} and is equal to

f(X') = -31025.560242.

4.7.6 Results of the Engineering Optimization Problems Using MSA and

Discussion

The MSA was used to solve engineering optimization problems. For each

problem the MSA was run 10 independent times. The result of the best, mean and

standard deviation of the minimum values of the optimization problems are listed

from Table 4-8 to Table 4-12. Based on the results that were found, it can be noted

that the MSA located the optimal solution of these problems successfully. The result

93
of the 10 independent runs of the MSA gives approximately the same result with a

standard deviation of less than 0.2, except in the Minimization of the Weight of a

Speed Reducer problem, where the standard deviation of the 10 independent runs is

1.007707619.

Table 4-8 Design of welded beam result using MSA

* 0.20572968
X l
* 3.4704885
X 2
X*3 9.0366230
X*4 0.20572968
s.cn -0.001163210898

&(**)
0.0
g^x') 0.0

*4co -3.432984059
gsiX*) -0.8072968337e-l
get*') -0.2355403211
gA**) -0.3395178484e-2
1.724852516
Best solution f(X )
1.724855838
Mean value of *' '
Standard deviation 0.6252637283e-5
f{X')

94
Table 4-9 Design of a pressure vessel result using MSA
* 0.81250000
X l
X*2 0.43750000
* 42.098446
X 3
X*4 176.63660
g,(0 0.0

-0.03588082902
s 2 (0
S3(0 -0.4724413157e-4
6059.714335
Best solution f(x )
6059.721589
Mean value of •> *• *
Standard deviation 0.008085231609

Table 4-10 Minimization of the weight of a tension/compression spring result using

MSA

* 0.05175624976
X l
X 2 0.3583353678
*
X 3 11.19487409
-0.3954639453e-5
s,(0
g 2 (0 -0.2039575076e-5
-5.831696027
s 3 (0
-1.089908382
s 4 (0
0.01266544711
Best solution f(X )
0.01266802395
Mean value of J ' '
Standard deviation 0.4553433447e-5
/(O

95
Table 4-11 Minimization of the weight of a speed reducer using MSA
* 3.5000000
X l

X*2 0.70000000
X*3 17
* 7.3000000
X 4
* 7.8000000
X 5
* 3.3502147
X 6
5.2866832
V
s,(0 -0.07391528040

s2(0 -0.4991722481

s3(0 -0.9014716976

s4(0 -.0666133815e-15

g 5 (0 -.02290371226e-9,

g 6 (0 -0.7025000000

s 7 (0 0
£ 8 (0 -.5833333333

s 9 (0 -0.05132575354

S,o(0 -0.01085236498
-0.1979985271
Sn(0
2996.348165
Best solution - ^ ^ )

2996.737817
Mean value of J ' '

Standard deviation 1.007707619

/(O
Table 4-12 Minimization of the Himmelblau's nonlinear problem using MSA

* 78.0000
X l
* 33.0000
X 2
* 27.0736
X 3
* 45.0000
X 4
* 44.9612
X 5
*.(**) -0.00132

g2(X') -91.9998

g>{X') -9.5968
g*{X*) -10.4032
gs(X') -5
g6(X*) 0.0
-31025.56700
Best solution f(X )
-31025.42400
Mean value of *' '
Standard deviation 0.1654058981

4.8 Summary and Conclusions

Most practical optimization problems are considered as constrained

optimization problems, where the search region is restricted by a set of equality and

inequality constraints. The penalty method is one of the methods that attempt to solve

the constrained optimization problems by converting them to unconstrained

optimization problems. The penalty method modifies the constrained objective

function by adding terms to represent the violation of the constraints for unfeasible

solutions. The minimum value of the new objective function is still the same as the

minimum value of the constrained objective function, since there is no violation of

97
the constraints at the feasible optimal solution. The penalty terms of the constraint are

usually multiplied by a factor called the penalty parameter. Penalty methods are

classified as static, dynamic and adaptive based on the penalty parameter. In this

chapter two proposed adaptive penalty methods were developed. The main idea of the

new proposed methods is balanced between the objective function and the constraints.

Most of the penalty methods increase the penalty parameters especially near the

optimal solution. The constraints violations are very small near the optimal solution,

but increasing the penalty parameters made the weight of the constraints higher than

the weight of the objective function, which increased the chance to locate nonoptimal

feasible solutions. On the other hand, if the penalty parameter was not increased in a

sufficient way, this might cause the optimization method to locate an unfeasible

optimal solution. The advantage of the new proposed methods is to have a balanced

change of the penalty parameter. The first proposed method was tested by solving a

set of mathematical benchmark optimization problems. Based on the result of the

proposed method, it can be noted that the first proposed method allows small

violations of some constraints, especially if this results a large change in the objective

function. The second proposed method was the modification of the first proposed

method to avoid constraint violations, where the modified proposed method returns

unfeasible solutions through the iteration to the boundaries of the feasible region.

This can be done if the violated constraints are separable functions. The modified

proposed method was tested by solving the engineering optimization benchmark

problems. The algorithm was located the feasible optimal solution of the engineering

98
optimization problems, but it is noted that when the MSA used the modified penalty

method, the search at the beginning of the iteration is very fast to locate the area near

the optimal solution, where all the searching points located by MSA through the

remaining iteration were feasible, though the algorithm becomes very slow near the

optimal solution.

99
 CHAPTER 5

OPTIMAL POWER FLOW IN POWER SYSTEM NETWORK

5.1 Introduction

Optimal power flow is one of the famous constrained optimization problems.

Here it is required to minimize the cost of generation units based on a number of

constraints and to maintain network stability. In the power system network, there are

several generation units; each generation unit has its capital and running cost. The

total cost of the generation unit C can be approximated by a quadratic equation of the

generated real power P . For the optimal power flow we need to minimize the total

fuel cost of the real powers that are delivered to the power system network based on

the cost of the generation units. The other constraints are related to the power system

network stability such as the voltage magnitude and the voltage angle. Figure 5.1

shows a simple graph of the optimal power flow problem in power system network,

where G, indicates the i!h generation unit.

Figure 5.1. Optimal power flow diagram

100
The main differences of the optimal power flow problem from the previous

constrained problems are: (i) there is no explicit formula for power losses in the

transmission lines and (ii) there are no explicit formulas that relate control variables

in the inequality constraints.

First let us start by illustrating the power flow on the power system network,

where the main parts of the power system network are [48]:

Generation: in this part the mechanical power is transformed to electrical

power. A source of mechanical power such as hydraulic or steam are used

to rotate the shaft of a synchronous generator to produce three phase AC

electrical power [48].

Transmission lines: the transmission lines are used to transfer the

electrical power from generation units to the loads. There are two types of

transmission lines: overhead transmission lines and underground cables

[48].

Transformers: the transformer is a static device used to convert the AC

voltage from one level to another level with high efficiency. Step up

transformers are used at the sending end of transmission line to increase the

voltage and decrease the current in the transmission line which will

decrease the losses in transmission lines. Step down transformers are

connected to the receiving end of the transmission line to return the voltage

and current to a suitable value for the load [48].

101
 - Load: there are several types of loads such as industrial factories and

residential houses.

Buses: the bus is a thick and long piece of copper or aluminum that

connects electrical apparatus in the power system network. The phase

voltage (\v\ZS ) and the apparent power (S = P + jQ) on buses are

necessary information to determine the power flow in the power network

and the network stability. There are three kinds of buses:

• Slack bus: or swing bus, which could be used as a reference bus. The

magnitude of the voltage |F| and the voltage angle 5 are known, and

it is required to calculate the bus active (real) power ( P ) and the bus

reactive power (Q) [48].

• Generation buses: these buses are connected with generation units.

The magnitude of the voltage |F|and the active power ( P ) are

specified while the reactive power (Q) and the angle of the voltage

{8) are unknown [48].

• Load buses: the active power (P) and reactive power (Q) are given at

these buses and it is needed to determine the magnitude of the voltage

(\V\) and its angle (S) [48].

5.2 Power Flow in Power System

The main aim is to find the magnitude and the angle of the voltage at each bus

in the network. After determining the bus voltages of all buses, it is then simple to

102
calculate the power flow on transmission lines, the apparent power on the buses and

the power losses on transmission lines.

The main difficulty in calculating the power flow in a power system is that the

given information at each bus depends on the bus type as mentioned in the previous

section. Mainly, the active and reactive power are the given data in most buses. Based

on the power buses equations this will yield to nonlinear algebraic equations as it will

be shown in this section.

Figure 5.2 shows a typical bus in a power system network [48]. Each bus has an

associated voltage v,, along with incoming and outgoing currents. For instance the ilh

bus is shows in Figure 5.2 has voltage v, and incoming current /,. The outgoing

currents are ix,i2,...,/„. After applying the Kirchoff current low (KCL) [48] on the

bus, the injected current /, at the bus is given by:

/ , = / 0 + z , + z 2 + . . . + /„ (5.1)

i, =y,ovl +y,\(v, -v\) + ya(v, - v 2 ) + --- +x„( v , ~vJ (5-2)

n n

7=0 7=1

Where y : is the admittance of the transmission line connected between bus / and bus

The real and reactive power at each bus are:

P,+jQ,=vX (5.4)

103
where, /* is the complex conjugate of the injected current at the i'h bus.

Substituting equation (5.3) into equation (5.4) yields:

(5.5)

In the JV-bus power system network there will be one slack bus, NL load buses and

iVG generation buses (i.e N = NL + NG +1). The voltages of the load buses are

unknown. To solve for the magnitude of the bus voltage and its angle there will be

two nonlinear algebraic equations for each load bus, and one equation for the

generation bus, so there will be 2NL + Ng nonlinear algebraic equations.

y Vj
'A u
3-
»v-»
b^h

•Jo

iQ

Figure 5.2. Bus on the power system network [48]

To illustrate the power equations, let us start illustrating a simple three-bus power

system network [48] as shown in Figure 5.3. Figure 5.3 shows one line diagram of

three-bus power system network. Bus 1 is a slack bus, bus 2 is a load bus and bus 3 is

a generation bus.

104
 v ^10-120
J V
2=|V2^
"12
-* EJ=400MW
y =10-i30 -» 0,= 250 Mvar
13
V =16 - j32
"23
Slack Bus
Vpl.05/10

v3 = 1.04ZA

/'"^WOOMW

Figure 5.3. Three-bus power system network [48]

To determine the power flow in the transmission lines, we need to determine

the voltage magnitude and voltage angle at bus 2 (|v2|Zc>2), and the voltage angle at

bus 3 (S 3 ).

The equation of the injected current at bus 2 (I2) and the power equations are equal

to:

h = y\l (V2 - Vl ) + ^23 (V2 - V3 ) = ~yUV\ + Ol2 + ^23 )V2 ~ ^23V3 (5.6)

Pl-jQl = 72V2 = -^12 V 2 V 1 +(yu +y23)V2Vl ~ ^23V2V3 (5.7)

The equations at generation bus 3 are:

h = yn (V3 ~ Vl ) + ^23 (V3 ~ V2 ) = ~yUVl ~ ^23 V2 + Ol3 + ^23 )V3 (5.8)

P3-JQ3 = 73V3 =-^13 V 3 V l +(7l3 +^2 3 ) V 3 V 2 ~ ^23V3 V2 (5.9)

105
The voltage and the transmission lines admittances at each bus can be represented in

polar form as:

v,=lvM (5-10)

Let N (5.11)

Transforming equations (5.7) and (5.9) to polar form yields:

Pi-jQi =h||v 2 ||r I 2 |z(^ -s2 +en)+\v2\2\Y22\ze22 +|v2||v3||r23|^(^3 -s2 +e23) (5.12)

P3-JQ3 = h l K | | r 1 3 | z ( ^ - s 3 +^3)+|V3|2|F33|^^33 +|v 2 ||v3||r 23 |z(5 2 -s 3 +e23) (5.13)

For the load bus 2 Equation (5.12) can be divided to two equations as follows:

p
i =h|v2|ft2|cos(($i -S2 + 6>12) + |v2|2|F22|cos(6>22) + |v2||v3||F23|cos(^3 -82 +923) (5.14)

Qi =+i|v 2 |y; 2 |sin(<5 I -82 +^ 2 )-|v 2 | 2 |7 22 |sin(^ 22 )-|v 2 ||v3|F 23 |sin(^3 -82 +023) (5.15)

For the generation bus 3 the magnitude of the voltage is given. The active (real)

power equation will be used to determine the voltage angle as follows:

P3 = |v,l^ll^|cos^, -S3 +#i3) + | v 3| |^331cos(#33)+|v2||v3||]'23|cos(c>2 -S3 +623) (5.16)

Returning back to the three- bus power system network in Figure 5.2, the numerical

values of the system data [48] are:

106
 Y
\2 =~y\2 = - 1 0 + 7"20 = 22.36Z2.034
Y22 = yl2+y23 = 26-y52 = 58.137Z-1.107
Y
2i = -y-a = ~ 1 6 + J32 = 35.78Z2.034
Yu =-yu = - 1 0 + y30 = 31.62Z1.89
(5 17)
^33 =^13+^23 = 2 6 - ; 6 2 = 67.23Z-1.17 '
|v,| = 1.05
Sx = 0
|v2| = 1.04

The values of the active and reactive power are usually represented in per unit. The

per unit value is calculated by dividing the real value of the power by the base power

value. The power is usually represented as a signed value to distinguish between the

generation power and the demand power, where the minus sign represent the demand

power and the positive sign represent the generation power. The values of the active

and reactive power of the three- bus power system network based on based power

=100 are:

P2 = -400MW = ^ ^ = -4 pu
2 y
100
-250
Q2 = -250Mvar = — — = -2.5 pu (5.18)
^2 1Q0 F v ;
P3 = 200MW = — = 2 pu
3
100

Substituting the numerical values into the equations (5.14),(5.15) and (5.16) yields:

P2 =23.48|v2|cos(-£2 +2.034) + 26|v2|2 +37.2l|v2|cos(£3 -S2 +2.034) = ^ ! (5.19)

Q2 =-23.48|v2|sin(-£2 + 2.034)+ 52|v2|2 -37.2l|v2|sin(J3 -82 +2.034) = -2.5 (5.20)

P3 = 34.53cos(-£3 +1.8952) + 28.1216 + 37.2l|v2|cos(J2 - S3 +2.0344) = 2 (5.21)

107
Equations (5.19), (5.20) and (5.21) are called the power flow equations, and are

nonlinear algebraic equations of three unknown variables |v21, S2 and S3. Numerical

technique such as Newton Raphson method can be used to solve the power equations

[48].

5.3 Newton Raphson Method for Solving Algebraic System of Equations

The Newton-Raphson method is an iterative technique that can be used to

solve nonlinear algebraic equations [49]. To solve the single variable equation that is

given by:

/(*) = 0 (5.22)

let x0 be the initial estimation of the root of f{x). Then the function at x0 can be

approximated by a tangent line using Taylor series as follows:

f(x)»Ax0) + (x-x0)f'(x0) (5.23)

The root of the tangent line will be the next estimation of the function root as follows

*,=*o-~4- (5-24)
x
/ \ o)

In general the equation of the new estimation of the root at n iteration is

^^-.-TTH- (5-25)
/Cvi)

The method converges to a solution when the relative error in the root is less than a

certain tolerance ( sx) and the relative error on the function evaluation is less than a

function tolerance ( s f)

108
 <er
(5.26)
f{xn)-f{xn_x)
<£,
/(*„)
Consider a set of N equations as follows:

7iW
F(X) (5.27)

fN(X)

where X = [x,, x2. • • •, xN ]

The solution of the equations using Newton Raphson method will be as follows

(
xw =x "-l) +AX("-]) =X(n-X} -J-^iX^FiX1"^) (5.28)

5/, df2 dfN

dxx dx2 dxN
df2 df2 dfN
where J: is the jacobian matrix J = <3x, dx2 dxN (5.29)

dfN V* ¥N
dx, dx. dx,

5.4 Application of Newton Raphson Method to Solve Power Flow Equations

The power flow equations for N-bus power system can be formulated in order

to use the Newton-Raphson method as follows [48]:

The admittance matrix of the N-bus power system network is

109
 Y Y 1
l
\N
-Ml \2

Y = Y2\ Y l
2N
1 i
22

Y Y 1
NN
V.1 NX l
N2 (5.30)
Where:
Y =Y =-v
N

i=j
1=0

The injected current at each bus is

i =YYV (5.31)
/=i

Writing equation (5.31) in polar form

(5.32)

The apparent power at the bus / is given by:

S,=P,+jQ,=VX (5.33)

Substituting equation (5.32) into equation (5.33) yields

(5.34)

^=ZWWW cos ( <y .- <y /-^)-

1=1
(5.35)

(5.36)
J=\

110
The partial derivatives of the active power Pt and the reactive power Q, respect to the

\V\ and 5, are given by [48]:

N
dP i ii i
^ =ZWKW s i n (^-^+^)>
L
(5-37)

^ = -|F,|^||^|sin(^-^+^) j*i, (5.38)

N
dP I I
^-r2\V, \V, cos(0„) + £ r , r, cos(£, -5, +0V), (5.39)

^ = |F,||^|cos(^-J,+^) j*i, (5.40)

BO " i M i

w
dO II i II i
^ = -2\Vl\\Vl\sm(eil)-Y,\Vl\\YIJ\sm(SJ -5, +0„),and (5.43)

• ^ = -|F,||yw|sin(£, -5l+0„) j * i. (5.44)

For a load bus, the active power P and the reactive power Q are given and

they will be used to determine the voltage at the load bus, so equations (5.35) and

(5.36) will be used to evaluate the bus voltags. For the generation bus, the magnitude

111
of the voltage is known and we need only the active power equation (5.35) to

determine the voltage angle at the generation bus.

The active power equation (3.35) will be applied for all the buses except the slack bus

yields to (NG + NL = N -1) nonlinear equations. The reactive equation (5.36) will be

applied for the load buses only, so there will be NL equations. The total power flow

equations are [48]:

Jx J2 Ad(k)
(5.45)
AQik) J3 J4 A\v\{k)

where

APXW
psch

psch
p(k)

p(k)
\m(k)] 'Qch-$k)~
k) AR (*) r
2 r
2 AQ2(k) Qlch-$]
M* = , A ^ = = (5.46)

AP{k) N0+NL
jsch
Na+NL r
p(k)
NG+NL _ J&\_ <-C
(*) (k)
dQSk)
(k)
8P 8PX ' 8Q
85, d5
NG + NL 85, 85 N + N,
G

•A = ,Jl = (5.47)
(k) {k) (k) (k)
8PNg + N, dp '
Ur
dQ, dQ„,
Nc,+N,
85, 8S
NC + NL 85 N + N
G L
85„

(k) (k) (k) (k)

8^ 8PX
a d
8\V,\
KI 8\v,\
K (5.48)
J, > J* =
(k) (k) (k) (k)
8PN G + NL
8P
U1
5QNL SQu
d N +N G

8\V,\
K 8\V,\ 8\VK,_

112
 \(k)
AS[k) AVt
AS (*) _ and A\V\ik) = (5.49)
\(k)
AS(k)

Then the new estimates of the bus voltage are:

(*)
;(*+!) _ W
= J£(.k) +A£ (5.50)
(k+X) (k) (k)
\V\ =\v\ + A\v\

Where:

8{xk) l>.l (*)

£w = 1 2|
•\(k)
(5.51)
(K)
S(k)
U
NG + NL . ^

Where: P'ch ,Q"h: the schedule active power and reactive power at each bus

respectively.

Solving power flow equations using Newton-Raphson method can be

summarized as follows [48]:

1. Start from initial estimation of the magnitude of bus voltage for load

buses (usually 1 per unit). The magnitude of the voltage of the

generation buses is specified.

2. Start from initial estimation of the bus voltage angles and it are usually

starting from zero radian.

3. Compute the P,{k),Q^k) from equations (5.35) and (5.36).

4. Compute AP(K),AQW from equations (5.46).

113
 APik)\<£
5. If ! W
', , then terminate the iteration method otherwise go to
|A0 |<^'i

step 6.

6. Calculate the jacobian matrix J = using equations (5.47)

and (5.48).

7. Solve for AS(k\&\vfk) using equation (5.49).

8. Increase the number of iteration (k +1) and the new estimation of the

buse voltages can be calculated from equations (5.50).

9. Go to step 3.

Return back to the three-bus power system network example and use Newton

Raphson method to solve the power flow equations that described by the equations

(5.17), (5.18) and (5.19). The numerical values of the |v 2 |,£ 2 and S3at each iteration

are listed in Table 5-1. The Newton Raphson method converges to a solution of

0.000225 desired accuracy in three iterations as shown in Table 5-1 [48].

Table 5-1 Power flow of the three-bus power system network using Newton Rraphson
method [48]

Num. iteration Error

hi 52 ^3

[pu] or [rad] [rad]

[dimensionless]
0 1 0 0
1 0.9735 -0.0453 -0.0077 0.0217
2 0.9717 -0.0471 -0.0087 0.0002166
3 0.9717 -0.0471 -0.0087

114
5.5. Optimal Power Flow

The objectives of the optimal power flow on the power network are to

minimize the fuel cost of generation units and to insure that the generation power

meets the demand power and the total power losses through the transmission lines to

keep the network within secure limits.

The power system network has several types of generation plants with

different fuel costs for each one. The cost is usually given by quadratic empirical

equation as follows [48, 50, 51]:

C,(Pl) = a,P,2+blPl+cl (5.52)

where: a, ,bl and c, are constants depending on the type of the generation plant. It is

further assumed that a physical power constraint exists for each plan and it is given

by

P • <P <P (5 53)

•* imm — ± i — A imax \^.->uj

The optimal power flow problem can be formulated as follows [50]:

ng
Objective : Minimize C, =^alPl2 +blPl + c, ng : number of generator units.

Subject to: Pimin <Pt<Pimax i = l,2,...,ng (5.54)

Q,min ^Q,^ Q,max i = h2,...,ng (5.55)

S^ZS^S^ i = \,2,...,n (5.56)

\Vimin\AV\AVimax\ i = \,2,...,n (5.57)

115
 ng nd
P =
Z > Z PL> + P
"»ses n
d • Number of load buses, (5.58)

1=1 i=\

Phases '• Power losses on transmission lines.

The first and the last constraints are called direct constraints because they are

functions of the real power (Pt ) so they are affecting directly the objective function.

The other constraints are called indirect constraints, because they affect the objective

function indirectly [50], through transmission line power losses and power system

network stability. The indirect constraints can be checked using the power flow on

power system network. If the optimal power flow solution does not satisfy the

reactive constraints, then it will be ignored. The total losses on transmission lines is a

non deterministic function of the voltage and the apparent power of the buses (i.e
p^ftp.&ns,)).
The power flow in a power system network changes based on any change on

the generation power distributions. For example, there may be three generation units

in a power system network and it is required to generate a 300 MW power to achieve

the demand power and the losses through the transmission lines. The power flow in

the transmission lines will be changed if the power from the generation units is

changed from one sequence (i.e. Pgl =100MW,Pg2 =\00MW,Pgi =100MW ) to

another sequence (i.e. Pg] = 50MW,Pg2 =80MW,Pg3 =\70MW) and the

transmission line losses will also be changed. For some power distribution sequences,

the power system may lose its stability, and this can be checked from the indirect

116
constraints. If there are some violations in the indirect constraints, then the system

will be unstable (i.e. the magnitude of the voltages exceed their limits).

The optimal power flow problem can be divided into two stages. The first

stage is to minimize the fuel cost based on the objective function and the direct

constraints, where the power losses through transmission lines are treated as constant.

The second stage is to calculate the power flow in the transmission lines in order to

check the power system stability and the power losses through transmission lines.

Figure 5.4 shows the input and the output of each stage.

Power flow stage Optimization stage

Po%ver losses
Indirect constraints Objective ftmction
Power flow equations
•violation
Direct constramts

p
G- Pgi V- y

Figure 5.4. Optimal power flow stages

Figure 5.5 shows a five- bus power system network [48], where there are three

generation buses; bus 1, bus 2 and bus 3. Bus 1 is considered a slack bus in this

network. There are two load buses in this network; bus 4 and bus 5. Even though

there is load connected to bus 2 and bus 3, they are considered a generation buses.

The costs of the generation units are shown in Table 5-2. The given data of the five-

bus power system network is listed in Table 5-3. The blanked boxes in Table 5-3 can

be determined after solving the power flow equation of the network.

117
 V 3 | = 1.03

^50MW
30Mvar

0.08+JG.24

60MW
4-OMvar
f^—^\ 20MW

IV2 |=1.045

Figure 5.5. Five-bus power system network [48]

Table 5-2 Fuel cost of the generation buses of the five-bus network

Generation cost (C) Power limits

Bus#
1 C, = 200 + 7P1+0.008P12 10 <P, <85
2 C2 =180 + 6.3P2+0.009JP22 10<P2<80
3 C3 =140 + 6.8P3 +O.OO7P32 10<P 3 <70

Table 5-3 Given data of the five-bus power system network

Bus# Vo tage Load Generation

Mag. angle MW Mvar MW Mvar
1 1.06 0.0 0.0 0.0
2 1.045 20 10 40
3 1.03 20 15 30
4 50 30 0 0
5 60 40 0 0
Total 150 95 70 0

118
 The power flow of the five-bus system network with the given generation

power in Figure 5.4 (i.e Pg2 = 40MW,P g3 = 30MW) is listed in Table 5-4. The total

cost of the generation units is 1633.24 $/h and the total transmission lines losses is

3.05248MW .

Table 5-4 Power flow of the five-bus power system network

Bus# Vo ltage Load Generation

Mag. angle MW Mvar MW Mvar
1 1.06 0.0 0.0 0.0
2 1.045 20 10 40
3 1.03 20 15 30
4 50 30 0 0
5 60 40 0 0
Total 150 95 70 0

The optimal power flow of the five- bus power system network using MSA is

shown in Table 5-5. Starting from the initial values of the generation power

PgX =80MW,P g2 =40MW,P g3 =30MW, the output from the MSA is

Pg] =32.6254MW, Pg2 = 66.9891MW, Pg3 = 50.3856MW and the total fuel cost is

1579.70$/h. The output from the optimization method is now considered input to the

power flow method to calculate the voltages and the current at the buses. The

generation power from the slack bus is modified in the power flow calculation to

achieve the desired generation voltage, then the output of the power flow method is

PgX =34.8688MW, Pg2 = 66.9891MW, Pg3 = 50.3856MW with total generation cost

1596.62 $/h and total losses of the transmission lines losses is 2.2434MW. The output

119
from the power flow calculation will be input to the optimization calculation, and the

total loss through the transmission lines is kept constant. The procedure will continue

as listed in Table 5-5 until the optimal solution satisfies the five-bus requirements.

Table 5-5 Optimal power flow of the five-bus power system network

Initial values for the optimization function Output from the optimization
calculation
P P p Fuel P P Fuel
,2 # losses .2 #
[MW] [MW] [MW] Cost [MW] [MW] [MW] Cost
[MW]
[$/h] [$/h]
80 40 30 0 1607.9 32.625 66.989 50.3856 1579.7
34.86 66.989 50.3856 2.243 1596.62 32.720 67.782 51.6417 1595.81
32.80 67.782 51.6417 2.2245 1596.42 32.686 67.854 51.685 1596.42

5.6 Optimal Power Flow Solution Using Lagrange Multiplier Function

To use the Lagrange multiplier method for solving the optimal power flow

equations, an explicit formula of the power losses through the transmission lines is

needed [48]. Based on the reasonable assumptions and simplifications the power

losses through the transmission lines can be represented by a quadratic function of the

generation power

ng ng ng

pL= BU + XWKPMZLWW ( 5 - 59 )

The coefficients B are called loss coefficient and they are defined as follows [48]:

120
 B
12 \ng BJ2'
5, B 22 B 2ng B02/2
H +H
= *[//] = (5.60)
B
5„Vigl 5„ngl B,ngng onJ2
2
B
0\l2 B
0l/2 W B00

where;

H= ¥'C'RhmC¥
(5.61)

. f i ?i
1-7
^>

.Q g2
1-7
g2
¥ = (5.62)
r;
• xfgng
1-7
g»g
0 0
V'
ng
0 0 0

and 70 : is defined as the total current flowing away from bus 1.

~11 ~12

z z v
2n
(5.63)
*A,=9^,J = * ^21 ^22

z z ••• z„

121
 1 0 0 0
0 1 0 0
1 0
PXZU P\£>22 P\Z\ng P\Zn
c= (5.64)
PlZ\\ PlZ 22 PlZ\ng PZu
PlZ\\ P^Z22 P3Z\ng P\Z\\

PkZU PkZ 22 PkZ\ng PkZ\

Pk (5.65)
T

l>=^ ,k = \,2,---,nd, and (5.66)

nd
(5.67)
k=\

where, nd is the total number of the load buses and 7 0 is the total demand current.

The Lagrange multiplier method converts the optimal power flow problem to

unconstrained optimization problem by adding the constraints given in eqs.(5.54)-

(5.58) to the cost function in eq.(5.52) as follows [48]:

ng ng ng ng ng

1=1 ;=i 1=1 1=1 1=1

ng ng
Z(^(^-^max)) + E(^(^-^m,„)) (5.68)
i=l (=1

The partial derivatives of the Lagrange function are:

ng
8L_
= 2a,Pt +b, + A[2BuPt + 2 £ ( 5 , / P / ) + 5 0 , -l] +Mu + Mll = 0, i = 1,2,...,;
dP (5.69)

122
 ar "g ng ng ng
— = YJ(Pt)-PD-(ZId(PA!Pl) + Z(PA,) + BOQ) = o
VA i=l i=l /=! i=l (5.70)

dL
= P,--Cax=0
(5.71)

8L
- P, - ^ m ,mn
= =P,-r, „ ==o
o
°V* (5.72)

The value of /uu and /J2I are zeros whenP(min < Pt < PimaJi, so there is no need

to solve equations (5.71) and (5.72).

Initially, it is assumed thatP;min < Pt <P, max so to find the optimal power flow,

equations (5.69) and (5.70) are needed to be solved. Equ.(5.69) can be rewritten as

follows:

»g
Ml-BJ-b, -2A£(V,)
P
'- 2a,+2B„Jr ' '"U"-"' (5 73>
'
Substituting the values of Pt from equ.(5.69) into equ.(5.70) yields

ng
A(\-BJ-b, -2A£(5„/»,)
7=1.
ng >*J ng ng ng
+
Z T~^~7 = PD+CZZ(PAP,) 11(PA,) + BW) (5.74)
i=l ^<3, + lDnA , =1 /=I (=]

The Newton-Raphson method can be used to solve equ. (5.74) with respect to A . The

value of X at each iteration can be given by:

123
 Ak =Ak-' + (5.75)
/'(A*' 1 )

where :

ng ng "g ng
(5.76)
(=i (=i

"g ng
(a,+5^)0-^,-2 X^^)-A,Wl-^o,)-^ -2/1 X(V,))
/ W = (5
K ±*rt • -7?)
2(a, + BltA)

5.7 Optimal Power Flow of IEEE-26-Bus Power System Network

The one line diagram of a IEEE-26-bus power system network is shown in

Figure 5.6 [45]. The parameters of the transmission lines, the tap transformers, the

buses and the generation units are given in appendix A.

^
f© ^ = ^ E 3 ^
26 •

is Itr 6
e -13

T
12-
T
14 JL r
li—!1 25- 19
T7
2-1 23 15.
J IT TT
.X
J~L
- 0 -r—i—
1 4^

17

Figure 5.6. One line diagram of IEEE-26-bus power system network [48]

124
The result of the optimal power flow using Lagarange method by Sadaat [48] is

shown in Table 5-6.

Table 5-6 Result of optimal power flow of IEEE-26-bus power system network using
Lagrange multiplier method [48]

Bus Bus Voltage Load Generation Injected

No.
Mag. Angle Active Reactive Active Reactive Reactive
[Pu] [degree] Power Power Power Power Power
[MW] [Mvar] [MW] [Mvar] [Mvar]
1 1.0250 0.000 51.00 41.00 447.6 250.6 4.00
2 1.0200 -0.2001 22.00 15.00 173.1 57.30 0.00
3 1.0450 -0.6393 64.00 50.00 263.4 78.28 0.00
4 1.0500 -2.101 25.00 10.00 138.7 33.45 2.00
5 1.0450 -1.453 50.00 30.00 166.1 142.9 5.00
6 1.0007 -2.874 76.00 29.00 0.00 0.00 2.00
7 0.9953 -2.406 0.00 0.00 0.00 0.00 0.00
8 0.9983 -2.278 0.00 0.00 0.00 0.00 0.00
9 1.0104 -4.387 89.00 50.00 0.00 0.00 3.00
10 0.9909 -4.311 0.00 0.00 0.00 0.00 0.00
11 0.9979 -2.824 25.00 15.00 0.00 0.00 1.500
12 0.9943 -3.282 89.00 48.00 0.00 0.00 2.00
13 1.0224 -1.261 31.00 15.00 0.00 0.00 0.00
14 1.0081 -2.445 24.00 12.00 0.00 0.00 0.00
15 0.9988 -3.229 70.00 31.00 0.00 0.00 .500
16 0.9901 -3.990 55.00 27.00 0.00 0.00 0.00
17 0.9827 -4.366 78.00 38.00 0.00 0.00 0.00
18 1.0075 -1.884 153.00 67.00 0.00 0.00 0.00
19 1.0046 -6.074 75.00 15.00 0.00 0.00 5.00
20 0.9830 -4.759 48.00 27.00 0.00 0.00 0.00
21 0.9771 -5.411 46.00 23.00 0.00 0.00 0.00
22 0.9796 -5.325 45.00 22.00 0.00 0.00 0.00
23 0.9780 -6.388 25.00 12.00 0.00 0.00 0.00
24 0.9691 -6.672 54.00 27.00 0.00 0.00 0.00
25 0.9751 -6.256 28.00 13.00 0.00 0.00 0.00
26 1.0150 -.2839 40.00 20.00 86.94 27.89 0.00
Total 1263.00 637.00 1275.8 590.396 25
Total generation cost = 154^\1.12%lh
Tota power lossses on transmission lin es= 12.800:!MW

125
The result of the power flow of IEEE-26-bus network using the Multi-Scale

Algorithm is shown in Table 5-7.

Table 5-7 Result of optimal power flow of IEEE-26-bus power system network using
MSA
Bus Bus Voltage Load Generation Injected
No.
Mag. Angle Active Reactive Active Reactive Reactive
[Pu] [degree] Power Power Power Power Power
[MW] [Mvar] [MW] [Mvar] [Mvar]
1 1.0250 0. 51.00 41.00 446.1 245.2 4.00
2 1.0200 -.2070 22.00 15.00 167.9 37.10 0.00
3 1.0450 -.6370 64.00 50.00 263.1 68.04 0.00
4 1.0600 -2.204 25.00 10.00 134.9 75.42 2.00
5 1.0450 -1.240 50.00 30.00 173.6 136.2 5.00
6 1.0020 -2.809 76.00 29.00 0.00 0.00 2.00
7 0.9990 -2.435 0.00 0.00 0.00 0.00 0.00
8 1.0030 -2.331 0.00 0.00 0.00 0.00 0.00
9 1.0150 -4.407 89.00 50.00 0.00 0.00 3.00
10 0.9950 -4.340 0.00 0.00 0.00 0.00 0.00
11 0.9990 -2.719 25.00 15.00 0.00 0.00 1.500
12 1.0000 -3.337 89.00 48.00 0.00 0.00 2.00
13 1.0230 -1.261 31.00 15.00 0.00 0.00 0.00
14 1.0100 -2.450 24.00 12.00 0.00 0.00 0.00
15 1.0010 -3.238 70.00 31.00 0.00 0.00 .500
16 0.9930 -3.992 55.00 27.00 0.00 0.00 0.00
17 0.9840 -4.323 78.00 38.00 0.00 0.00 0.00
18 1.0080 -1.860 153.00 67.00 0.00 0.00 0.00
19 1.0070 -6.021 75.00 15.00 0.00 0.00 5.00
20 0.9870 -4.772 48.00 27.00 0.00 0.00 0.00
21 0.9800 -5.361 46.00 23.00 0.00 0.00 0.00
22 0.9840 -5.331 45.00 22.00 0.00 0.00 0.00
23 0.9810 -6.355 25.00 12.00 0.00 0.00 0.00
24 0.9720 -6.637 54.00 27.00 0.00 0.00 0.00
25 0.9780 -6.189 28.00 13.00 0.00 0.00 0.00
26 1.0150 -.1030 40.00 20.00 89.82 26.29 0.00
Total 1263.00 637.00 1275.5 588.256 25
Total g eneration cost = 15442..9064$/h
Tota power los ses on transmission Hiies= 12.500 MW

126
 The total generation cost using the Lagrange multiplier method is 15447.72$/h

and the real power losses on transmission lines are 12.8003 MW [48]. While the total

generation cost using the multi-scale algorithm is 15442.9064 and the real power

losses on transmission lines are 12.5 MW. The multi scale algorithm has 4.8136$/h

generation cost less than the Lagrange multiplier method, which means that the

annual total saving is more than 42,000 dollars.

5.8 Optimal Power Flow of IEEE-30-Bus Power System Network

The one line diagram of IEEE-30-bus power system network is shown in

Figure 5.7 [50,51,52]. The IEEE -30-bus data of the transmission lines, the tap

transformers, the buses and the generation units are given in appendix B. The optimal

power flow of the IEEE-30-bus power system network using MSA is listed in Table

5-8.

Figure 5.7. One line diagram of IEEE-30-bus power system network [50]

127
Table 5-8 Results of optimal power flow of IEEE-30-bus using MSA

Bus Bus Voltage Load Generation Injected

No.
Mag. Angle Active Reactive Active Reactive Reactive
[Pu] [degree] Power Power Power Power Power
[MW] [Mvar] [MW] [Mvar] [Mvar]
1 1.060 0 0 0 176.5180 -7.899 -7.8990
2 1.045 -3.599 21.70 12.70 48.1550 28.031 28.03
3 1.031 -5.550 2.40 1.200 0 0 0
4 1.023 -6.666 7.60 1.600 0 0 0
5 1.010 -10.305 94.20 19.00 19.7470 21.195 21.19
6 1.024 -7.831 0 0 0 0 0
7 1.010 -9.371 22.80 10.90 0 0 0
8 1.025 -8.074 30.00 30.00 23.6980 33.590 33.59
9 1.058 -9.819 0 0 0 0 0
10 1.052 -11.559 5.80 2.00 0 0 0
11 1.082 -8.544 0 0 12.2470 12.720 12.72
12 1.063 -10.741 11.20 7.500 0 0 0
13 1.071 -9.869 0 0 12.3810 6.015 6.015
14 1.049 -11.639 6.20 1.600 0 0 0
15 1.044 -11.750 8.20 2.500 0 0 0
16 1.051 -11.358 3.50 1.800 0 0 0
17 1.046 -11.704 9.00 5.800 0 0 0
18 1.035 -12.368 3.20 0.900 0 0 0
19 1.032 -12.547 9.50 3.400 0 0 0
20 1.036 -12.357 2.20 0.700 0 0 0
21 1.040 -12.013 17.50 11.20 0 0 0
22 1.040 -12.005 0 0 0 0 0
23 1.034 -12.193 3.20 1.600 0 0 0
24 1.029 -12.446 8.70 6.700 0 0 0
25 1.028 -12.288 0 0 0 0 0
26 1.010 -12.699 3.50 2.300 0 0 0
27 1.036 -11.934 0 0 0 0 0
28 1.023 -8.343 0 0 0 0 0
29 1.016 -13.134 2.40 0.900 0 0 0
30 1.005 -13.995 10.60 1.900 0 0 0
Total 283.400 126.200 292.746 93.6520 23.3000
Total igeneration cost = 801 9993$/h
Tc tal power lo sses on tra nsmission ]ines = 9.346 MW

128
 There are several methods were used to find the optimal power flow of the

IEEE-30-bus power system network, such as: Evolutionary Method by Yuryevich

[52], Genetic Algorithm by L. Lai [53] and Simple Genetic Algorithm by Bouktir

[50].

The results of optimal power flow of the IEEE-30-bus power system network

using Genetic algorithms, Evolutionary method and Multi-scale method are shown in

Table 5-9. The Multi scale method has the least generation cost compare to the other

methods (see Table 5-9).

Table 5-9 Results of optimal power flow of IEEE-30-bus using different optimization
methods [50] and MSA
Method Total generation cost Real power losses
($/h) [MW]
Genetic Algorithm L. Lai 802.4484 9.6372
Evolutionary Method 802.5557 9.3683
Yuryevich
Genetic Algorithm T. Bouktir 803.699 9.5177
Multi-Scale Algorithm 801.9993 9.3467

W. Liu, M. Li and X.Wang [54] proposed an improved algorithm of practical

swarm optimization to solve the "Premature convergence" problem in swarm

optimization algorithm. The improved algorithm (IPSO) was proposed to solve the

optimal power flow of IEEE 30-bus power system network. They compared the result

of the optimal power flow using IPSO with two other optimization methods; the

Genetic Algorithm (GA) and the Practical Swarm Optimization (PSO). The result of

the optimal power flow using these optimization methods were listed in Table 5-10.

129
 I. Oumarou, D. Jiang and C. Yijia [55] were solved the optimal power flow of

IEEE 30-bus using a simple swarm optimization method PSOOPF. They compared

their results with the Matpower method (see Table 5-10). "The Matpower was created

by Ray Zimmerman and Deqiang Gan of PSERC at Cornell University under the

direction of Robert Thomas. Its main aim was to provide a simulation tool within

Matlab that was easy to use and modify" [55 page 287].

P. Vu, D. Le, N. Vo and J. Tlusty [56] proposed a Weight-Improved Particle

Swarm Optimization (WIPSO) method to solve the optimal power flow problem of

the IEEE 30-bus power system network. In this paper, the voltage of the slack bus is

modified, where the algorithm located the best solution of the optimal power flow

compared to other optimization methods (799.1665$/h). The optimal power flow of

IEEE 30-bus was calculated by MSA subject to a fixed slack bus voltage, which

means that the optimal power flow in this paper is a different problem compared to

the optimal power flow that was solved by the MSA. By investigating the results that

were shown in this paper, the best result for the same slack bus voltage (1.06 pu) was

found by differential evolution algorithm (DE), where the total generation cost of the

IEEE 30-bus was 802.230 $/h.

S. Sayah and K. Zehar [57] solved the optimal power flow problem using

Improved Differential Evolution algorithm (IDE). The voltages of the generation

buses were not fixed in this paper, and they were modified to locate the optimal

power flow of IEEE-30 bus. Case 2.1 in this paper has the similar voltage buses as it

was used in the MSA. The result of case2.1 was shown in Table 5-10.

130
 H. Hoseinabadi, S. Hosseini and M. Hajian [58] proposed the Modified

Practical Swarm Optimization Algorithm (MPSO) to solve the optimal power flow

problem of IEEE 30-bus power system network. The modification in this algorithm is

that each individual in the swarm group learns from the best and the worst experience

of all the swarm individuals. The result of the optimal power flow of IEEE 30-bus

power system network using the MPSO was compared to the optimal power flow

solutions of the practical swarm optimization algorithm (PSO), Improved practical

swarm optimization (improved PSO) and Evolutionary programming (EP). The

optimal power flow of these optimization methods are listed in Table 5-10.

A. Basak, S. Pal, V. Pandi, B. Panigrahi and S. Das [59] solved the optimal

power flow of IEEE 30-bus power system network using several optimization

methods such as: Hybrid of Invasive Weed Optimization (IWO), Simple Bacterial

Foraging (SBF), Genetic Algorithm (GA), Invasive Weed Optimization (IWO),

Differential Evolution (DE), Hybrid Bacterial Foraging and Differential Evolution

(HBFDE) and Adaptive Bacterial Foraging with Nelder Mead (ABFNM). The results

of the optimal power flow were listed in Table 5-10.

131
Table 5-10 The optimal power flow of IEEE-30-bus using several optimization
methods

Optimization Total Power Running time Processor

Method Generation Losses [s] [GHz]
Cost [$/h] [MW]
GA [54] 834.46 10.67 1645
PSO [54] 831.16 10.10 1053
IPSO [54] 827.13 9.12 926
Matpower [55] 802.1 9.41
PSOOPF [55] 801.995 9.384 15 1.5
IDE [57] 802.37 9.4597 23.25 3.0
MPSO [58] 802.398 9.5
PSO [58] 803.397 9.974
Improved PSO [58] 802.63
EP [58] 802.62
DIWO [59] 803.0206 9.6503
IWO [59] 803.0206 9.6503
DE [59] 803.0206 9.6503
ABFNM [59] 803.0206 9.65
HBFDE [59] 803.0206 9.648
SBF [59] 803.022 9.645
GA [59] 803.032 9.719
MSA 801.9993 9.3467 3.242 2

The blank cells in Table 5-10 mean that there is no available data. The MSA

has the least total generation cost compared to the other optimization problems that

were listed in Table 5-10. For the total power losses the IPSO [51] has the least total

power loss (9.12 MW), while the total generation cost is 827.13 $/h which is higher

than the generation cost was found by the MSA. The CPU time for the MSA is

consider small compared to the other optimization methods.

5.9 Summary and Conclusions

The optimal load (power) flow in the power system network is a constrained

optimization problem. Here it is required to minimize the cost of the generation units

132
subject to a set of equality and inequality constraints to maintain the power system

network stablity. The constraints in this problem can be divided into two groups;

direct and indirect constraints. The direct constraints are function of the real power

control variables Pt which are the same as the control variables of the objective

function. The indirect constraints are functions of the other control variables that

affect the objective function indirectly such as Vl,Ql and S,. The difficulty of the

optimal power flow problem is that there is no explicit formula to calculate the power

losses in transmission lines and there is also no explicit formula to calculate the

violation of the indirect constraints. The power losses through transmission lines and

the indirect constraints violations can be determined by solving a set of nonlinear

algabric equation numerically. The MSA was used to calculate the optimal power

flow of IEEE-26-bus and IEEE-30- bus power system networks. The optimal power

flow solution that was found using the MSA was better than other optimization

methods.

133
 CHAPTER 6

FUTURE WORK AND CONCLUSIONS

6.1 Conclusions

The Multi-Scale algorithm (MSA) is a new optimization algorithm based on

the genetic approach. It uses a real coded representation to decrease the disadvantage

of coding operations. An adaptive mutation operation is used to provide a large scale

at the beginning of the search and then to a small scale in the immediate

neighborhood of the optimal solution. The adaption is done by selecting the most

significant digits to be changed in the first steps of the search then selecting the least

significant digits in the final iterations, where the probability to choose any digits

through the search will never be zero.

MSA successfully located the optimal solution of several unconstrained

optimization problems with objective functions having different features such as

multiple local optimum points, variable frequency components and flat regions

around the optimum points. In each case the MSA successfully converged to the

global optimal point.

The violation of the constraints near the optimal solution is very small and it

may be not sufficient to maintain the search inside the feasible space, for that reason

the penalty parameter must be modified to increase the weight of the penalty function

when constraints are violated. Increasing the penalty parameter above a certain limit

134
may slow down the optimization method and decrease the effect of the objective

function, which means that it increases the probability of locating a feasible solution

more than locating the optimal feasible solution. For constrained problems, two new

methods based on the penalty function were proposed to deal with the constraints.

The penalty parameter was modified adaptively through the search toward the

optimal solution. The penalty parameters in the proposed methods are changed based

on the variations of both of objective function and penalty function. The main aim of

the penalty parameter modification is to allow a smooth and balance weighting

between the objective and the penalty functions to obtain an optimal feasible solution.

The first proposed method with the MSA was used to locate the optimal solution of

several constrained optimization problems. The method succeeds in locating feasible

solution of most of them while it allowed small constraints violation of the other test

functions, especially when small violation of constraints allow a large change in the

objective function.

A second penalty method was proposed to permit constraints violations, so

that when the MSA locates unfeasible point, this point is replaced immediately by a

point located in the border of the feasible space near to the current unfeasible point.

The second proposed method with the MSA was used to locate the optimal solution

of several engineering optimization problems. The method successfully located

optimal feasible solutions. The drawback of the second proposed penalty method that

135
it is very fast at the beginning of the optimal solution search then it becomes very

slow near the optimal solution.

Finally, MSA was used to solve the optimal power flow for a power system

network. The optimal power flow is one of the important optimization problems in

power system networks, the objective function of the optimal power flow is to

minimize the fuel cost of the generation units and maintain the power system network

in secure limits. The main features of the optimal power flow problem are; (i) there is

no explicit formula for the power losses in transmission lines, where the power losses

can estimated numerically after solving the power flow equations a set of nonlinear

algebraic equations (ii) some of constraints affect the objective function indirectly

and the constraints violation can only be determined after power flow calculations.

The optimal power flow of IEE-26-bus power system network was calculated using

MSA and the Lagrange multiplier method, in the Lagrange multiplier method the

power losses was represented by an approximation formula. The results show that

MSA achieved better result in sense of the total cost and the power losses in

transmission lines.

MSA was also used to determine the optimal power flow of IEEE-30-bus power

network. Comparing the results obtained by other optimization methods such as

Genetic Algorithms, Practical Swarm Optimization, and Evolutionary Optimization

methods shows that the MSA locates better results compared to these methods.

136
6.2 Future Work

The random generator in the mutation operation is triangle membership function.

There are other possibilities of the membership functions that may be used such as

trapezoidal, Gaussian,...etc.

The mutation operation is the only operation used in the MSA to generate

offspring. The crossover operation is another possible operation that can be used to

generate them.

Even though the two proposed penalty methods located the optimal solution of

most of the test function successfully, they still have some drawbacks. The first

proposed penalty method allows small constraint violations while the second one is

very slow near the optimal solution, so enhancing the two proposed methods will be

one area for future work.

The MSA may be used to solve another kind of optimization such as multiple

objective functions, optimal control,..., etc.

Since the MSA is a heuristic algorithm additional stastical studies are needed to fully

characterize the MSA.

137
 REFERENCES

1 D. Beasley, R. Bull and R. Martin, An Overview of Genetic Algorithms: Part 1,

Fundementals, Research Topics. University Computing, 1993, pp. 58-169.

2 A. Wright, Foundations of Genetic Algorithms, Morgan Kaufmann Publishers

Inc., 1991.

3 D. Beasley, D. Bull and R.R. Martin, An Overview of Genetic Algorithms: Part

2, Research Topics. University Computing, 1993, pp. 170-181.

4 T. Gwiazda, Crossover for Single-Objective Numerical Optimization Problems,

Genetic Algorithms Reference Vol. 1 2006.

5 Z. Janikow and Z. Michalewicz, An Experimental Comparison of Binary and

Floating Point Representations in Genetic Algorithms, in Proc. 4th Int. Conf. on
Genetic Algorithms. San Mateo, CA, Morgan Kaufmann, 1991, pp. 31-36.

6 S. Singiresu, Engineering Optimization Theory and Practice, New Age

International Limited, third edition, 2004.

7 S. Kirkpatrick, C. Gelatt and M. Vecchi, Optimization by Simulated Annealing,

Science Vol 220, 1983, pp. 671-680.

8 R. Rubinstein and D. Kroese, The Cross-Entropy Method, Springer-Verlag,

2004.

9 A. Zhigljavsky, Theory of Global Random Search, Kluwer Academic, 1991.

138
10 A. Nayeem, J. Vila and A. Harold, A Comparative Study of The Simulated-
Annealing and Monte Carlo-with-Minimization Approaches to the Minimum-
Energy Structures of Polypeptides, J. Comp. Chem. 12 (no. 5), 1991, pp. 594-
605.

11 W. Wenzel and K. Hamacher, Stochastic Tunneling Approach for Global

Optimization of Complex Potential Energy Landscapes, Phys., 1999.

12 D.Goldberg, Genetic Algorithms in Search, Optimization, and Machine

Learning, Addison-Wesley, 1989.

13 J. Rennard, Introduction to Genetic Algorithms Evolution and Optimization,

2006.

14 D. Goldberg, Real-Coded Genetic Algorithms, Virtual Alphabets and Blocking,

Complex Systems, 1992, pp. 139-167.

15 J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan

Press, 1975.

16 M. Wu, J. Horng and H. Guang, Schema Genetic Algorithm for Fractal Image
Compression, Engineering Applications of Artificial Intelligence, Vol. 20, Issue
4, June 2007, pp. 531-538.

17 L. Altenberg and The Schema Theorem and Price's Theorem. In Foundations of

Genetic Algorithms, San Francisco, pp. 23-49, 2001.

18 N. Chakraborti, Differential Evolution: The Real-Parameter Genetic Algorithm

Applied to Material and Metallurgy, iitk.ac.in, 2005, pp.71-75.

19 T. Back, Optimal Mutation Rates in Genetic Search, In: Forrest, 1993, pp.2-8.

139
20 R. Weinberg, Computer Simulation of a Living Cell (Doctoral dissertation,
University of Michigan). Dissertations Abstracts International, 31(9), 5312B.
(University Microlms No. 71-4766), 1970.

21 J. J. Collins and M. Eaton. Genocodes for Genetic Algorithms. In CTsmera, pp.

23-30.

22 T. Shigeyoshi, A. Ghosh, D. Corne and Y. Fujimoto, A Real Coded Genetic

Algorithm with an Explorer and an Exploiter Populations. Proceedings of the 7th
International Conference on Genetic Algorithms (ICGA-97), pp. 238-245.

23 T. Srikanth and V. Kamala, A Real Coded Genetic Algorithm for Optimization

of Cutting Parameters in Turning, IJCSNS International Journal of Computer
Science and Network Security, VOL.8 No.6, June 2008, pp. 189-193.

24 M. Arakawa and I. Hagiwara, Development of Adaptive Real Range (ARRange)

Genetic Algorithms, JSME International Journal, Series C, Vol. 41, No. 4,
1998, pp. 969-977.

25 A. Oyama, Sh. Obayashi and K. Nakahashi, Real-Coded Adaptive Range Genetic

Algorithm and Its Application to Aerodynamic Design, Lecture Notes in
Computer Science, Vol. 1917, Springer-Verlag, Newyork,2000, pp. 712-721.

26 M. Raghuwanshi and O. Kakde, Survey on ,Multiobjective Evolutionary and

Real Coded Genetic Algorithms, Cairns, Australia: Monash University, Dec.
2004, pp. 150-161.

27 D. Tate and A. Smith, Expected Allele Coverage and the Role of Mutation in
Genetic Algorithms. In Proceedings of the Fifth International Conference on
Genetic Algorithms, edited by S. Forrest. Morgan Kaufmann, 1993, pp.31-37.

28 R. Haupt and S. Haupt, Practical Genetic Algorithms, John Willey & Sons Inc.,
second edition, 2004.

140
29 C. Janikow and Z. Michalewicz, An Experimental Comparison of Binary and
Floating Point Representations in Genetic Algorithms, in Proc. 4th Int. Conf. on
Genetic Algorithms, San Mateo, CA, Morgan Kaufmann, 1991, pp. 31-36.

30 J. Leite, S. Avila, N. Batistela, W. Carpes, Jr. Sadowski, P. Kuo-Peng, and J.

Bastos, Real Coded Genetic Algorithm for Jiles-Atherton Model Parameters
Identification, IEEE Transactions on Magnetics, Vol. 40, No. 2, MARCH 2004,
pp. 888-891.

31 S. Eva, M. Barbara and B. Meghna, Multiscale Island Injection Genetic

Algorithm for Ground Water Remediation, World Water and Environmental
Resources Congress: Critical Transitions in Water and Environmental Resources
Management conference,_Salt Lake City, UT, United states, 2004, pp. 1930-
1937.

32 B. Meghna and M. Barbara, Multiscale Strategies for Solving Water Resources

Management Problems with Genetic Algorithms, World Water and
Environmental Resources Congress, Philadelphia, PA, United states, 2003, pp.
1029-1038.

33 K. Seung' J. Hoon and K. Young, Multiscale Multiresolution Genetic Algorithm

with a Golden Sectioned Population Composition, International Journal for
Numerical Methods in Engineering, Vol. 74, 2008, pp. 349-367.

34 D. Bertsekas, Nonlinear Programming Theory and Algorithms, John Wiley &

Sons Inc., 1979.

35 S. Singiresu, Engineering Optimization Theory and Practice, new age

international (P) limited, third edition, 2004.

36 K. S. Lee and Z. W. Geem, A New Meta-Heuristic Algorithm for Continuous

Engineering Optimization: Harmony Search Theory and Practice, Computer
Methods in Applied Mechanics and Engineering, vol.194, 2005, pp. 3902-3933.

141
37 A. E. Smith and D. W. Coit, Penalty functions, in Handbook of Evolutionary
Computation, T. Baeck, D. Foge'l, and Z. Michalewicz, Eds:A Joint Publication
of Oxford University Press and Institute of Physics, 1995.

38 O. Yeniay. Penalty Function Methods for Constrained Optimization with Genetic

Algorithms, Mathematical and Computational Applications, 2005, pp. 45-56.

39 D. Coit, A. Smith and D. Tate, Adaptive Penalty Methods for Genetic

Optimization of Constrained Combinatorial Problems, INFORMS J. Computing,
vol 8, 1996 Spr,pp 173-182.

40 W. Crossley, and E. Williams, A Study of Adaptive Penalty Functions for

Constrained Genetic Algorithm Based Optimization, In AIAA 35th Aerospace
Sciences Meeting and Exhibit, Reno, Nevada, January 1997, pp. 97-0083.

41 Z. Michalewicz, Genetic Algorithms, Numerical Optimization, and Constraints,

In J. Eshelman, editor, Proceedings of the Sixth International Conference on
Genetic Algorithms, Morgan Kaufmann (San Francisco), 1995, pp. 151-158.

42 Z. Michalewicz, A Survey of Constraint Handling Techniques in Evolutionary

Computation Methods, In Proceedings of the 4th Annual Conference on
Evolutionary Programming, MIT Press (Cambridge, MA), 1995, pp. 135-155.

43 A. Carols and C. Coello, Self Adaptive Penalties for GA-based Optimization, In

Procedings of the 1999 congress on Evolutionary computation, Washington,
D.C., July 1999, IEEE, pp. 573-580.

44 Z. Michalewicz, and G. Nazhiyath, GENOCOP III: A Co-Evolutionary

Algorithm for Numerical Optimization Problems with Nonlinear Constraints,
Proceedings of the Second IEEE International Conference on Evolutionary
Computation, IEEE Press, 1995, pp. 647-651.

142
45 D. Kalyanmoy, A. Pratap, S. Agarwal, and T. Meyarivan. A Fast and Elitist
Multi Objective Genetic Algorithm NSGA-II, IEEE Transactions on
Evolutionary Computation, 2002, pp. 182-197.

46 C. Cagnina, S. Esquivel and C. Coello, Solving Engineering Optimization

Problems with the Simple Constrained Particle Swarm Optimizer, Informatica
32, 2008, pp 319-326.

47 A. Aguirre, A. Zavala, E. Diharce and S. Rionda, COPSO: Constrained

Optimization via PSO Algorithm, Technical report No. 1-07-04/22-02-2007,
Center for Research in Mathematics (CIMAT), 2007, pp. .

48 H. Sadaat, Power System Analysis, McGraw-Hill, 2004.

49 J. Leader, Numerical Analysis and Scientific Computation, Boston, MA Pearson

Addison Wesley, 2004.

50 T. Bouktir, and L. Slimani, A Genetic Algorithm for Solving the Optimal Power
Flow Problem, Leonardo J. Sci., 2004, pp. 44-58.

51 D. Sun, B. Ashley, B. Brewer, A. Hughes, and W. Tinney, Optimal power flow

by Newton approach, IEEE Trans. Power Apparat. Syst, vol. PAS-103, 1984,
pp. 2864-2880.

52 J. Yuryevich, K. Wong, Evolutionary Programming Based Optimal Power Flow

Algorithm, IEEE Transaction on power Systems, Vol. 14, No. 4, November
1999. pp. 1245-1250.

53 L. Lai and J. Ma, Improved Genetic Algorithms for Optimal Power Flow Under
Both Normal and Contingent Operation States, Electrical Power & Energy
System, Vol. 19, No. 5, 1997, pp. 287-292.

143
54 W. Liu, M. Li and X.Wang, An Improved Particle Swarm Optimization
Algorithm for Optimal Power Flow, Power Electronics and Motion Control
Conference IPEMC, 2009, pp.2448-2450.

55 I. Oumarou, D. Jiang and C. Yijia, Particle Swarm Optimization Applied to

Optimal Power Flow Solution, Proceedings of 2009 Fifth Conference on Natural
Computation, vol. 3, 2009, pp. 284- 288.

56 P. Vu, D. Le, N. Vo and J. Tlusty, A Novel Weight-Improved Particle Swarm

Optimization Algorithm for Optimal Power Flow and Economic Load Dispatch
Problems, IEEE Transmission and Distribution Conference and Exposition,
2010, pp. 1-7.

57 S. Sayah and K. Zehar, Modified Differential Evolution Algorithm for Optimal

Power Flow with Non-Smooth Cost Function, Energy conversion and
Management, 2009, pp. 3036-3042.

58 H. Hoseinabadi, S. Hosseini and M. Hajian, Optimal Power Flow Solution by a

Modified Particle Swarm Optimization Algorithm, Universities Power
Engineering Conference, UPEC, 43rd International, 2008, pp. 1-4.

59 A. Basak, S. Pal, V. Pandi, B. Panigrahi and S. Das, A Hybrid Differential

Invasive Weed Algorithm for Congestion Management, 1 st Swarm, Evolutionary
and Memetic Computing Conference, SEMCCO 2010, pp. 630-638.

144
 Appendix A

Specifications of the IEEE-26-Bus Power System Network [48]

Table A-1 The Generation buses data of IEEE-26-bus [48]

Generation buses data

Bus Voltage P P Cost coefficents
2>min i^min
number magnitude mm max
[MW] [MW] [Mvar] [Mvar] a. b, c,
1 1.025 100 500 40 250 0.0070 7.0 240
2 1.020 50 200 40 250 0.0095 10.0 200
3 1.025 80 300 40 150 0.0090 8.5 220
4 1.050 50 150 40 80 0.0090 11.0 200
5 1.045 50 200 40 160 0.0080 10.5 220
26 1.015 50 120 15 50 0.0075 12.0 190

Table A-2 Load data of IEEE-26-bus [48]

Load data
Bus number Active load [MW] Reactive load [Mvar]
1 51 41
2 22 15
3 64 50
4 25 10
5 50 30
6 76 29
7 0 0
8 0 0
9 89 50
10 0 0
11 25 15
12 89 48
13 31 15
14 24 12
15 70 31
16 55 27
17 78 38
18 153 67
19 75 15
20 48 27
21 46 23
22 45 22
23 25 12
24 54 27
25 28 13
26 40 20

145
Table A-3 Transmission lines data of IEEE-26-bus [48]
Transmission line data
Bus Bus Resistance Reactance Susbitance Tap setting
Number Number rpui fpul [pul value
1 2 0.00055 0.00480 0.0300 1
1 18 0.00130 0.01150 0.0600 1
2 3 0.00146 0.05130 0.0500 0.96
2 7 0.01030 0.05860 0.01800 1
2 8 0.00740 0.03210 0.03900 1
2 13 0.00357 0.09670 0.02500 0.96
2 26 0.03230 0.19670 0.00000 1
3 13 0.00070 0.00548 0.00050 1.017
4 8 0.00080 0.02400 0.00010 1.050
4 12 0.00160 0.02070 0.01500 1.050
5 6 0.00690 0.03000 0.09900 1
6 7 0.00535 0.03060 0.00105 1
6 11 0.00970 0.05700 0.00010 1
6 18 0.00374 0.02220 0.00120 1
6 19 0.00350 0.06600 0.04500 0.95
6 21 0.00500 0.09000 0.02260 1
7 8 0.00120 0.00693 0.00010 1
7 9 0.00095 0.04290 0.02500 0.95
8 12 0.00200 0.01800 0.02000 1
9 10 0.00104 0.04930 0.00100 1
10 12 0.00247 0.01320 0.01000 1
10 19 0.05470 0.23600 0.00000 1
10 20 0.00660 0.01600 0.00100 1
10 22 0.00690 0.02980 0.00500 1
11 25 0.09600 0.27000 0.01000 1
11 26 0.01650 0.09700 0.00400 1
12 14 0.03270 0.08020 0.00000 1
12 15 0.01800 0.05980 0.00000 1
13 14 0.00460 0.02710 0.00100 1
13 15 0.01160 0.06100 0.00000 1
13 16 0.01793 0.08880 0.00100 1
14 15 0.00690 0.03820 0.00000 1
15 16 0.02090 0.05120 0.00000 1
16 17 0.09900 0.06000 0.00000 1
16 20 0.02390 0.05850 0.00000 1
17 18 0.00320 0.06000 0.03800 1
17 21 0.22900 0.44500 0.00000 1
19 23 0.03000 0.13100 0.00000 1
19 24 0.03000 0.12500 0.00200 1
19 25 0.11900 0.22490 0.00400 1
20 21 0.06570 0.15700 0.00000 1
20 22 0.01500 0.03660 0.00000 1
21 24 0.04760 0.15100 0.00000 1
22 23 0.02900 0.09900 0.00000 1
22 24 0.03100 0.08800 0.00000 1
23 25 0.09870 0.11680 0.00000 1

146
Table A-4 Shunt capacitance data of IEEE-26-bus [48]

Shunt capacitance data

Bus number Reactive power [Mvar]

1 4.0
4 2.0
5 5.0
6 2.0
9 3.0
11 1.5
12 2.0
15 0.5
19 5.0

147
 Appendix B

Specifications of the IEEE-30-Bus Power System Network [50]

Table B-1 Load data on buses of IEEE-30-bus [48]

oad data
Bus number Active load [MW] Reactive load [Mvar]
1 0 0
2 21.7000 12.7000
3 2.4000 1.2000
4 7.6000 1.6000
5 94.2000 19.0000
6 0 0
7 22.8000 10.9000
8 30.0000 30.0000
9 0 0
10 5.8000 2.0000
11 0 0
12 11.2000 7.5000
13 0 0
14 6.2000 1.6000
15 8.2000 2.5000
16 3.5000 1.8000
17 9.0000 5.8000
18 3.2000 0.9000
19 9.5000 3.4000
20 2.2000 0.7000
21 17.5000 11.2000
22 0 0
23 3.2000 1.6000
24 8.7000 6.7000
25 0 0
26 3.5000 2.3000
27 0 0
28 0 0
29 2.4000 0.9000
30 10.6000 1.9000

148
Table B-2 Transmission lines data of IEEE-30-bus [48]

Transmission lines data

Bus Bus Resistance Reactance Susbitance Tap setting
Number Number rpui [pul rpui value
1 2 0.0192 0.0575 0.0264 1.0000
1 3 0.0452 0.1852 0.0204 1.0000
2 4 0.0570 0.1737 0.0184 1.0000
3 4 0.0132 0.0379 0.0042 1.0000
2 5 0.0472 0.1983 0.0209 1.0000
2 6 0.0581 0.1763 0.0187 1.0000
4 6 0.0119 0.0414 0.0045 1.0000
5 7 0.0460 0.1160 0.0102 1.0000
6 7 0.0267 0.0820 0.0085 1.0000
6 8 0.0120 0.0420 0.0045 1.0000
6 9 0 0.2080 0 1.0155
6 10 0 0.5560 0 0.9629
9 11 0 0.2080 0 1.0000
9 10 0 0.1100 0 1.0000
4 12 0 0.2560 0 1.0129
12 13 0 0.1400 0 1.0000
12 14 0.1231 0.2559 0 1.0000
12 15 0.0662 0.1304 0 1.0000
12 16 0.0945 0.1987 0 1.0000
14 15 0.2210 0.1997 0 1.0000
16 17 0.0824 0.1932 0 1.0000
15 18 0.1070 0.2185 0 1.0000
18 19 0.0639 0.1292 0 1.0000
19 20 0.0340 0.0680 0 1.0000
10 20 0.0936 0.2090 0 1.0000
10 17 0.0324 0.0845 0 1.0000
10 21 0.0348 0.0749 0 1.0000
10 22 0.0727 0.1499 0 1.0000
21 22 0.0116 0.0236 0 1.0000
15 23 0.1000 0.2020 0 1.0000
22 24 0.1150 0.1790 0 1.0000
23 24 0.1320 0.2700 0 1.0000
24 25 0.1885 0.3292 0 1.0000
25 26 0.2544 0.3800 0 1.0000
25 27 0.1093 0.2087 0 1.0000
27 28 0 0.3960 0 0.9581
27 29 0.2198 0.4153 0 1.0000
27 30 0.3202 0.6027 0 1.0000
29 30 0.2399 0.4533 0 1.0000
8 28 0.0636 0.2000 0.0214 1.0000
6 28 0.0169 0.0599 0.0065 1.0000

149
Table B-3 Generation buses data of IEEE-30-bus [50]

Bus Voltage P P V V
c-min zimvn min max
[pu] min
[MW]
max
[Mvar] [Mvar] min max
[MW] [pu] [pu] [deg] [deg]
1 1.0600 50 200 -20 200 0.95 1.10 -14 0
2 1.0450 20 80 -20 100 0.95 1.10 -14 0
5 1.0100 15 50 -15 80 0.95 1.10 -14 0
8 1.0250 10 35 -15 60 0.95 1.10 -14 0
11 1.0820 10 30 -10 50 0.95 1.10 -14 0
13 1.0710 12 40 -15 60 0.95 1.10 -14 0

Table B-4 Generation cost coefficients of IEEE-30-bus [50]

Generation cost coefficients

Bus a, b, c.
1 0.0375 200 0
2 0.1750 175 0
5 0.1000 100 0
8 0.3250 325 0
11 0.3000 300 0
13 0.3000 300 0

150
 Appendix C

Pseudo-Code for the Optimal Power Flow of the IEEE-30-Bus Power System
Network Using MSA with the Proposed Penalty Method

r='00.000000' % Starting value of the radius.

thl = '0.0000000' % Starting value of the angle theta.

th2 = '0.0000000' % Starting value of the angle theta.

th3 = '0.0000000' % Starting value of the angle theta.

th4= '0.0000000' % Starting value of the angle theta.

th5 = '0.0000000' % Starting value of the angle theta.

MWLimits=[50 200 % the power limits matrix

20 80
15 50
10 35
10 30
12 40]

cost=[0 200 0.375 % fuel cost of the generation units

0 175 1.75
0 100 6.25
0 325 0.83
0 300 2.5
0 300 2.5]

% start from random values of the generation power within their limits
Pgli =Mwlimits(l,l) +random(l,l)*(MWlimits(l,2)-MWlimits(l,l))

Pg2 i=Mwlimits(2,l) +random(l,l)*(MWlimits(2,2)-MWlimits(2,l))

Pg5i =Mwlimits(3,l) +random(3,l)*(MWlimits(3,2)-MWlimits(3,l))

Pg8i =Mwlimits(4,l) +random(4,l)*(MWlimits(4,2)-MWlimits(4,l))

Pgl li =Mwlimits(5,l) +random(5,l)*(MWlimits(5,2)-MWlimits(5,l))

Pgl3i =Mwlimits(6,l) +random(l,l)*(MWlimits(6,2)-MWlimits(6,l))

Inequality violation= 100 % initial value of the inequality constraints

151
Power mismatch=100 % mismatch power of the slack bus between the

% Power flow problem and the optimal power flow

plosses =0; % initial value of the total losses in the transmission lines

iter=l % iteration counter connects the power flow with optimal power flow

itermax = 10 % maximum iteration of calling the power flow function

While ( power mismatch > le-6 & inequality violation > le-6 & iter <itermax)

% initial value of the penalty parameter

cl=l

c2=l

pdemand=283.4 % the demand power

% The Generation cost function is created to calculate the fuel costs of the

% generation plants

f =Generation cost (pgli,pg2i,pg5i,pg8i,pgl li,pgl3i)

h= pgli+pg2i+pg5i+pg8i+pgl li+pgl3i-pdemand-plosses % The equality

% constraint

fcon=cl*f+c2*hA2

err= 100 % initial value of the error

k=l % initial value of the iteration number

kmax=100 % maximum number of iteration

tolx=le-6; tolf=le-6; % desired tolerances

152
 while ( err > 0 & k < kmax)

rtr=mutation(r,k) % mutate the radius r

% theta is uniformly distributed from [0 2 ;r ]

thltr=2*pi*rand(l,l)

th2tr=2*pi*rand(l,l)

th3tr=2*pi*rand(l,l)

th4tr=2 *pi*rand(l,l)

th5tr=2*pi*rand(l,l)

% calculate the next strin gs based on the new mutated radius rtr and new angles

pgl=pgli+rtr*cos(thltr)*cos(th2tr)...

*cos(th3tr)*cos(th4tr)*cos(th5tr)

pg2=pg2i+rtr*cos(thltr)*cos(th2tr)...

* cos(th3 tr) *cos(th4tr) * sin(th5 tr)

pg5=pg5i+rtr*cos(thltr)*cos(th2tr)*cos(th3tr)*sin(th4tr)

pg8=pg8i+rtr*cos(thltr)*cos(th2tr) *sin(th3tr)

pgl l=pgl li+rtr*cos(thltr)*sin(th2tr)

pgl3=pgl3i+rtr*sin(thltr)

% keep the real power within their limits

pg 1 =min(M Wlimits( 1,2),max(pg 1 ,M Wlimits( 1,1))

pg2=min(MWlimits(2,2),max(pg2,MWlimits(2,l))

pg5=min(MWlimits(3,2),max(pg5,MWlimits(3,l))

pg8=min(MWlimits(4,2),max(pg8,MWlimits(4,1))

153
pgl l=min(MWlimits(5,2),max(pgl l,MWlimits(5,l))

pg 13=min(MWlimits(6,2),max(pg 13 ,M Wlimits(6,1))

fl=Generation cost (pgl,pg2,pg5,pg8,pgl l,pgl3)

h 1 =pg 1 +pg2+pg5+pg8+pg 11 +pgl 3-pdemand-plosses

fconl=cl*fl+c2*hl A 2

If ( fconl < fcon) % is the next string better than the previous one

r=rtr;

thl=thltr

th2=th2tr

th3=th3tr

th4=th4tr

th5=th5tr

lfabsolute(hl)>le-6

violrel=abs(hl-h)/h;

frel=(fl-f)/f

If frel >absolute(hl)

c2=c2+frel/violrel

Else

cl=cl+violrel/frel

Endif

Endif

154
 f=fl

fcon=fconl

h=hl

End if

k=k+l

End

% Newton raphson method is a function created to calculate the power flow equations.

[plosses , constraint violations,pglm]=Newton raphsom I,pg2,pg5,pg8,pgl lmpgl3)

Iter=iter+1

Power mismatch = absolute(pgl-pglm)

Pgl=pglm

End

% the Generation cost is a function created to calculate the total generation cost of the

% plgeneration plants the inputs of this function are the generation real power pgl, pg2, pg5,
% pg8, pgll and pgl3

F= Generation cost (pg2,pg2,pg3,pg8,pgl l,pgl33)

Pm=[pgl,pg2, pg8, pg8, pgl 1, pgl3] % pm is a vector contains the generation power

cost =[0 200 0.375 % fuel cost of the generation units

0 175 1.75
0 100 6.25
0 325 0.83
0 300 2.5
0 300 2.5]
F=0
For 1=1 To 6

F=F+ cost(1,1 )+cost(I,2)*Pm(I)+cos(I,3)*Pm(I)A2

End

155