Chapter 2

OPTIMAL CONTROL

PROBLEMS

2.1 Linear control system

In this chapter solutions for linear optimal control problems are derived by solving
matrix Riccati differential equations using grammar based GP and nonlinear optimal
control problems using standard GP. The obtained solutions have been compared with
solution by traditional Runge-Kutta (RK) method for computational time and conver­
gence of results.

Optimal control theory offers modern methods regarding the control of systems and

thus plays an important role in linear control theory. The use of optimal control in
the class of linear systems permits a substantial diminution of computation of the
optimal control law. As the theory of optimal control for linear systems with quadratic
performance criteria is well developed, the results are most complete and close to use
in many practical designing problems. In this section, optimal control for linear control
system is obtained by solving MRDE using GP.

51
A variety of numerical algorithms have been discussed for solving the MRDE (see [23]).
Solving the MRDE is a central issue in optimal control theory. The needs for solving
such equations often arise in analysis and synthesis such as linear quadratic optimal
control systems, robust control systems with H 2 and Hoo control [96] performance cri­
teria, stochastic filtering and control systems, model reduction, differential games etc.,

and many others. In particular, the solution of MRDE in continuous time (or) discrete
time is an important task for nonlinear matrix equations arising in mathematics and

engineering. This equation, in one form or another, has an important role in optimal

control problems, multivariate and large scale systems, scattering theory estimation, -
detection, transportation and radiative transfer [45]. The solution of this equation is
difficult to obtain from two points of view. One is nonlinear and the other is matrix
form. Most general methods to solve MRDE with a terminal boundary condition are

obtained on transforming MRDE into an equivalent linear differential Hamiltonian sys­

tem [9, 46]. By using this approach, the solution of MRDE is obtained by partitioning
the transition matrix of the associated Hamiltonian system (see [75, 88]). Another class

of method is based on transforming MRDE into a linear matrix differential equation

and then solving MRDE analytically (or) computationally (see [58, 76, 77]). In [58],
an analytic procedure of solving the MRDE of the linear quadratic control problem for
homing missile systems is presented. The solution K (t) of the MRDE is obtained by
using K(t) = p(t)/f{t) , where p(t) and f(t) are solutions of certain first, order ordi­
nary linear differential equations. However the given technique is restricted to single
input.

GP uses the principles and ideas from biological evolution to guide the computer to

acquire desired solution. The generation is achieved with the help of GE, because

GE can produce programs in an arbitrary language. Also the genetic operations are
faster and it is more convenient to symbolically differentiate mathematical expression.
The code production is performed using a mapping process governed by grammar
expressed in BNF [69]. In analogy to nature, the potential solution as an individual in

52
special functions, which cannot be computed easily. In such cases too numerical inte­
gration can be used. RK method have always been considered as the best tool for the
numerical integration of ODEs.

2.1.2.2 Genetic Programming approach

In this approach, the following steps are carried out to find the desired solution of (2.8).

• First initialize genetic random population and commutate the valid function of
each chromosome of the population using GP.

• Second construct the fitness function.

• Third create new generation of chromosome with the use of genetic operations.

The process is repeated until the value of the fitness function tends to zero.

The schematical overview of the GP algorithm is given in Figure 2.1.

Initialization of the population

The first step is to initialize the population. The initial population of the desired size
is generated randomly by using GE. The length of each chromosome is to be set.

Grammatical evolution

GE is an EA that can produce code in any programming language. The algorithms

start from the start symbol of the grammar and gradually create the program string, by
replacing non terminal symbol with right hand of the selected production rule. Then
read an element from the chromosome (with value V) and compute

rule = V ( M O D ) N R (2.10)

57
2.2 Nonlinear control system

In this section the solution for optimal control problem has been for nonlinear system
using standard genetic programming (SGP). In theoretical point of view, the control­
lability and observability problems for nonlinear systems are considered by the authors
[93, 94], The theory of the quadratic cost control problem has been treated as a more

interesting problem. The optimal feedback with minimum cost control has been char­

acterized by the solution of a Riccati equation. Da Prato and Ichikawa [30] showed that

the optimal feedback control and the minimum cost are characterized by the solution
of a Riccati equation.

Optimal control of nonlinear systems is one of the most active subjects in control theory.
One of the main, difficulties with classic optimal control theory is that, to determine op­

timal control for a nonlinear system, the Hamilton-Jacobi-Bellman (HJB) PDEs have
to be solved. There are few papers on the optimal regulator problem for nonlinear sys­
tems. For example, Brunovsky [10] approached the problem via Lyapunov functions

under the assumptions of complete controllability. Lukes [60] requires that the system
be stabilizable and then uses Lyapunov like theory to obtain results for feedback con­
trollers. But the results in these paper are not applicable to real physical simulations,
hence approximations into each realizable forms are needed; that is why sub optimal
regulator problems have been studied by many authors [16, 57]. There is rarely an
analytical solution although several numerical computation approaches have been pro­
posed (see [74]). The approximating sequence of Riccati equations feedback algorithm
for nonlinear optimal control provides outstanding performance in many practical ap­

plications, in particular, nonlinear solitary wave motion [6] and optimal maneuvering

of super-tankers at high speeds [24], These makes up for seeking an efficient method
for solving the nonlinear optimal control problems. However, many existing numerical
algorithms for solving MRDE for nonlinear optimal control optimality have not been
proved.

69
GP introduced by Koza, and his group [49] is popular for its ability to learn relation­
ships hidden in data and express them automatically in a mathematical manner. GP

evolves a population of computer programs, which are possible solutions to a given

optimization problem, using the Darwinian principle of survival of the fittest. GA [38]
usually operates on (coded) strings of numbers GP uses the principles and ideas from
biological evolution to guide the computer to acquire desired solution. It uses biologi­
cally inspired operations like reproduction, crossover and mutation. Each program or

individual in the population is generally represented as a tree composed of function

and data/terminals appropriate to the problem domain. To ease the process of creat­

ing new programs from two parent programs, the programs are written as trees. New

programs are produced by removing branches from one tree and inserting them into
another. Each new generation is produced using the selected individuals from previous
generation and their structures are modified by employing analogs of genetic opera­
tion. In analogy to nature, the potential solution as an individual in some collection
or population of potential solutions. The individuals who are stronger, meaning higher
ranked according to fitness function, will be used to determine the next collection of

potential solution. Each individual encodes a candidate solution and is associated with
a fitness value that measures the goodness of fit of that solution. The capability of GP
in solving challenging problems, coming from different application domains, has been

largely recognized. Many problems have been solved by means of GP. For difficult
problems, however, in order to find a good solution, GP may require large number of
generations by using a population of sufficient size. The choice of population size is
determined by the complexity of the problem.