1

Introduction

Classical control system design is generally a trial-and-error process in

which various methods of analysis are used iteratively to determine the design
parameters of an "acceptable" system. Acceptable performance is generally
defined in terms of time and frequency domain criteria such as rise time,
settling time, peak overshoot, gain and phase margin, and bandwidth. Radi-
cally different performance criteria must be satisfied, however, by the com-
plex, multiple-input, mUltiple-output systems required to meet the demands
of modern technology. For example, the design of a spacecraft attitude
control system that minimizes fuel expenditure is not amenable to solution
by classical methods. A new and direct approach to the synthesis of these
complex systems, called optimal control theory, has been made feasible by
the development of the digital computer.
The objective of optimal control theory is to determine the control signals
that will cause a process to satisfy the physical constraints and at the same
time minimize (or maximize) some performance criterion. Later, we shall
give a more explicit mathematical statement of "the optimal control prob-
lem," but first let us consider the matter of problem formulation.

1.1 PROBLEM FORMULATION

The axiom "A problem well put is a problem half solved" may be a slight
exaggeration, but its intent is nonetheless appropriate. In this section, we
3
4 Describing the System and Evaluating Its Performance Sec. 1.1

shall review the important aspects of problem formulation, and introduce

the notation and nomenclature to be used in the following chapters.
The formulation of an optimal control problem requires:
1. A mathematical description (or model) of the process to be controlled.
2. A statement of the physical constraints.
3. Specification of a performance criterion.

The Mathematical Model

A nontrivial part of any control problem is modeling the process. The

objective is to obtain the simplest mathematical description that adequately
predicts the response of the physical system to all anticipated inputs. Our
discussion will be restricted to systems described by ordinary differential
equations (in state variable form). t Thus, if

XI (t), xlt), ... , xn(t)

are the state variables (or simply the states) of the process at time t, and

are control inputs to the process at time t, then the system may be described
by n first-order differential equations

XI (t) = a I (x I (t), xit), ... , xit), U I (t), u 2 (t), ... , um(t), t)

x (t)
2 = a 2 (x I (t), x 2 (t), ... , x.(t), ul(t), uz(t), ... , um(t), t)
(1.1-1)

Xn(t) = an(xl(t), xz(t), ., ., xn(t), ul(t), u 2 (t), ... , um(t), t).t

We shall define

x(t) ~

xn(t)

as the state vector of the system, and

t The reader will find the concepts much the same for discrete systems (see [A-I]).
t Note that Xi(t) is in general a nonlinear time-varying function al of the states, the
control inputs, and time.
Sec. 1.1 Introduction 5

nCt) ~

U.,Ct)

as the control vector. The state equations can then be written

xCt) = a(xCt), oCt), t), (1.1-1a)

where the definition of a is apparent by comparison with (1.1-1).

Car

~~ o e
• d

Figure 1·1 A simplified control problem

Example 1.1-1. The car shown parked in Fig. 1-1 is to be driven in a

straight line away from point O. The distance of the car from 0 at time
t is denoted by d(t). To simplify the model, let us approximate the car
by a unit point mass that can be accelerated by using the throttle or
decelerated by using the brake. The differential equation is

d(t) = lX(t) + P(t), (1.1-2)

where the control IX is throttle acceleration and-p is braking deceleration.

Selecting position and velocity as state variables, that is,

and letting

we find that the state equations become

Xl (t) = X2(t)
(1.1-3)
X2(t) = Ul(t) + U2(t),
or, using matrix notation,

i(t) = [~ ~J x(t) + [~ ~J u(t). (1.1-3a)

This is the mathematical model of the process in state form.

6 Describing the System and Evaluating Its Performance Sec. 1.1

Before we move on to the matter of physical constraints, let us consider

two definitions that will be useful later. Let the system be described by Eq.
(l.1-1a) for t E [to, tf].t

DEFINITION 1-1
A history of control input values during the interval [to, tf ] is de-
noted by u and is called a control history, or simply a control.

DEFINITION 1-2
A history of state values in the interval [to, tf ] is called a state tra-
jectory and is denoted by x.

The terms "history," "curve," "function," and "trajectory" will be used

interchangeably. It is most important to keep in mind the difference between
a/unction and the value 0/ a/unction. Figure 1-2 shows a single-valued func-
tion of time which is denoted by x. The value of the function at time t I is
denoted by x(t l ).
x(t)

x
•

---r--~--------------~---------L------~Time
to II If

Figure 1-2 A function, x, and its value at time tl, X(/l)

Physical Constraints

After we have selected a mathematical model, the next step is to define

the physical constraints on the state and control values. To illustrate some
typical constraints, let us return to the automobile whose model was deter-
mined in Example 1.1-1.

Example 1.1-2. Consider the problem of driving the car in Fig. 1-1
between the points 0 and e. Assume that the car starts from rest and
stops upon reaching point e. Translate this to math?

t This notation means to:::::; I:::::; If.

Sec. 1.1 Introduction 7

First let us define the state constraints. If 10 is the time of leaving 0,

and II is the time of arrival at e, then, clearly,

XI(/ o) = 0
(1.1-4)
XI(II) = e.

In addition, since the automobile starts from rest and slops at e,

X2(tO) =0
(1.1-5)
X2(11) = o.
In matrix notation these boundary condi/ions are

(1.1-6)

If we assume that the car does not back up, then the additional constraints

O:S;:XI(/):S;:e
(1.1-7)
o:s;: X2(/)
are also imposed.
What are the constraints on the control inputs (acceleration)? We
know that the acceleration is bounded by some upper limit which depends
on the capability of the engine, and that the maximum deceleration is
limited by the braking system parameters. If the maximum acceleration
is MI > 0, and the maximum deceleration is M2 > 0, then the controls
must satisfy
o :s;: u I (I) :s;: MI
(1.1-8)
-M2 :s;: U2(/) :s;: o.
In addition, if the car starts with G gallons of gas and there are no service
stations on the way, another constraint is

(1.1-9)

which assumes that the rate of gas consumption is proportional to both

acceleration and speed with constants of proportionality kl and k 2 •

Now that we have an idea of typical constraints that may be encountered,

let us make these concepts more precise.

DEFINITION 1-3
A control history which satisfies the control constraints during the
entire time interval [to, t/] is called an admissible control.
8 Describing the System and Evaluating Its Performance Sec. 1.1

We shall denote the set of admissible controls by U, and the notation U E U

means that the control history u is admissible.
To illustrate the concept of admissibility Fig. 1-3 shows four possible
acceleration histories for Example 1.1-2. U\2) and U\4) are not admissible;

(a)

(b)

• t
(c)

• t
(d)

Figure 1-3 Some acceleration histories

u\1l and uP) are admissible if they satisfy the consumed-fuel constraint of
Eq. (1.1-9). In this example, the set of admissible controls U is defined by the
inequalities in (1.1-8) and (1.1-9).

DEFINITION 1-4

A state trajectory which satisfies the state variable constraints

during the entire time interval [to, tfl is called an admissible tra-
jectory.
Sec. 1.1 Introduction 9

The set of admissible state trajectories will be denoted by X, and x E X

means that the trajectory x is admissible.
In Example 1.1-2 the set of admissible state trajectories X is specified by
the conditions given in Eqs. (Ll-6), (Ll-7), and (1.1-9). In general, the final
state of a system will be required to lie in a specified region S of the (n + 1)-
dimensional state-time space. We shall call S the target set. If the final state
and the final time are fixed, then S is a point. In the automobile problem of
Example Ll-2 the target set was the line shown in Fig. 1-4(a). If the auto-
mobile had been required to arrive within three feet of e with zero terminal
velocity, the target set would have been as shown in Fig. 1-4(b).
Admissibility is an important concept, because it reduces the range of
values that can be assumed by the states and controls. Rather than consider
all control histories and their trajectories to see which are best (according to
some criterion), we investigate only those trajectories and controls that are
admissible.

- ------ - ------~~-------

(a)

(b)

Figure 1-4 (a) The target set for Example 1.1-2. (b) The target set
defined by IXI(t) - el < 3, X2(t) = 0
10 Describing the System and Evaluating Its Performance Sec. 1.1

The Performance Measure

In order to evaluate the performance of a system quantitatively, the

designer selects a performance measure. An 0Plimal control is defined as one
that minimizes (or maximizes) the performance measure. In certain cases
the problem statement may clearly indicate what to select for a performance
measure, whereas in other problems the selection is a subjective matter.
For example, the statement, "Transfer the system from point A to point B
as quickly as possible," clearly indicates that elapsed time is the performance
measure to be minimized. On the other hand, the statement, "Maintain the
position and velocity of the system near zero with a small expenditure of
control energy," does not instantly suggest a unique performance measure.
In such problems the designer may be required to try several performan~e
measures before selecting one which yields what he considers to be optimal
performance. We shall discuss the selection of a performance measure in
more detail in Chapter 2.

Example 1.1-3. Let us return to the automobile problem begun in

Example 1.1-1. The state equations and physical constraints have been
defined; now we turn to the selection of a performance measure. Suppose
the objective is to make the car reach point e as quickly as possible;
then the performance measure J is given by

J = tf - to. (1.1-10)

In all that follows it will be assumed that the performance of a system is

evaluated by a measure of the form

J = h(x(t,), I,) + St' g(X(/), U(/), I) dl,

t,
(1.1-11)

where 10 and I, are the initial and final time; hand g are scalar functions.
I, may be specified or "free," depending on the problem statement.
Starting from the initial state X(/o) = Xo and applying a control signal
U(/), for I E [to, I,], causes a system to follow some state trajectory; the
performance measure assigns a unique real number to each trajectory of the
system.
With the background material we have accumulated it is now possible
to present an explicit statement of "the optimal control problem."

The Optimal Control Problem

The theory developed in the subsequent chapters IS aimed at solving

the following problem.
Sec. 1.1 Introduction 11

Find an admissible control u* which causes the system

x(t) = a(x(t), u(t), t) (1.1-12)

to follow an admissible trajectory x* that minimizes the performance meas-

ure

J = h(x(tI)' t I) +f t
to' g(x(t), u(t), t) dt. (1.1-l3)

u* is called an optimal control and x* an optimal trajectory.

Several comments are in order here. First, we may not know in advance
that an optimal control exists; that is, it may be impossible to find a control
which (a) is admissible and (b) causes the system to follow an admissible
trajectory. Since existence theorems are in rather short supply, we shall,
in most cases, attempt to find an optimal control rather than try to prove
that one exists.
Second, even if an optimal control exists, it may not be unique. Nonunique
optimal controls may complicate computational procedures, but they do
allow the possibility of choosing among several controller configurations.
This is certainly helpful to the designer, because he can then consider other
factors, such as cost, size, reliability, etc., which may not have been included
in the performance measure.
Third, when we say that u* causes the performance measure to be mini-
mized, we mean that

J* ~ h(x*(tl)' tl) + ft' g(x*(t), u*(t), t) dt

I,
(1.l-14)
< h(x(tl)' t I) + s:: g(x(t), u(t), 't) dt
for all u E U, which make x E X. The above inequality states that an
optimal control and its trajectory cause the performance measure to have a
value smaller than (or perhaps equal to) the performance measure for any
other admissible control and trajectory. Thus, we are seeking the absolute
or global minimum of J, not merely local minima. Of course, one way to find
the global minimum is to determine all of the local minima and then simply
pick out one (or more) that yields the smallest value for the performance
measure.
It may be helpful to visualize the optimization as shown in Fig. 1-5.
U CIl , U(2), u(J), and U (4 ) are "points" at which J has local, or relative, minima;
U CIl is the "point" where J has its global, or absolute, minimum.
Finally, observe that if the objective is to maximize some measure of
system performance, the theory we shall develop still applies because this
12 Describing the System and Evaluating Its Performance Sec. 1.1

t-+---Admissible control region---+I

J. ---

-+-~~~---~-----~~-~~-~
u(1) = u· u(2) u(3) u(4)
u

Figure 1-5 A representation of the optimization problem

is the same as minimizing the negative of this performance measure. Hence-

forth, we shall speak, with no lack of generality, of minimizing the perfor-
mance measure.

Example 1.1-4. To illustrate a complete problem formulation, let us now

summarize the results of Example 1.1-1, using the notation and definitions
which have been developed.
The state equations are

x\(t) = Xl(t)
(1.1-3)
Xl(t) = u\(t) + Ul(t).
The set of admissible states X is partially specified by the boundary condi-
tions

x(t o) = 0,

and the inequalities

0:::;; x\(t) :::;; e

(1.1-7)
0:::;; Xl(t).

The set of admissible controls U is partially defined by ·the constraints

0:::;; u\(t):::;; M\
(1.1-8)
-Ml :::;; U2(t) :::;; O.
Sec. 1.1 Introduction

The inequality constraint

(1.1-9)

completes the description of the admissible states and controls.

The solution to this problem (which is left as an exercise for the reader
at the end of Chapter 5) is shown in Fig. 1-6 for the situation where MI =
Mz ~ M. We have also assumed that the car has enough fuel available to
reach point e using the control shown.

a*(t)

t.
(j*(t)
i (to +11)

! (to +11)
I
tl

I~
• 1

• 1

-M I'" I

• 1

xj(t)

e
t

Figure 1-6 The optimal control and trajectory for the automobile
problem

Example 1.1-5. Let us now consider what would happen if the preceding
problem had been improperly formulated. Suppose that the control
constraints had not been recognized. If we let

(1.1-15)
14 Describing the System and Evaluating Its Performance Sec. 1.1

where oCt - to) is a unit impulse function that occurs at time to,t then

X2(t) = e oCt - to) (1.1-16)

and
(1.1-17)

[n (t - to) represents a unit step function at t = tol. Figure 1-7 shows

the state trajectory which results from applying the "optimal" control
in (l.1-15). Unfortunately, although the desired transfer from point 0

x;(t)

(e)

to

Figure 1-7 The optimal trajectory resulting from unconstrained

controls

to point e is accomplished in infinitesimal time, the control required,

apart from being rather unsafe, is physically impossible! Thus, we see
the importance of correctly formulating problems before attempting
their solution .

. Form of the Optimal Control

DEFINITION 1-5
If a functional relationship of the form

u*(t) = f(x(t), m (1.1-18)

t See reference [Z-lJ.
:1: Here we write x(t) instead of x*(t) to emphasize that the control law is optimal for all
admissible x(/), not just for some special state value at time t.
Sec. 1.1 Introduction 1&

can be found for the optimal control at time t, then the function f
is called the optimal control law, or the optimal policy.t

Notice that Eq. (1.1-18) implies that f is a rule which determines the
optimal control at time t for any (admissible) state value at time t. For
example, if
u*(t) = Fx(t), (1.1-19)

where F is an m X n matrix of real constants, then we would say that the

optimal control law is linear, time-invariant feedback of the states.

DEFINITION 1-6
If the optimal control is determined as a function of time for a speci-
fied initial state value, that is,
u*(t) = e(x(to), t), (1.1-20)

then the optimal control is said to be in open-loop form.

Thus the optimal open-loop control is optimal only for a particular initial
state value, whereas, if the optimal control law is known, the optimal con-
trol history starting from any state value can be generated.
Conceptually, it is helpful to imagine the difference between an optimal
control law and an open-loop optimal control as shown in Fig. 1-8; notice,

Opens at to

(b)

Figure 1-8 (a) Open-loop optimal control. (b) Optimal control law

however, that the mere presence of connections from the states to a con-
troller does not, in general, guarantee an optimal controllaw.t

t The terms optimal feedback control, closed-loop optimal control, and optimal control
strategy are also often used.
t This is pursued further in reference [K-IJ.
16 Describing the System and Evaluating Its Performance Sec. 1.2

Although engineers normally prefer closed-loop solutions to optimal

control problems, there are cases when an open-loop control may be feasible.
For example, in the radar tracking of a satellite, once the orbit is set very
little can happen to cause an undesired change in the trajectory parameters.
In this situation a pre-programmed control for the radar antenna might well
be used.
A typical example of feedback control is in the classic servomechanism
problem where the actual and desired outputs are compared and any devia-
tion produces a control signal that attempts to reduce the discrepancy to
zero.

1.2 STATE VARIABLE REPRESENTATION OF

SYSTEMS

The starting point for optimal control investigations is a mathematical

model in state variable form. In this section we shall summarize the results
and notation to be used in the subsequent discllssion. There are several
excellent texts available for the reader who needs additional background
materia1.t

Why Use State Variables?

Having the mathematical model in state variable form is convenient

because
1. The differential equations are ideally suited for digital or analog
solution.
2. The state form provides a unified framework for the study of non-
linear and linear systems.
3. The state variable form is invaluable in theoretical investigations.
4. The concept of state has strong physical motivation.

Definition of State of a System

When referring to the state of a system, we shall have the following

definition in mind.

DEFINITION 1-7
The state of a system is a set of quantities xl(t), x 2(t), ... ,x.(t)
t See [D-I], [0-1], [S-I], [S-2], [T-I], [W-I], [Z-I].
Sec. 1.2 Introduction 17

which if known at t = to are determined for t > to by specifying

the inputs to the system for t > to'

System Classification

Systems are described by the terms linear, nonlinear, time-invariant, t

and time-varying. We shall classify systems according to the form of their
state equations.+ For example, if a system is nonlinear and time-varying,
the state equations are written

i(t) = a(x(t), u(t), t). (1.2-1)

Nonlinear, time-invariant systems are represented by state equations of the

form

i(t) = a(x(t), u(t». (1.2-2)

If a system is linear and time-varying its state equations are

i(t) = A(t)x(t) + B(t)u(t), (1.2-3)

where A(t) and B(t) are n X nand n X m matrices with time-varying elements.
State equations for linear, time-invariant systems have the form

i(t) = Ax(t) + Bu(t), (1.2-4)

where A and B are constant matrices.

Output Equations

The physical quantities that can be measured are called the outputs and are
denoted by YI (t), Y2(t), ... ,yit). If the outputs are nonlinear, time-varying
functions of the states and controls, we write the output equations
y(t) = c(x(t), u(t), t). (1.2-5)

If the output is related to the states and controls by a linear, time-invariant

relationship, then
y(t) = Cx(t) + Du(t), (1.2-6)

where C and Dare q X nand q X m constant matrices. A nonlinear, time-

t Time-invariant, stationary, and fixed will be used interchangeably.

~ See Chapter 1 of [S-lJ for an excellent discussion of system classification.
 r-----'
I
r - - - - J>~Os§.S~ IO_BE £QN.I!~.o....!--!:~___ -, I :

r(t)
I
uU) x(t) f x(t) I I
I: I

~ CONTROLLER

r _____r-l
a c y(t)
I I
y(t) :~
I Measurement t
I process
I
L ____________________ ~
I

..
CIII
(a)
r-------,
I I

u(t) PROCESS TO BE CONTROLLED

------------ ---------1
I
r(t) x(t) xU) I
CONTROLLER
f I
rI
C
1+
y(t)

I
y(t)
I L ______ J
1 Measurement
I process
L _____________________ JI

(b)

Figure 1-9 (a) Nonlinear system representation. (b) Linear system representation
Sec. 1.2 Introduction 19

varying system and a linear, time-invariant system are shown in Fig. 1-9.
ret), which has not been included in the state equations and represents any
inputs that are not controlled, is called the reference or command input.
In our discussion of optimal control theory we shall make the simplify-
ing assumption that the states are all available for measurement; that is,
yet) = x(t).

Solution of the State Equations-Linear Systems

For linear systems the state equations (1.2-3) have the solution

x(t) = fII(t, to)x(t o) + s:. fII(t, -r)B(-r)u(-r) d-r: (1.2-7)

where fII(t, to) is the state transition matrixt of the system. If the system is
time-invariant as well as linear, to can be set equal to 0 and the solution of
the state equations is given by any of the three equivalent forms

x(t) =.P-1{[sI - Arlx(O) + [sI - ArIBU(s)}, (1.2-8a)

x(t) = 'p-I{cD(s)x(O) + H(s)U(s)}, (1.2-8b)

x(t) = EAtX(O) + EAt f: E-ATBu(-r) d-r, (I.2-8c)

where U(s) and cD(s) are the Laplace transforms of u(t) and fII(t), g-l{ . }
denotes the inverse Laplace transform of { . }, and EAt is the n X n matrix

Equation (1.2-8a) results when the state equations (1.2-4) are Laplace trans-
formed and solved for Xes). Equation (1.2-8b) can be obtained by drawing
a block diagram (or signal flow graph) of the system and applying Mason's
gain formula.:f: Notice that H(s) is the transfer function matrix. The solution
in (1.2-8c) can be found by classical methods. The equivalence of these three
solutions establishes the correspondences

EAt = 'p-I{cD(s)} = 'p-I{[sI - A]" I} ~ fII(t), (1.2-10)

EAt f: E-ATBu(-r) d-r = 'p-l{H(s)U(s)} =g-l{[sI - A]"IBU(s)}

(1.2-11)
~ fII(t) f: fII(--r)Bu(-r)d-r.

t Ip(t, to) is also called the fundamental matrix.

~ See tWo!].
20 Describing the System and Evaluating Its Performance Sec. 1.2

Properties of the State Transition Matrix

It can be verified that the state transition matrix has the properties shown
in Table 1-1 for all t, to, tl, and t 2.

Table 1-1 PROPERTIES OF TIlE LINEAR SYSTEM STATE TRANSITION MATRIX

Time-invariant systems Time-varying systems

IjI(O) = I ljI(t, t) = I

d d
(jjljl(t) = Acp(t) (jjljl(t, to) = A(t)ljI(t, to)

Determination of the State Transition Matrix

For systems having a constant A matrix, the state transition matrix, cp(t),
can be determined by any of the following methods:
1. Inverting the matrix [sI - A] and finding the inverse Laplace trans-
form of each element.
2. Using Mason's gain formula to find cI>(s) from a block diagram or
signal flow graph of the system [the ijth element of the matrix cI>(s) is
given by the transmission X;(s)/xj(O)] and evaluating the inverse La-
place transform of cI>(s).
3. Evaluating the matrix expansion

For high-order systems (n > 4), evaluating fAr numerically (with the
aid of a digital computer) is the most feasible of these methods.
For systems having a time-varying A matrix the state transition matrix
can be found by numerical integration of the matrix differential equation

(1.2-12)

with the initial condition cp(t 0' to) = I.

t Although a digital computer program for the evaluation of this expansion is easy to
write, the running time may be excessive because of convergence properties of the
series. For a discussion of more efficient numerical techniques see [0-1], p. 315ff.
Sec. 1.2 Introduction 21

Control/ability and Observabilityt

Consider the system

x(t) = a(x(t), u(t), t) (1.2-13)

for t > to with initial state x(t o) = Xo'

DEFINITION 1-8
If there is a finite time tl > to and a control u(t), t E [to, t 1 ], which
transfers the state Xo to the origin at time t l ' the state Xo is said to be
controllable at time to' If all values of Xo are controllable for all
to, the system is completely controllable, or simply controllable.

Controllability is very important, because we shall consider problems

in which the goal is to transfer a system from an arbitrary initial state to
the origin while minimizing some performance measure; thus, controlla-
bility of the system is a necessary condition for the existence of a solution.
Kalmant has shown that a linear, time-invariant system is controllable
if and only if the n X mn matrix

has rank n. If there is only one control input (m = 1), a necessary and suffi-
cient condition for controllability is that the n X n matrix E be nonsingular.
The concept of observability is defined by considering the system (1.2-13)
with the control u(t) = 0 for t > to'§

DEFINITION 1-9
If by observing the output y(t) during the finite time interval [to, t)]
the state x(t o) = Xo can be determined, the state Xo is said to be
observable at time to' If all states Xo are observable for every to, the
system is called completely observable, or simply observable.

Analogous to the test for controllability, it can be shown that the linear,
time-invariant system

x(t) = Ax(t) + Bu(t) (1.2-14)

y(t) = Cx(t) (1.2-15)
t See [K-2], [K-3].
t See [K-2].
§ If the system is linear and time-invariant, u can be any known function-see [Z-I], p. 502.
22 Describing the System and Evaluating Its Performance Sec. 1.3

is observable if and only if the n X qn matrix

has rank n. If there is only one output (q = 1) G is an n X n matrix and a

necessary and sufficient condition for observability is that G be nonsingular.
Since we have made the simplifying assumption that all of the states can
be physically measured (y(t) = x(t», the question of observability will not
arise in our subsequent discussion.

1.3 CONCLUDING REMARKS

In control system design, the ultimate objective is to obtain a controller

that will cause a system to perform in a desirable manner. Usually, other
factors, such as weight, volume, cost, and reliability also influence the con-
troller design, and compromises between performance requirements and
implementation considerations must be made. Classical design procedures
are best suited for linear, single-input, single-output systems with zero initial
conditions. Using simulation, mathematical analysis, or graphical methods,
the designer evaluates the effects of inserting various physical devices into
the system. By trial and error either an acceptable controller design is ob-
tained, or the designer concludes that the performance requirements cannot
be satisfied.
Many complex aerospace problems that are not amenable to classical
techniques have been solved by using optimal control theory. However, we
are forced to admit that optimal control theory does not, at the present time,
constitute a generally applicable procedure for the design of simple con-
trollers. The optimal control law, if it can be obtained, usually requires a
digital computer for implementation (an important exception is the linear
regulator problem discussed in Section 5.2), and all of the states must be
available for feedback to the controller. These limitations may preclude
implementation of the optimal control law; however, the theory of optimal
control is still useful, because
1. Knowing the optimal control law may provide insight helpful in
designing a suboptimal, but easily implemented controller.
2. The optimal control law provides a standard for evaluating proposed
suboptimal designs. In other words, by knowing the optimal control
law we have a quantitative measure of performance degradation caused
by using a suboptimal controller.
Problems Introduction 23

REFERENCES

A-I Athans, M., "The Status of Optimal Control Theory and Applications for.
Deterministic Systems," IEEE Trans. Automatic Control (1966), 580-596.
D-l Derusso, P. M., R. J. Roy, and C. M. Close, State Variables for Engineers.
New York: John Wiley & Sons, Inc., 1965.
K-l Kliger, I., "On Closed-Loop Optimal Control," IEEE Trans. Automatic
Control (1965), 207.
K-2 Kalman, R. E., "On the General Theory of Control Systems," Proc. First
[FAC Congress (1960), 481-493.
K-3 Kalman, R. E., Y. C. Ho, and K S. Narendra, "Controllability of Linear
Dynamical Systems," in Contributions to Differential Equations, Vol. 1.
New York: John Wiley & Sons, Inc., 1962.
0-1 Ogata, K, State Space Analysis of Control Systems. Englewood Cliffs, N.J.:
Prentice-Hall, Inc., 1967.
Sol Schwarz, R. J., and B. Friedland, Linear Systems. New York: McGraW-Hili,
Inc., 1965.
S-2 Schultz, D. G., and J. L. Melsa, State Functions and Linear Control Systems.
New York: McGraw-Hill, Inc., 1967.
T-l Timothy, L. K, and B. E. Bona, State Space Analysis: An Introduction.
New York: McGraw-Hili, Inc., 1968.
W-l Ward, J. R., and R. D. Strum, State Variable Analysis (A Programmed
Text). Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
Z-1 Zadeh, L. A., and C. A. Desoer, Linear System Theory: The State Space
Approach. New York: McGraw-Hill, Inc., 1963.

PROBLEMS

1-1. The tanks A and B shown in Fig. I-PI each have a capacity of 50 gal. Both
tanks are filled at t = 0, tank A with 60 lb of salt dissolved in water, and

Figure I-PI
24 Describing the System and Evaluating Its Performance Problems

tank B with water. Fresh water enters tank A at the rate of 8 gal/min, the
mixture of salt and water (assumed uniform) leaves A and enters B at the
rate of 8 gal/min, and the flow is incompressible. Let q(t) and pet) be
the number of pounds of salt contained in tanks A and B, respectively.
(a) Write a set of state equations for the system.
(b) Draw a block diagram (or signal flow graph) for the system.
(c) Find the state transition matrix <p(t).
(d) Determine q(t) and pet) for t :;:::: O.
1-2. (a) Using the capacitor voltage ve(t) and the inductor current iL(t) as states,
write state equations for the RLC series circuit shown in Fig. I-P2.

R L e(t)

'}--TI ,
c
+--+-+- Figure I-P2

(b) Find the state transition matrix <p(t) if R = 3 n, L = 1 H, C = i F.

(c) If ve(O) = 0, iL(O) = 0, and e(t) is as shown, determine ve(t) and iL(t)
for t:;:::: O.
1-3. (a) Write a set of state equations for the mechanical system shown in. Fig.
I-P3. The applied force isJ(t), the block has mass M, the spring constant
is K, and the coefficient of viscous friction is B. The displacement of the
block, yet), is measured from the equilibrium position with no force
applied.

y(t)

f(t)

Figure I-P3

(b) Draw a block diagram (or signal flow graph) for the system.
(c) Let M = I kg, K = 2 N/m, B = 2 N/m/sec, and determine the state
transition matrix <p(t).
(d) If yeO) = 0.2 m, yeO) = 0, and J(t) = 2c 2t N for t:;:::: 0, determine yet)
and yet) for t :;:::: O.
Problems Introduction 25

1-4. Write a set of state equations for the electrical network shown in Fig. I-P4.

Figure I-P4

1-5. Write state equations for the mechanical system in Fig. I-PS. A. is the applied
torque, I is the moment of inertia, K is the spring constant, and B is the
coefficient of viscous friction. The angular displacement (J(t) is measured
from the equilibrium position with no torque applied.

Figure I-PS

1-6. A chemical mixing process is shown in Fig. I-P6. Water enters the tanks
at rates of WI(t) and W2(t) ftlfmin, and met) ft3fmin of dye enters tank 1.
VI(t) and V2(t) ftl of dye are present in tanks 1 and 2 at time t. The
tanks have cross-sectional areas /XI and /X2. Assume that the flow rate between
the two tanks, q(t), is proportional to the difference in head with propor-

~W2(t)

Area 0/[
/1\, il~ / 1\\

T co cb
T
1 --
q(t)

I
Tank I Tank 2
Figure I-P6
26 Describing the System and Evaluating Its Performance Problems

tionality constant k ftJ 1ft-min, and that the mixtures in the tanks are homo-
geneous. Write the differential equations of the system, using hl(t), h 2 (t),
VI (t), and V2(t) as state variables.

1-7. Write a set of state equations for the electromechanical system shown in
Fig. I-P7. The amplifier gain is K., and the developed torque is A(t) = K,if(t),
where K. and K, are known constants.

Amplifier
+

+ Gain
K. viscous friction, B
.,;;:;;;;;;;;~

Figure l-P7

1-8. Write a set of state equations for the mechanical system shown in Fig. loPS.
The displacements Yl(t) and Y2(t) are measured from the equilibrium posi-
tion of the system with no force applied.

Equilibrium position of M2

Y2«)

Figure l·PS
Problems Introduction 27

1-9. Write a set of differential equations, in state form, for the coupled RLC
network shown in Fig. 1-P9.

Figure 1-P9

1-10. Write a set of state equations for the network shown in Fig. 1-PI0. R 2 (t)
is a time-varying resistor, and the circuit also contains a nonlinear resistor.

+
e Nonlinear
resistor

Figure 1-P10

1-11. Show that the state transition matrix satisfies the properties given in Table
1-1.
Hint:

is a solution of
:i.(t) = A(t)x(t).

1-12. Draw a block diagram, or signal flow graph, and write state and output
equations that correspond to the transfer functions:
a yes) _ _5_ b Yes) _ .l
( ) U(s) - s[s + 1] ( ) U(s) - S2
C yes) = 10 d Yes) _ 8
() U(s) S3 + + +
5s 2 6s 3 ( ) U(s) - 2s 4 + 6s 3 + 14s2 + 7s + 1
(e) yes) = 5[s+ 2] (f) Yes) = [s + 1][s + 2]
U(s) s[s+ 1] U(s) S2
yes) _ + +
1O[s2 2s 3] h yes) _ 4
(g) U(s) - S3 + + +
5s 2 6s 3 ( ) U(s) - [s + 1][s + 2]
. Yes) _ [S2+ + 7s 12] . yes) _ 8[S3 + s + 2]
(1) U(s) - s[s+ + 1][s 2] (J) U(s) - 2s 4 + 6s 3 + 14s 2 + 7s + 1
28 Describing the System and Evaluating Its Performance Problems

1·13. Find the state transition matrices cp(t) for the systems (a), (b), (e), (f), (h),
and (i) in Problem 1·12.
1·14. For each of the following systems determine:
(i) If the system is controllable.
(ii) If the system is observable.
(iii) The block diagram or signal flow graph of the system.
(a) x(t) = [~ ~ ] x(t) + [~J u(t); yet) = XI (t).

(b) x(t) = [~ ~J x(t) + [~J u(t); yet) = Xz(t).

(c) The coupled circuit in Problem 1-9 with M = 0, yet) = [~:j:J

(d) The coupled circuit in Problem 1-9 with M = O.5H, Ll = 1.0H, L z =
O.5H, RI = 2.0 n, R z = 1.0 n, C = 0.5F, and yet) = ve(t).

(e) x(t) = [-~ -~ ~]X(t) + [~~]U(t); yet) = XI(t).

-3 -4 -2 1 0

(f) x(t) = [-~ -~ ~]X(t) + [~~]U(t); yet) = XI(t).

-3 0 -2 1 0

(g) x(t) = [~ ~ : : ]X(t) + [~]U(t);

-ao -al -az -a3 1
yet) = XI(t); at -=/=. 0, i = 0, 1,2,3.
1·15. What are the requirements for the system

i(l)
[1.
~ ~
0
Az
0
0
~ }(I)+ [:});
0 A3
0 0 A4 b4
yet) = [CI Cz C3 c41x(t)

to be:
(i) Controllable?
(ii) Observable?
Assume that Ai> i = 1, ... , 4 are real and distinct.