Adama Science and Technology

University

PCE6101 Linear Systems

Theory
(Optimal Control)

ASTU
School of EEC

1. P-Type Optimal Regulator

SISO system
x  Ax  Bu
y  Cx  Du
Quadratic function
1  T
J
2 0 x Qx  uT Rudt

Find a control law u= -Kx for a dynamical system in

terms of minimizing the objective function where Q
and R are user-defined weighting matrix and
coefficient, respectively.

1
Cont…
1 
J
2 
0
q11 x12  q22 x22  ru 2 dt

x2
x1

Cont…
Feedback control law to minimize J
u  Kx
where K is given by
K  R 1 BT P
P is the solution that satisfies the following Riccatti
equation:
AT P  PA  PBR 1 BT P  Q  0
If A is a real matrix and Q is a real symmetric matrix,
then P is a real symmetric matrix and PD.

2
 Cont…
How to choose Q and R
• LQR control is to minimize the quadratic cost function of
1 
J
2 
0
xT Qx  uT Rudt

• Q and R are design parameters. The larger Q and R are,

the more state variables and control signals are penalized.
• Choosing large R stabilizes the system with less control
signal (expensive control strategy).
• On the other hand, choosing small R stabilizes the system
with larger control signal (cheap control strategy). That is,
small R implies less concern about the changes of u.

Cont…
• Similarly, choosing large Q stabilizes the system with the
smaller changes in the states.
• On the other hand, choosing small Q stabilizes the system
with larger changes in the states(less concern about the
changes).
• There is a trade-off between the choice of these two.
• One method is to keep Q and alter R only:
- Increase R, if there is a limit on the control output signal
(for instance, causing actuator's saturation)
- Decrease R, if having a large control signal is not a
problem for the system.

3
 Cont…
Optimal control
Step 1: Find the solution P of the Riccatti equation from
A, B, Q, and R.

AT P  PA  PBR 1 BT P  Q  0
Step 2: Find the feedback gain matrix K.
K  R 1 BT P
% MATLAB command
[K,P,E]= lqr(A,B,Q,R) % E are the eigenvalues of
(A-BK)

Cont…
For example, consider a LTI system
 1 1  0  1
x    x  1  u , x (0)  1
 0 1   
y  1 0 x
Find the optimal regulator to minimize J.
1 
J
2 0
4 x12  x22  u 2 dt

4 0
From J, Q   , R 1
 0 1 

4
 Cont…
Letting AT P  PA  PBR 1 BT P  Q  0
p p2 
P 1
 p2 p3 

and substituting this to the Riccati equation gives the

following equation:
-1 0   p1 p2   p1 p2  -1 1 
 1 -1  p p    p p   0 -1

  2 3  2 3
 p1 p2  0   p1 p2   4 0 
    1 0 1   0
 p2 p3  1   p2 p3   0 1 

Cont…
From -2 p1  p22  4  0
p1  2 p2  p2 p3  0
2( p2  p3 )  p32  1  0

Solve the above equation and take a positive value

p34  8 p33  18 p32  8 p3  23  0
p22  5.621 p2  4  0
1 2
p1   p2  2
2
 p1  1.7959, p2  0.6390, p3  0.8105

5
Cont…
p p2  1.7959 0.6390 
P 1 
 p2 p3  0.6390 0.1085
1.7959 0.6390 
 K  R 1 BT P  1 0 1  
0.6390 0.8105
  0.6390 0.8105
 u    0.6390 0.8105 x

Cont…
MATLAB program

A= [-1 1;0 -1]; B= [0;1]; C= [1 0]; D= 0;

Q= [4 0;0 1]; R= 1;
K= lqr(A,B,Q,R)

AA= A-B*K; BB= [0;0]; CC= eye(2); DD= D;

t= 0:0.02:5;
r= zeros(size(t));
x0=[1;1];
y= lsim(AA,BB,CC,DD,r,t,x0);
plot(t,y(:,1),t,y(:,2))

6
Cont…
Responses of x1(t) and x2(t)
1.2

1 Q= diag(4,1), R= 1
0.8
x(0)= [1 1]T
x1(t)
0.6

0.4

0.2
x2(t)
0

-0.2
0 1 2 3 4
t 5

Cont…
Responses of u(t)

0.5

u(t)
0

-0.5

-1
Q= diag(4,1), R= 1
-1.5
0 1 2 3 4 5
t

7
Cont…
Responses to different Q

1
Q= diag(4,1), R= 1
x1(t) Q= diag(10,10), R= 1
0.5

x2(t)
-0.5
0 1 2 3 4 5
t

Cont…
Responses to different Q

0
u(t)
-1

-2
Q= diag(4,1), R= 1
-3
Q= diag(10,10), R= 1
-4
0 1 2 3 4 5
t

8
 2. PI-Type Optimal Regulator
(Problem) Consider a SISO plant with constant
disturbance.
x  Ax  Bu +Wd , x (0)
y  Cx  Du
P-type regulator introduces a steady-state error for
constant disturbance. So, let's design a PI-type
regulator where d means disturbance.
The new state variable is defined as
z   ydt

Cont…
Consider the augumented system again

x  Ax
   B u +W d
y  Cx   Du


 x  A 0    B   W 
where x    , A    , B   D  ,W   0  ,
z
   C 0     
C  C 0 , D  D

9
 Cont…
Find a control law for the augumented system in terms
of minimizing the following objective function where Q
and R are user-defined weighting matrices.
1  T 
J
2 0  
x Qx  uT Rudt

Feedback control to minimize J


u  Kx
where K  R 1 B T P and P is the solution that
satisfies the following Riccatti equation:
A T P  PA
   PB
  R 1 B T P  Q  0

Cont…
Consider a scalar system.
x   x  2u  d , x(0)= 0.5
yx
From the system, a= -1, b= 2, w= 1, c=1.
The augmented system is
 x   a 0   x  b   w 
 z    c  u+
0   z   0   0 
d
  
 1 0  x   2  w
  u+ d
1 0   z   0   0 
A B

10
 Cont…
Find the optimal regulator to minimize J.
1 
J
2 
0
x 2  4 z 2  u 2 dt
1 0 
From J, Q   , R 1
0 4

AT= [-1 0;1 0]; BT= [2;0]; CT= [1 0]; D= 0;

QT= [1 0;0 4]; R= 1;
KT= lqr(AT,BT,QT,R)

KT = 1.3028 2.0000

Cont…
Thus, the PI-type regulator yields

    1.3028 2  x 
u   Kx z
 
 1.3028 x  2 ydt 

11
Cont…
MATLAB program
AT= [-1 0;1 0]; BT= [2;0]; CT= [1 0]; D= 0;
QT= [1 0;0 4]; R= 1; w= 1;
KT= lqr(AT,BT,QT,R)
AA= AT-BT*KT; BB= [w;0]; CC= [1 0]; DD= D;
t= 0:0.02:10;
r= ones(size(t));
x0= [0.5;0];
y= lsim(AA,BB,CC,DD,r,t,x0);
plot(t,y(:,1),'linewidth',2)
xlabel('t'),ylabel('y(t)'),grid on

Cont…
Response
0.5

0.4

0.3

0.2

0.1

-0.1
0 2 4 6 8 10
t

12
3. PI-Type Optimal Tracking Cont.
(Problem) Consider a SISO LTI system with
constant disturbance d:
x  Ax  Bu+Wd , x (0)
y  Cx  Du

In order to design the PI-type tracking controller

without steady-state error for a constant disturbance
or the type-0 plant, the new state variable is defined
as follows:

z  ( y  r )dt

Cont…
Differentiating both sides once gives
z = y  r
Consider the augumented system again
   B u +W d  0  r
x  Ax 1 
 
y  Cx   Du

 x  A 0    B   W 
where x    , A    , B   D  ,W   0  ,
z
   C 0     
C  C 0 , D  D

13
 Cont…
Find a control law for the augumented system in terms
of minimizing the following objective function where Q
and R are user-defined weighting matrices.
1  T 
J
2 0  
x Qx  uT Rudt

Feedback control to minimize J


u  Kx
where K  R 1 B T P and P is the solution that
satisfies the following Riccatti equation:
A T P  PA
   PB
  R 1 B T P  Q  0

Cont…
The PI-type tracking controller can be written as
     K k   x    K x  k (r  y )dt
u   Kx 1 2  
z
1 2 
d
r + + u y
 k2 Plant
- -
K1 x

• Disturbance d is considered constant

• Setpoint(or reference) input r is considered a step
signal

14
 Cont…
Design procedure
Step 1: Write the augmented system from the system.
(Original) x  Ax  Bu +Wd , x (0)
y  Cx  Du
(Augmented)
   B u  W d  0  r , x (0)1
x  Ax 1 
 

y  Cx  Du
where
 A 0    B   W  
A    , B   0  , W   0  , C  C 0 , D  D
C 0     

Cont…
 , B
Step 2: Obtain K= [K1 k2] from the given A
and Q and R .

Step 3: Implement the PI-type tracking controller

u= -K1x - k2z
= -K1x + k2(r - y)dt

15
Cont…
Consider the previous scalar system with disturbance
d= 1 and x(0)= 0.
x   x  2u  d , x(0)= 0
yx
From the system, a= -1, b= 2, w= 1, c=1.
The augmented system is
 x   a 0   x  b   w   0 
 z    c  u+ d-
0   z  0   0  1 
r
  
 1 0   x   2   w 0 
  u+ d- r
1 0   z   0   0  1 
A B

Cont…
Find the optimal tracking controller to minimize J.
1 
J
2 
0
x 2  4 z 2  u 2 dt
1 0  
From J, Q   , R 1
 0 4 

AT= [-1 0;1 0]; BT= [2;0]; CT= [1 0]; D= 0;

QT= [1 0;0 4]; RT= 1;
KT= lqr(AT,BT,QT,RT)

KT = 1.3028 2.0000

16
 Cont…
Thus, the PI-type optimal tracking controller gives

    1.3028 2  x 
u   Kx z
 
 1.3028 x  2 z


 1.3028 x  2 (r  y )dt

The MATLAB program for obtaining the unit step

response for 20sec gives

Cont…
Feedback system d
r +
 + u + + x x y
2 2  1
- - +
-1
PI-type
1.3028
controller

x   x  2u  d , x(0)  0
yx


u  1.3028 x  2 (r  y )dt

17
Cont…
MATLAB program
function prog13_02
t= 0; x= 0; h= 0.02; r= 1; z= 0; loop= 1000;
k1= 1.3028; k2= 2; buf= [t x];
for i=1:loop
e= r-x; z= z+e*h; % rectangle rule
u= -k1*x+2*z;
x= RK4(@fun,t,x,u,h); % 4th-order R-K method
t= t+h;
buf= [buf;t x];
end

Cont…
plot(buf(:,1),buf(:,2),'linewidth',2)
xlabel('t'),ylabel('y(t)'),grid on

function xdot= fun(t,x,u)

d= 1; % disturbance
xdot= -x+2*u+d;

18
Cont…
Unit step response
1.2

0.8

0.6

0.4

0.2

0
0 2 4 6 8 10 12 14 16 18 20
t

Cont…
Example 1: Consider a system with disturbance d= 1
and x(0)= 0.

 0 1 0 0  0 

x   0 0 1  x  0  u + 0  d

 18 15 2  1  1 
y  1 0 0 x
Find a PI-type optimal tracking controller to minimize
J and draw the unit step response for 50sec.
1 
J
2  0
5 x12  2 x22  2 x32  10 z 2  u 2 dt

19
Cont…
Solution: The augumented system gives
   B u +W d  0  r
x  Ax 1 
 
y  Cx   Du

 0 1 0 0 0  0 
 0 0 1 0   0   0 

where A   ,B  ,W  ,
 18 15 2 0  1  1 
     
 1 0 0 0 0  0 
C  1 0 0 0 , D  D

Cont…
From J,
1 
J
2 
0
5 x12  2 x22  2 x32  10 z 2  u 2 dt

Q  diag (5, 2, 2,10), R  1

AT= [0 1 0 0;0 0 1 0;-18 -15 -2 0;1 0 0 0];

BT= [0;0;1;0]; CT= [1 0 0 0]; DT= 0;
QT= diag([5 2 2 10]); RT= 1;
KT= lqr(AT,BT,QT,RT)

KT = 2.8282 1.5725 1.0241 3.1623

20
 Cont…
Thus, the PI-type optimal tracking controller gives


u   Kx
 x
   2.8282 1.5725 1.0241 3.1623  
z


   2.8282 1.5725 1.0241 x  3.1623 ( r  y ) dt

Cont…
MATLAB program
function prog13_03
t= 0; x= [0;0;0]; h= 0.02; r= 1; z= 0; loop= 2500;
K1= [2.8282 1.5725 1.0241]; k2= 3.1623;
buf= [t x(1)];
for i= 1:loop
e= r-x(1); z= z+e*h; % rectangle rule
u= -K1*x+k2*z;
x= RK4(@fun,t,x,u,h);
t= t+h;
buf= [buf;t x(1)];
end

21
Cont…
plot(buf(:,1),buf(:,2),'linewidth',2)
axis([0 50 0 1.2])
xlabel('t'),ylabel('y(t)'),grid on

function xdot= fun(t,x,u)

d= 1;
xdot= zeros(3,1);
xdot(1)= x(2);
xdot(2)= x(3);
xdot(3)= -18*x(1)-15*x(2)-2*x(3)+u+d;

Cont…
Unit step response
1.2

0.8

0.6

0.4

0.2

0
0 5 10 15 20 25 30 35 40 45 50
t

22
Cont…
(Problem) Consider a SISO plant with constant
disturbance.
x  Ax  Bu +Wd , x (0)
y  Cx  Du
Let's design a PI-type regulator under disturbance d
to stabilize the system while considering the
following conditions:
• Transient response specifications
• Magnitude constraints on x and u

Cont…
Design procedure
Step 1: Write the augmented system from the system.
(Original) x  Ax  Bu +Wd , x (0)
y  Cx  Du
(Augmented)
   B u  W d  0  r , x (0)1
x  Ax 1 
 
y  Cx   Du

where
 A 0    B   W  
A    , B   0  , W   0  , C  C 0 , D  D
C 0     

23
 Cont…
Step 2: Choose Q and R and obtain K= [K1 k2] from the
given A and B
Step 3: Obtain the response of the closed-loop system
with the PI-type tracking controller for different initial
conditions
x  Ax  Bu +Wd , x (0)
y  Cx  Du
u= -K1x + k2(r - y)dt
Step 4: If the transient response specifications and/or
the magnitude constraints are not met, go back to Step 2
and rechoose Q and/or R.

Cont…
Typical Choice:

Q  diag (q11 ,q22 , ,qnn ), R = r

1 1
qii  , r  ,  0
tsi ( xi max ) 2 (umax ) 2

where
• tsi is the desired settling time of xi
• ximax is a constraint on |xi|
• uimax is a constraint on |ui|
• ρ is chosen to tradeoff regulation versus control
effort

24
 Cont…
Example 2: Consider a system with disturbance d
and x(0)= 0.
 0 1 0  0  0 
x   0 0 1  x  0  u + 0  d
 0.4 4.2 2.1 1  1 
y  1 0 0 x
Find a PI-type optimal tracking controller with
Q  diag (1,1,1,10), R  0.01
Q  diag (1,1,1,10), R  1
Q  diag (1,1,1, 20), R  0.01
Q  diag (1,1,1, 20), R  1

Cont…

When disturbance
y(t)

d= 0
u(t)

25
 Cont…
1.5

When disturbance 1

d= -10 0.5

y(t)
Q= diag(1,1,1,10; R= 0.01
0
Q= diag(1,1,1,10; R= 1
Q= diag(1,1,1,20; R= 0.01
-0.5 Q= diag(1,1,1,20; R= 1

0 2 4 6 8 10
t

10

6
u(t)

4 Q= diag(1,1,1,10; R= 0.01
Q= diag(1,1,1,10; R= 1
2 Q= diag(1,1,1,20; R= 0.01
Q= diag(1,1,1,20; R= 1
0
0 2 4 6 8 10
t

26