Computational Control

Data-Driven Predictive Control

Saverio Bolognani

Automatic Control Laboratory (IfA)

ETH Zurich
System response
t−1
X
yt = CAt x + CAt−1−j Buj
| {z }0
j=0
free response | {z }
convolution

Simplifying assumption (SISO)

For now, assume a scalar input u ∈ R and a scalar output y ∈ R.

y0 C 0 0 ... u0
     
 y1  CA   CB 0 ...   u1 
  CA2 
      
y2  x0 +  CAB CB 0 ...   u2 
=
 
  . 
.. ..  . 
  
  ..    .. 
 
 .  .
yL−1 CAL−1 CAL−2 B CAL−3 B ... CAB CB 0 uL−1
| {z } | {z }
OL GL

OL ∈ RL×n (tall) is the extended observability matrix

GL ∈ RL×L (square) is the convolution matrix
1 / 36
Free response

y0 C
  
 y1  CA 
  CA2 
   
 y2=  x0
  . 
..

  .. 

 .
yL−1 CAL−1
| {z } | {z }
yL OL

Assumption: Observable system (SISO)

On , with n the state dimension, is full rank.

Last week we focused on whether we can reconstruct x0 by looking at the output

sequence.
If L < n, then x0 cannot be reconstructed (not enough information)
If L = n, then x0 can be reconstructed: x0 = On−1 yn
If L > n, then the question is overdetermined:
▶ x0 can be reconstructed from a valid output sequence
▶ not all y sequences are valid!

2 / 36
Example, n = 1
1
xt+1 = xt + ut
2
yt = xt

C 1
   
 CA   1/2 
 2   
OL =  CA  =  1/4 
   
 .   .. 
 ..   . 
CAL−1 1/2L−1
Assuming no input, can you reconstruct x0 from
y0 = 1 ?
y0 = 2, y1 = 1?
y0 = 1, y1 = 1?

What are all the output sequences produced by the free response of this system?
Exercise: Try with an undamped oscillator, n = 2.

3 / 36
Suppose that the input sequence u0 , . . . , uL is non zero, but known.

y0 C 0 0 ... u0
      
 y1   CA   CB 0 ...   u1 
  CA2 
      
y2  x0 +  CAB CB 0 ...   u2 
=
 
..  .  ..  . 
  
  ..    .. 
 
 .  .
L−2
yL−1 CAL−1 CA B CAL−1 B ... CAB CB 0 uL−1
| {z } | {z }
OL GL

Can you determine x0 ?

y0 0 0 ... u0 C
      
 y1   CB 0 ...   u1   CA 
  u2   CA2 
      
y2   CAB CB 0 ...
− =  x0

.. ..  .   . 
 
  ..   .. 
  
 .  .
L−2
yL−1 CA B CAL−1 B ... CAB CB 0 uL−1 CAL−1
| {z } | {z }
GL OL

4 / 36
 yL − GL uL = OL x0

Similar to before, the problem can be solved if

yL − GL uL

is in the column image of OL .

For scalar systems

if L < n, underdetermined
if L = n, then x0 = On−1 (yn − Gn un )
if L > n, overdetermined

Admissible input-output sequences

Interesting consequence: for L > n, yL − GL uL needs to belong to a subspace!

5 / 36
All input-output trajectories of length N of the system need to satisfy
    
uL I 0 uL
= L
yL GL OL x0
| {z }
ΛL

ΛL ∈ R(L+L)×(L+n)
if L ≥ n, ΛL has rank L + n

Behavioral representation
Trajectories uL , yL of the system need to belong to the column image of ΛL .

ΛL is a full description of the system, exactly like A, B, C

If you are given the matrix ΛL , you can tell whether an input/output sequence
is compatible with that plant
▶ compatible = “there exists an initial state x0 such that...”

“Behavioral system theory” by Jan C. Willems, from approx the 80s.

System theory without classifying signals as inputs and outputs!

6 / 36
L = n (observability problem)
    
un In 0 un
=
yn Gn On x0
| {z }
Λn

Λn has dimension 2n × 2n
Λn has rank 2n
→ invertible matrix

Any input/output sequence of length n can be “explained” by an initial condition x0 .

Toy example:

xt+1 = xt + ut
yt = xt

7 / 36
L=n+1

u0
 
 .. 

 . 


u0

 un−1 
 .. 
.
  
un  = In+1 0
   
 
un−1

y0 G O
 
n+1 n+1
   
un
  |
 ..  {z } 

 . 
 Λn+1
x0
 yn−1 
yn

Λn+1 has dimension 2(n + 1) × 2n + 1

Λn+1 has rank L + n = 2n + 1
the input/output sequence is a vector in R2(n+1)
→ the input/output sequence belongs to a subspace of dimension 2n + 1

Toy example:

8 / 36
L = n + 1 (prediction problem)

u0
 
 .. 

 . 


u0

 un−1 
 .. 
.
  
un  = In+1 0
   
 
un−1

y0 G O
 
n+1 n+1
   
un
  |
 ..  {z } 

 . 
 Λn+1
x0
 yn−1 
yn

Prediction problem
Suppose that yn is missing. Can you reconstruct it?

Intuitively:
▶ We already know that we can reconstruct x0 from the first n input/output samples
▶ Once we have x0 we can “simulate” the output until yn .

9 / 36
L = n + K (prediction problem)

u0
 
 .. 

 . 

 un−1 
u0
   
 un 
  ..
..
 
.
   
.
   
un−1
     
un+K −1 In+K 0
   
= un
   
y0 G O

 | n+K {z n+K } 
 
  .. 

 ..   . 
 .  Λn+K  
   un+K −1 
 yn−1 
  x0

 yn 

 .. 
 . 
yn+K −1

Prediction problem
Suppose that the sequence yn . . . yn+K −1 is missing. Can you reconstruct it?

10 / 36
“Linear-algebra” interpretation
Any I/O sequence of length L > n belongs to a subspace of dimension L + n

u0
 
 .. 

 . 


 un−1 


 un 

 .. 
.
 
 
un+K −1
 
 = Hg
 
y0

 

 .. 


 . 


 yn−1 


 yn 

 .. 
 . 
yn+K −1
Image representation
columns of H ∈ R2L×(L+n) are a basis of the subspace
g ∈ RL+n is a vector of coefficients
11 / 36
A basis for this subspace is given by the columns on ΛL
▶ We need A, B, C to build ΛL
Can you find another basis?
Hint: m linearly independent vectors from a subspace of dim. m form a basis.
How would you generate columns from this subspace? Data!
" #" #" # " #
(1) (2) (3) (L+n)
uL uL uL uL
(1) (2) (3) ··· (L+n)
yL yL yL yL

I/O sequences must belong to the subspace of valid signals

Assuming linear independence, L + n of them will form a basis.
All and only the valid I/O sequences are of the form
  " (1) (2) (3) (L+n)
#
uL uL uL uL . . . uL
= (1) (2) (3) (L+n) g
yL yL yL yL . . . yL
| {z }
H

with g ∈ RL+n .

12 / 36
13 / 36
Exercise

Consider the scalar (n = 1) SISO system

xt+1 = xt + ut
yt = xt

Consider input/output trajectories of length L = 3.

Generate trajectories of length L (random initial conditions, random inputs)
Check the rank of the matrix H for an increasing number of columns
What is the maximum rank?

14 / 36
Data matrix

Data is expensive and hard to obtain.

to predict K steps in the future for a system of dimension n we need
trajectories of length L = K + n
we need at least L + n = K + 2n of them

A very efficient use of your data points:

... u0 u1 u2 u3 u4 u5 u6 u7 ...

rearranged as  
u0 u1 u2 ... uL+n−1

 u1 u2 u3 

HL (u) = 
 u2 u3 u4 

 ... 
uL−1 u2L+n−2
Have you met this matrix before?

15 / 36
  
u0 u1 u2 ... uL+n−1
 u1 u2 u3 
 
HL (u) = 
 u2 u3 u4 

 ... 
uL−1 u2L+n−2
 
y0 y1 y2 ... yL+n−1
 y1 y2 y3 
 
HL (y) = 
 y2 y3 y4 

 ... 
yL−1 y2L+n−2

All system trajectories of length L are generated as

 
HL (u)
g
HL (y)

u and y sequences of length 2L + n − 1

g ∈ RL+n free vector of parameters

16 / 36
K -step predictor (L = n + K )
Given the past input, past output, and future input, compute future output.
 
  upast
HL (udata ) ufuture 
g= 
HL (ydata )  ypast 
yfuture

g ∈ R2n+K and yfuture ∈ RK unknowns → total 2(n + K )

2(n + K ) equations

Many uses:
Filtering
Recover missing data
Prediction / simulation
Control

17 / 36
Model-based Predictive Control

Consider an output tracking MPC problem (SISO, for now).

At each time, the state x is measured and the following optimization problem is
solved:

K
X −1
min yk2 + ruk2
u,x,y
k=0

subject to xk+1 = Axk + Buk

yk = Cxk
x0 = x
uk ∈ Uk ∀k
yk ∈ Yk ∀k

The model equations enforce that u and y are trajectories of the plant for the next
K steps.

18 / 36
Data-enabled Predictive Control

Replace the state-space model with our new behavioral model

K
X −1
min yk2 + ruk2
u,y,g
k=0
 
  upast
HL (udata )  u 
subject to g= 
HL (ydata ) ypast 
y
uk ∈ Uk ∀k
yk ∈ Yk ∀k

where
a new decision variable g has been introduced
upast and ypast are the last n inputs and outputs of the system, and play the
role of the initial condition

19 / 36
 K −1 K
X −1
min yk2 + ruk2
X
min yk2 + ruk2 u,y,g
u,x,y k=0
k=0  
subject to xk+1 = Axk + Buk   upast
HL (udata )  u 
yk = Cxk subject to g= 
HL (ydata )  ypast 
x0 = x y
uk ∈ Uk ∀k uk ∈ Uk ∀k
yk ∈ Yk ∀k yk ∈ Yk ∀k

Computationally more complex (2n + K additional decision variables)

Larger memory footprint (data instead of parametric model A, B, C)
No additional state estimator needed.

20 / 36
A few remarks
Assumption
We assumed persistence of excitation in the collected data
technically, to ensure that the data matrix has full rank
intuitively, because we want the data to represent all dynamics of the system

This is a standard requirement in identification, and it is typically resolved by

noisy/random identification input sequences
relying on ambient/process noise (for some applications)
using historical data + a posteriori linear independence test

21 / 36
MIMO case

The theory extends very naturally to multiple inputs and multiple outputs.

Lag of a system
The lag of a system is the smallest ℓ such that
 
C
 CA 
Oℓ =   ... 


CAℓ−1

has rank n.

A past trajectory of length Tini ≥ ℓ is needed as “initial conditions”.

For MIMO systems
▶ input u ∈ Rm
▶ output y ∈ Rp
▶ state x ∈ Rn
an input/output sequences of at length T ≥ (m + 1)(Tini + K ) + n − 1 is needed
For SISO systems, ℓ = n.

22 / 36
Lag and system size estimation

→ a past trajectory of length Tini ≥ ℓ is needed as “initial conditions”

→ input/output sequences of at length T ≥ (m + 1)(Tini + K ) + n − 1 is needed

As ℓ ≤ n, in practice we need an a-priori estimate of the minimal realization size n.

First principles
▶ for example, electro-mechanical systems
A posteriori verification
▶ Build increasingly bigger Hankel matrices and verify the rank

Once the order is estimated, Tini = n is a valid solution, although in some cases
side information can allow to choose a smaller Tini .
K is determined by the control specifications (similar to MPC).

23 / 36
The effect of noise
In theory, adding data to our data matrix does not hurt
it ensures that we have enough data in case we are uncertain about n
it ensures that all the system modes have been excited
the rank of H stops increasing after mL + n
" #
(1) (2) (3)
uL uL uL ...
(1) (2) (3) g
yL yL yL ...
| {z }
H

This is true only in the noiseless case.

In the presence of noise in the data, the rank of H will increase past mL + n.

24 / 36
Consequences of noise in the data
Consider the (pathological) case in which one of the outputs is identically zero.
 
⋆ ⋆ ⋆
yk = Cxk = x
0 0 0 k
All output sequences should be of the form
 
⋆
yk =
0

However, if the data contains noise, some columns in the data matrix may present
a non-zero second coordinate in the output
" #
(1) (2) (3)
uL uL uL ...
(1) (2) (3) g
yL yL yL + ϵ . . .
| {z }
H

The controller is led to

(3)
“assume” that the input sequence uL is capable of affecting the last
coordinate of the output!
use a large g (→ large input!) to try to affect the uncontrollable output

25 / 36
Enforcing the rank condition
Model selection and enforcement of model dimension was easier in identification.
Remember?
 
Σ1 0r×n−r ⊤
= U1 Σ1 V1⊤
  
Hk,ℓ = U1 U2 V1 V2
0m−r×r 0m−r×n−r

Suppose that we know that the system has dimension exactly n.

K
X −1
min ∥yk ∥2Q + ∥uk ∥2R
u,y,g
k=0
 
  upast
HL (udata )  u 
subject to g= 
HL (ydata ) ypast 
| {z } y
H

uk ∈ Uk ∀k
yk ∈ Yk ∀k

H should have rank L + n but most likely it won’t.

26 / 36
Rank constrained data-driven control

K
X −1
min ∥yk ∥2Q + ∥uk ∥2R
u,y,g,ûdata ,ŷdata
k=0

subject to (ûdata , ŷdata ) = argmin ∥ûdata − udata ∥2 + ∥ŷdata − ydata ∥2

 
HL (ûdata )
subject to rank =L+n
HL (ŷdata )
 
  upast
HL (ûdata )  u 
g=  
HL (ŷdata ) ypast 
y
uk ∈ Uk ∀k
yk ∈ Yk ∀k

nested identification problem

non convex (rank constraint)
computationally hard

27 / 36
Step 1: At most L + n non-zero elements of g are needed

K
X −1
min ∥yk ∥2Q + ∥uk ∥2R
u,y,g,ûdata ,ŷdata
k=0

subject to (ûdata , ŷdata ) = argmin ∥ûdata − udata ∥2 + ∥ŷdata − ydata ∥2

 
HL (ûdata )
subject to rank =L+n
HL (ŷdata )
 
  upast
HL (ûdata )  u 
g=  
HL (ŷdata ) ypast 
y
∥g∥0 ≤ L + n
uk ∈ Uk ∀k
yk ∈ Yk ∀k

28 / 36
Step 2: Convex relaxation → drop the rank constraint

K
X −1
min ∥yk ∥2Q + ∥uk ∥2R
u,y,g,ûdata ,ŷdata
k=0

subject to (ûdata , ŷdata ) = argmin ∥ûdata − udata ∥2 + ∥ŷdata − ydata ∥2

 
  upast
HL (ûdata )  u 
g= 
HL (ŷdata ) ypast 
y
∥g∥0 ≤ L + n
uk ∈ Uk ∀k
yk ∈ Yk ∀k

What is the solution to the nested argmin?

29 / 36
Step 3: from sparsity of g to a bound on ∥g∥1

K
X −1
min ∥yk ∥2Q + ∥uk ∥2R
u,y,g
k=0
 
  upast
HL (udata )  u 
subject to g= 
HL (ydata ) ypast 
y
∥g∥1 ≤ α
uk ∈ Uk ∀k
yk ∈ Yk ∀k

Is it a convex problem now?

30 / 36
Step 4: Replace the bound on ∥g∥1 with a regularization term

Regularized data-enabled predictive control

K
X −1
min yk2 + ruk2 +λg ∥g∥1
u,y,g
k=0
 
  upast
HL (udata )  u 
subject to g= 
HL (ydata ) ypast 
y
uk ∈ Uk ∀k
yk ∈ Yk ∀k

larger λg increase robustness to noise but affect the optimality

larger λg corresponds to selecting a model with smaller n
a posteriori, ∥g∥0 informs about the model size

31 / 36
Noise in the real-time measurements

Noise may also be present in the “initial conditions” upast , ypast .

a noisy sequence of Tini measurements may not be compatible with the data

K
X −1
min yk2 + ruk2 +λg ∥g∥1 +λσ ∥σ∥1
u,y,g,σ
k=0
 
  upast
HL (udata )  u 
subject to g= 
HL (ydata ) ypast +σ 
y
uk ∈ Uk ∀k
yk ∈ Yk ∀k

We also need to deal with noise in System Identification!

However, there is increasing (and very recent!) evidence that data-enabled
control deals with noise better than data → sys ID → state-space → MPC.

32 / 36
What is hard to do with data-driven control

State constraint
If a state is known and its value needs to be controlled, then a model-based
approach may work better.

Prior/side information
If some system information is available (range of parameters, nominal values,
structure of the problem) then sysID can typically incorporate this information.

Nonlinearity
The theory we have seen today strongly relies on LTI system theory.
Extensions to nonlinear systems are complicated.

Computational complexity
A data representation is less compact than state-space models (what is a
model, by the way?). This requires memory and computation time.

33 / 36
https://control.ee.ethz.ch/research/theory/data-enabled-predictive-control.html

34 / 36
Surely, nobody would put apply such a shaky data-driven method
on the world’s most complex engineered system (the electric grid),
using the world’s biggest actuators (Gigawatt-sized HVDC links),
and subject to real-time, safety, & stability constraints . . . right?

!"#$%&'()'(%#(*%+,-$'#(.
%
/% 0123% 21)4'5"*% #% 6"$7% 8#6-1$#),"% $"6'"9% -8% 7-1$% :#:"$% ;<=% 9>'?>% /% )",'"6"% ?-1,*% )"% -8%
'4:-$3#(?"%3-%-1$%9-$@%#3%A'3#?>'%B-9"$%C$'*2D%E"%*-%>#6"%%;<=%$"F1'$"%-GH,'("%31('(I%3>#3%%;<=%%
?-4'22'2'-('(I%"(I'(""$%?#(%*-%-(%>'2%-9(%%;<=%%#(%#*#:J6"%#::$-#?>%9-1,*%)"%6"$7%'(3"$"2J(ID
<%8"9%*#72%#Q"$%
% 2"(*'(I%-1$%?-*"
/8% :-22'),"% /% 9-1,*% ,'@"% 3-% 3$7% 3>"% *"?"(3$#,'K"*% !""BL% #::$-#?>% 9'3>% -1$% 4-$"% *"3#',"*% AM!L%
2723"4% 4-*",2% -(% 3>"% '(3"$#$"#% -2?',,#J-(% :$-),"4D% L-1,*% 2-% 2-4"% ?-*"% )"% 4#*"% #6#',#),"%%
;<=%%N%E-1,*%7-1%)"%'(3"$"23"*%'(%9-$@'(I%3-I"3>"$%3-%*-%21?>%#%*"4-(23$#J-(%N%%;<=

%/3%9-$@2O%<%"6"(%
-(%#(%"(J$",7%
*'G"$"(3%4-*",%P%
2-Q9#$"%:,#R-$4

35 / 36
The control engineer flowchart

36 / 36
 This work is licensed under a
Creative Commons Attribution-ShareAlike 4.0 International License

https://bsaver.io/COCO