Chapter 7: Systolic Architecture

Design

Keshab K. Parhi
• Systolic architectures are designed by using linear
mapping techniques on regular dependence graphs (DG).
• Regular Dependence Graph : The presence of an edge in
a certain direction at any node in the DG represents
presence of an edge in the same direction at all nodes
in the DG.
• DG corresponds to space representation à no time
instance is assigned to any computation ⇒ t=0.
• Systolic architectures have a space-time
representation where each node is mapped to a certain
processing element(PE) and is scheduled at a particular
time instance.
• Systolic design methodology maps an N-dimensional DG
to a lower dimensional systolic architecture.
• Mapping of N-dimensional DG to (N-1) dimensional
systolic array is considered.
Chap. 7 2
• Definitions :
 d1
Ø Projection vector (also called iteration vector), d =  
 d 2
Two nodes that are displaced by d or multiples of d are
executed by the same processor.
ØProcessor space vector, p T
= ( p1 p2 )
Any node with index IT=(i,j) would be executed by proc-
essor; i
pT I = ( p1 p 2 ) 
 j

ØScheduling vector, sT = (s1 s2). Any node with index I would

would be executed at time, sTI.
ØHardware Utilization Efficiency, HUE = 1/|STd|. This is
because two tasks executed by the same processor are
spaced |STd| time units apart.
ØProcessor space vector and projection vector must be
orthogonal to each other ⇒ pTd = 0.
Chap. 7 3
Ø If A and B are mapped to the same processor, then they
cannot be executed at the same time, i.e., STIA ≠ STIB, i.e.,
STd ≠ 0.
Ø Edge mapping : If an edge e exists in the space
representation or DG, then an edge pTe is introduced in the
systolic array with sTe delays.
Ø A DG can be transformed to a space-time representation by
interpreting one of the spatial dimensions as temporal
dimension. For a 2-D DG, the general transformation is
described by i’ = t = 0, j’ = pTI, and t’ = sTI, i.e.,
 i'   i  0 0 1  i 
      
 j ' = T  j  =  p' 0  j 
 t'  t   s' 0  t 
    
j’ ⇒ processor axis
t’ ⇒ scheduling time instance
Chap. 7 4
FIR Filter Design B1(Broadcast Inputs, Move Results,
Weights Stay)
dT = (1 0), pT = (0 1), sT = (1 0)
Ø Any node with index IT = (i , j)
Ø is mapped to processor pTI=j.
Ø is executed at time sTI=i.
Ø Since sTd=1 we have HUE = 1/|sTd| = 1.
Ø Edge mapping : The 3 fundamental edges corresponding to
weight, input, and result can be mapped to corresponding
edges in the systolic array as per the following table:

e pTe sTe
wt(1 0) 0 1
i/p(0 1) 1 0
result(1 –1) -1 1

Chap. 7 5
 Block diagram of B1 design

Low-level implementation of B1 design

Chap. 7 6
 Space-time representation of B1 design

Chap. 7 7
Design B2(Broadcast Inputs, Move Weights, Results Stay)
dT = (1 -1), pT = (1 1), sT = (1 0)
ØAny node with index IT = (i , j)
Øis mapped to processor pTI=i+j.
Øis executed at time sTI=i.
ØSince sTd=1 we have HUE = 1/|sTd| = 1.
ØEdge mapping :
e pTe sTe
wt(1 0) 1 1
i/p(0 1) 1 0
result(1 –1) 0 1

Chap. 7 8
 Block diagram of B2 design

Low-level implementation of B2 design

Chap. 7 9
• Applying space time transformation we get :
j’ = pT(i j)T = i + j
t’ = sT(i j)T = i

Space-time representation of B2 design

Chap. 7 10
Design F(Fan-In Results, Move Inputs, Weights Stay)
dT = (1 0), pT = (0 1), sT = (1 1)
ØSince sTd=1 we have HUE = 1/|sTd| = 1.
ØEdge mapping :
e pTe sTe
wt(1 0) 0 1
i/p(0 1) 1 1
result(1 –1) -1 0

Block diagram of F design

Chap. 7 11
 Low-level implementation of F design

Space-time representation of F design

Chap. 7 12
Design R1(Results Stay, Inputs and Weights Move in
Opposite Direction)
dT = (1 -1), pT = (1 1), sT = (1 -1)
ØSince sTd=2 we have HUE = 1/|sTd| = ½.
ØEdge mapping :
e pTe sTe
wt(1 0) 1 1
i/p(0 -1) -1 1
result(1 –1) 0 2

Chap. 7 Block diagram of R1 design 13

 Low-level implementation of R1 design

Note : R1 can be obtained from B2 by 2-slow transformation

and then retiming after changing the direction of signal x.

Chap. 7 14
Design R2 and Dual R2(Results Stay, Inputs and
Weights Move in Same Direction but at Different Speeds)
dT = (1 -1), pT = (1 1),
R2 : sT = (2 1); Dual R2 : sT = (1 2);
ØSince sTd=1 for both of them we have HUE = 1/|sTd| = 1 for
both.
ØEdge mapping :
R2 Dual R2
e pTe sTe e pTe sTe
wt(1, 0) 1 2 wt(1, 0) 1 1
i/p(0,1) 1 1 i/p(0,1) 1 2
result(1, -1) 0 1 result(-1, 1) 0 1

Note : The result edge in design dual R2has been reversed to

Guarantee sTe ≥ 0. 15
Design W1 (Weights Stay, Inputs and Results Move in
Opposite Directions)
dT = (1 0), pT = (0 1), sT = (2 1)
ØSince sTd=2 for both of them we have HUE = 1/|sTd| = ½.
ØEdge mapping :

e pTe sTe
wt(1 0) 0 2
i/p(0 -1) 1 1
result(1 –1) -1 1

Chap. 7 16
Design W2 and Dual W2(Weights Stay, Inputs and
Results Move in Same Direction but at Different Speeds)
dT = (1 0), pT = (0 1),
W2 : sT = (1 2); Dual W2 : sT = (1 -1);
ØSince sTd=1 for both of them we have HUE = 1/|sTd| = 1 for
both.
ØEdge mapping :
W2 Dual W2
e pTe sTe e pTe sTe
wt(1, 0) 0 1 wt(1, 0) 0 1
i/p(0,1) 1 2 i/p(0,-1) -1 1
result(1, -1) 1 1 result(1, -1) -1 2

Chap. 7 17
• Relating Systolic Designs Using Transformations :
Ø FIR systolic architectures obtained using the
same projection vector and processor vector,
but different scheduling vectors, can be
derived from each other by using
transformations like edge reversal,
associativity, slow-down, retiming and pipelining.
• Example 1 : R1 can be obtained from B2 by slow-
down, edge reversal and retiming.

Chap. 7 18
• Example 2:

Derivation of design F from B1 using cutset retiming

Chap. 7 19
Ø Selection of sT based on scheduling inequalities:
For a dependence relation X àY, where IxT= (ix, jx)T and IyT=
(iy, jy)T are respectively the indices of the nodes X and Y.
The scheduling inequality for this dependence is given by,
Sy ≥ Sx + Tx
where Tx is the computation time of node X. The scheduling
equations can be classified into the following two types :
ØLinear scheduling , where
Sx = sT Ix = (s1 s2)(ix jx )T
Sy = sT Iy = (s1 s2)(iy jy)T
ØAffine Scheduling, where
Sx = sT Ix + γx= (s1 s2)(ix jx )T + γx
Sx = sT Ix + γy = (s1 s2)(ix jx)T + γy
So scheduling equation for affine scheduling is as follows:
sT Ix + γy ≥ sT Ix + γx + Tx
Chap. 7 20
Each edge of a DG leads to an inequality for selection of the
scheduling vectors which consists of 2 steps.
– Capture all fundamental edges. The reduced
dependence graph (RDG) is used to capture the
fundamental edges and the regular iterative algorithm
(RIA) description of the corresponding problem is used
to construct RDGs.
– Construct the scheduling inequalities according to
sT Ix + γy ≥ sT Ix + γx + Tx
and solve them for feasible sT.

Chap. 7 21
• RIA Description : The RIA has two forms
⇒ The RIA is in standard input RIA form if the index of the
inputs are the same for all equations.
⇒ The RIA is in standard output RIA form if all the output
indices are the same.
• For the FIR filtering example we have,
W(i+1, j) = W(i, j)
X(i, j+1) = X(i, j)
Y(i+1, j-1) = Y(i, j) + W(i+1, j-1)X(i+1, j-1)
The FIR filtering problem cannot be expressed in standard
input RIA form. Expressing it in standard output RIA form
we get,
W(i, j) = W(i-1, j)
X(i, j) = X(i, j-1)
Y(i, j) = Y(i-1, j+1) + W(i, j)X(i, j)

Chap. 7 22
• The reduced DG for FIR filtering is shown below.

Example :
Tmult = 5, Tadd = 2, Tcom = 1
Applying the scheduling equations to the five edges of the
above figure we get ;
W-->Y : e = (0 0)T , γx - γw ≥ 0
X -->X : e = (0 1)T , s2 + γx - γx ≥ 1
W-->W: e = (1 0)T , s1 + γw - γw ≥ 1
X -->Y : e = (0 0)T , γy - γx ≥ 0
Y --> Y: e = (1 -1)T , s1 - s2 + γy - γy ≥ 5 + 2 + 1
For linear scheduling γx =γy = γw = 0. Solving we get, s1 ≥ 1,
s2 ≥ 1 and s1 - s2 ≥ 8.
Chap. 7 23
• Taking sT = (9 1), d = (1 -1) such that sTd ≠ 0 and pT = (1,1)
such that pTd = 0 we get HUE = 1/8. The edge mapping is as
follows :
e pTe sTe
wt(1 0) 1 9
i/p(0 1) 1 1
result(1 –1) 0 8

Systolic architecture for the example

Chap. 7 24
 Matrix-Matrix multiplication and 2-D Systolic Array Design

C11 = a11b11 + a12 b21

C12 = a11b12 + a12 b22
C21 = a21b11 + a22 b21
C22 = a21b12 + a22 b22

The iteration in standard output RIA form is as follows :

a(i,j,k) = a(i,j-1,k)
b(i,j,k) = b(i-1,j,k)
c(i,j,k) = c(i,j,k-1) + a(i,j,k) b(i,j,k)
Chap. 7 25
• Applying scheduling inequality with
Tmult -add = 1, and Tcom = 0 we get
s2 ≥ 0, s1 ≥ 0, s3 ≥ 1, γc - γa ≥ 0
and γc - γb ≥ 0. Take γa =γb = γc = 0
for linear scheduling.
• Solution 1 :
sT = (1,1,1), dT = (0,0,1), p1 = (1,0,0),
p2 = (0,1,0), PT = (p1 p2)T

Chap. 7 26
• Solution 2 :
sT = (1,1,1), dT = (1,1,-1), p1 = (1,0,1),
p2 = (0,1,1), PT = (p1 p2)T

Sol. 1 Sol. 2
e pTe sTe e pTe sTe
a(0, 1, 0) (0, 1) 1 a(0, 1, 0) (0, 1) 1
b(1, 0, 0) (1, 0) 1 b(1, 0, 0) (1, 0) 1
C(0, 0, 1) (0, 0) 1 C(0, 0, 1) (1, 1) 1

Chap. 7 27