ĐHBK Tp HCM

BMĐT
GV: Hồ Trung Mỹ

Ch.04
Retiming (Tái định thì)

TLTK:
1. Các slide từ sách của Prof. Parhi
2. Slide của Prof. Viktor Öwall
3. Slide của Prof. Lan-Da Van
1
 Outline
4.1 Introduction
4.2 Definitions and Properties
 Quantitative Description of Retiming
 Properties of Retiming
4.3 Retiming Techniques
 Cutset Retiming and Pipelining
 Retiming for Clock Period Minimization
 Retiming for Register Minimization
4.4 Conclusions

2
 4.1 INTRODUCTION
 Retiming is a transformation technique used to
change the locations of delay elements in a circuit
without affecting the input/output characteristics of
the circuit.
 Retiming is a transformation technique to change the
locations of delay element such that
 The input/output characteristics are kept
 The latency of the system is not changed
 The critical path of the system is reduced
 The number of registers is reduced
 The power consumption is reduced
3
 Example

33% reduction

Tcritical = 3 u.t. Tcritical = 2 u.t.

The filter in Fig. 4.1(a) is described by The filter in Fig. 4.1(b) is described by

4
 Example
 Example 1: Retiming by moving delays from all inputs
to all outputs

D retiming

D D

 Example 2: Retiming by change the locations of delay

elements
D
(2) (4) retiming (2) (4)
A B A B
2D D

 Critical path=6,  Critical path=4

 Loop bound=6/2=3  Loop bound=6/2=3

5
4.2 DEFINITIONS AND PROPERTIES
4.2.1 Quantitative Description of Retiming
 Retiming maps a circuit (graph) G to a retimed circuit (graph) Gr.
 A retiming solution is characterized by a value r(V) for each node V in the
graph G.
 r(V): the number of delays moved from the output edge of the node V
to each of its input edges
 Let w(e) denote the weight of the edge e in the original graph G, and let
wr(e) denote the weight of the edge e in the retimed graph Gr. The weight
of the edge UV in the retimed graph is computed by:
wr(e) = w(e) + r(V) – r(U)
 A retiming solution is feasible if wr(e) ≥ 0 holds for all edges
 The IIR example:

(1) r(1)=0, (1)

1 2D 1 2D
r(2)=1
D D r(3)=0 D
(1) (2) r(4)=0 (1) D (2)
2 3 2 3

(G) D
4 (2) (Gr) 4 (2)
6
 4.2.2 Properties of Retiming
Paths: a sequence of nodes and edges
V0 V1  Vk 1 Vk
e0 e1 ek 2 ek 1

k 1
 Weight of the path p: w( p)   w(e )
i 0
i

k
 Delay of the path p: t ( p )   t (V )
i 0
i

Cycle: also a sequence of nodes and edges

V0 V1  Vk 1 V0
e0 e1 ek 21 ek 1

k 1
 Weight of the path p: w( p)   w(ei )
k i 0
 Delay of the path p: t ( p)   t (Vi )
i 0
 Properties of Retiming
 Property 1: The weight of the retimed path p=V0->V1->…->Vk is given by
wr(p)=w(p)+r(Vk)-r(V0).
k 1 k 1
wr ( p)   wr (ei )   ( w(ei )  r (Vi 1 )  r (Vi ))
i 0 i 0
k 1
 k 1 k 1

  w(ei )    r (Vi 1 )   r (Vi )   w( p)  r (Vk )  r (V0 )
i 0  i 0 i 0 
 Property 2: Retiming does not change the number of delays in a
cycle.
 Property 3: Retiming does not alter the iteration bound in a DFG.
 Property 4: Adding the constant value j to the retiming value of
each node does not change the mapping from G to Gr.

8
 DFG Illustration of the Example

T = max. {(1+2+1)/2, (1+2+1)/3} = 2 T = max. {(1+2+1)/2, (1+2+1)/3} = 2

Critical Path delay = 2+1 = 3 u.t. Critical Path Delay = 1+1 = 2 u.t

9
 4.3 RETIMING TECHNIQUES

 Cutset retiming and pipelining

 Retiming to minimize the clock period and
retiming to minimize the number of registers

10
 4.3.1 Cutset Retiming and Pipelining
Cutset Retiming

 min w(e)  k  min w(e)

G1 
 G2
e
G2 
 G1
e
𝑤 𝑤

11
 Node retiming
 Cutset retiming – If one of the subgraph G2 is a single node
and the other subgraph is the rest of the graph minus the
edges going into and out of the chosen node. We call this
node retiming.
G1
(1) (1)
1 1 2D 1 2D
2D
D D D
D
D (2)
(1) (2) 2 3 (1) 2 3
2 3
G2 D
Node
4 4 retiming 4 (2)
(2)
 Subtracting one delay from each
edge outgoing from the node
 Adding one delay to each edge
incident into the node
12
 Node retiming
 To recap the feasibility:
 Feasibility constraints: for each edge U->V of the retimed
graph, the number of delay elements, i.e. the weight of
the edge must be non-negative, we have
wr (e)  w(e)  r (V )  r (U )  0
r (U )  r (V )  w(e)
where wr(e) and w(e) are the number of delay elements on
the edge U->V after and before retiming, respectively.
 For a circuit graph, there is a retiming design space
defined by the system of inequalities (each edge forms an
inequality).
 This requires solving systems of inequalities.

13
 Pipelining – Feedforward Cutset
 Pipelining – special case of cutset retiming, where
there are not edges in the cutset from the subgraph
G2 to the subgraph G1, termed as feedforward cutset.
In D D D
In D G1

a b c cutset a b c
G2

In D D Retime with k=2

a b c
2D 2D
2D
14
 Example-Lattice Filter
Critical path

D D

Tcritical = 2Tmult+(N+1)Tadd where N is the number of stages

Cutset retiming

New critical path

D

D D

Tcritical = 4Tmult+4Tadd , a constant independent of # of stages, N

15
 N slow-down with retiming
 Cutset retiming is used in combination with slow-down.
 Replace each delay in the graph with N delays to create an N-slow
version of the graph.
 During slow down, the processor clock cycle time remains
unchanged. Only the sampling cycle time increased.
 Perform cutset retiming on the N-slow graph
 N-1 null operations must be interleaved after each useful signal
sample to preserve the functionality of the algorithm.

(1) (1) (1) (1)

1 2 1 2 Tclk=2
D 2-slow 2D Tsample=2*2 = 4
transformation
Tclk=2
Tsample=2 D Retiming of the
2-slow graph
(1) (1)
1 2
D Tclk=1
Tsample=2
16
Another example- A 100-stage lattice filter
Stage 1 Stage 2 Critical path Stage 100
……

D D ……D

Tcritical = 2Tmult+(N+1)Tadd where N is the number of stages

In this case it is 2Tmult+101Tadd
2-slow transformation

Stage 1 Stage 2 Critical path Stage 100

……

2D

2D 2D ……
2D

17
 Another example- A 100-stage lattice filter
Stage 1 Stage 2 Critical path Stage 100
……

2D

2D 2D ……
2D

Cutset retiming Inserting null operation to

retiming preserve behavior

Stage 1 Stage 2 Critical path Stage 100

D ……
D D

2D
……
D D D

Tcritical = 2Tmult+2Tadd

Since the circuit is 2-slow, the sample period is 2*Tcritical 18

4.3.2 Retiming for Clock Period Minimization
 Retiming for clock period minimization is the tool
used to cause a recursive DFG to have a clock period
to equal the iteration bound.
The algorithm for clock period minimization is based
on Floyd-Warshall algorithm which finds the shortest
path between every pair of nodes in a graph.

19
Retiming for Clock Period Minimization (1/2)

20
Retiming for Clock Period Minimization (2/2)

21
Find W(U,V) and D(U,V) by manual calculation
Find W(U,V): the minimum delay out of all paths from U to V
W(1,1) = W(2,2) = W(3,3) = W(4,4) =0
Between 1 and 2, there are two paths
P1: 1 – 3 – 2
P2: 1 – 4 – 2
W(1, 2) = Min [(W (13) + W (32)) , (W (14) + W (42))]
= Min [(1 + 0), (2 + 0)]
= Min (1, 2)
=1
There is only one path from 1 to 3: W (1, 3) = 1
Similarly,
W (1, 4) = 2
W (2, 1) = 1
W (2, 3) = 1 + 1 = 2 (path 2 -1 -3)
W (2, 4) = 1 + 2 = 3 (path 2 -1 -4)
W (3, 1) = 0 + 1 = 1 (path 3 -2 -1)
W (3, 2) = 0
W (3, 4) = 0 + 1 + 2 = 3 (path 3 -2 -1 -4)
W (4, 1) = 0 + 1 = 1 (path 4 - 2 -1)
W (4, 2) = 0
W (4, 3) = 0 + 1 + 1 = 2 (path 4 - 2 -1 -3)
22
Find W(U,V) and D(U,V) by manual calculation
Find D(U,V): the maximum computation time among all paths having
path delay = W (U, V)
D (1, 1) = t1 = 1
D (1, 2) = t1 + t3 + t2 = 1 + 2 + 1 = 4
D (1, 3) = t1 + t3 = 1 + 2 = 3
D(U,V)
D (1, 4) = t1 +t4 = 1 + 2 = 3

D (2, 1) = t2 + t1 = 1 + 1 = 2
D (2, 2) = t2 = 1
D (2, 3) = t2 + t1 + t3 = 1 + 1 + 2 = 4
D (2, 4) = t2 + t1 +t4 = 1 + 1 + 2 = 4

D (3, 1) = t3 + t2 + t1 = 2 + 1 + 1 = 4
D (3, 2) = t3 + t2 = 2 + 1 = 3
D (3, 3) = t3 = 1
D (3, 4) = t3 + t2 + t1 +t4 = 2 + 1 + 1 + 2 = 6

D (4, 1) = t4 + t2 + t1 = 2 + 1 + 1 = 4
D (4, 2) = t4 + t2 = 2 + 1 = 3
D (4, 3) = t4+ t2 + t1 + t3 = 2 + 1 + 1 + 2 = 6
D (4, 4) = t4 = 1 23
Find W(U,V) and D(U,V) by Algorithm

24
 Find W(U,V) and D(U,V) by Algorithm
 Step 3: Find S’(U,V): the shortest distance matrix

 The S(U,V) or r(1) (U, V) matrix can be written as:

S(U,V)
V
(or r(1)U(U, V))
G’

 Apply the Floyd-Warshall algorithm to find the shortest distance matrix

r(2) (U, V)

25
 Find W(U,V) and D(U,V) by Algorithm
 Step 3: Find S’(U,V): the shortest distance matrix
r(3) (U, V)

G’

r(4) (U, V)

r(5) (U, V) = S’ (U, V)

26
 Find W(U,V) and D(U,V) by Algorithm
 Step 4: Find W(U,V) and D(U,V) :
𝑆 ′ 𝑈,𝑉
 If U≠V, then 𝑊 𝑈, 𝑉 = and D(U,V)= M.W(U,V)−S’UV+t(V)
𝑀
 If U=V, then W(U,V)=0 and D(U,V)=t(U).
 𝑀 = 𝑡𝑚𝑎𝑥 ∙ 𝑛, where tmax is the maximum computation time of the
nodes in G and 𝑛 is the number of nodes in G.
𝑫 𝑼, 𝑽
𝑾 𝑼, 𝑽
V
V U
U

Some sample calculations for W(U,V) Some sample calculations for D(U,V)
5 15
𝑊 1,2 = =1 𝑊 1,4 = =2
8 8
7 20
𝑊 1,3 = =1 𝑊 3,4 = =3
8 8 27
Example with c=3

𝒓 𝑼 −𝒓 𝑽 ≤𝒘 𝒆

𝒓 𝑼 − 𝒓 𝑽 ≤ 𝑾 𝑼, 𝑽 − 𝟏
𝑫 𝑼, 𝑽 > 𝒄
28
 Example with c=3

(without changing)

Remark: There are total 5 + 7 = 12 inequalities. There is no overlap

between the two sets. We can solve the inequalities by Bellman-Ford
algorithm but notice that there are no negative entries in these
constraints. So no better retiming is possible.
29
 Example with c=2
 Now following 4 constraints get added

 As the new constraints are more stringent, we will retain

them and ignore the old. The combined list of constraint is:

There are total

12 inequalities.

30
Example with c=2

two edges with

negative weights
31
 Example with c=2
 Constraint Graph

 Let us apply Bellman-Ford algorithm to find the solution (see

on the next 3 slides).

32
 Bellman-Ford algorithm
 Find shortest path from origin = 5 to every other node.
 Initial value of r(1)(V) = (0, 0, 0, 0)
 𝑘 = 2; 𝑉 = 1, 2, 3, 4 and r(1)(V) = (0, 0, 0, 0)  first iteration
 V=1
 Incoming (2 1): r(1) (2) + W(2 1) = 0 + 0 = 0
 Incoming (3 1): r(1) (3) + W(3 1) = 0 + 0 = 0
 Incoming (4 1): r(1) (4) + W(4 1) = 0 + 1 = 1
 No change in r(2) (1) ⇒ r(2) (1) = 0
 V=2
 Incoming (1 2): r(1) (1) + W(1 2) = 0 + 1 = 1
 Incoming (3 2): r(1) (3) + W(3 2) = 0 + 1 = 1
 Incoming (4 2): r(1) (4) + W(4 2) = 0 + 2 = 2
 No change in r(2) (2) ⇒ r(2) (2) = 0
 V=3
 Incoming (1  3): r(1) (1) + W(1 3) = 0 + 0 = 0
 Incoming (2  3): r(1) (2) + W(2 3) = 0 + (-1) = -1
 Incoming (4  3): r(1) (4) + W(4 3) = 0 + 2 = 2
 r(2) (3) is changed from 0 to -1
 V=4
 Incoming (1  4): r(1) (1) + W(1 4) = 0 + 0 = 0
 Incoming (2  4): r(1) (2) + W(2 4) = 0 + (-1) = -1
 Incoming (3  4): r(1) (3) + W(3 4) = 0 + 1 = 1
 r(2) (4) is changed from 0 to -1
So r(2) (V) = (0, 0, -1, -1)
33
 Bellman-Ford algorithm
 𝑘 = 3; 𝑉 = 1, 2, 3, 4 and r(2)(V) = (0, 0, -1, -1)  second iteration
 V=1
 Incoming (2 1): r(2) (2) + W(2 1) = 0 + 0 = 0
 Incoming (3 1): r(2) (3) + W(3 1) = -1 + 0 = -1
 Incoming (4 1): r(2) (4) + W(4 1) = -1 + 1 = 0
 r(3) (1) is changed from 0 to -1
 V=2
 Incoming (1 2): r(2) (1) + W(1 2) = 0 + 1 = 1
 Incoming (3 2): r(2) (3) + W(3 2) = -1 + 1 = 0
 Incoming (4 2): r(2) (4) + W(4 2) = -1 + 2 = 1
 No change in r(3) (2) ⇒ r(3) (2) = 0
 V=3
 Incoming (1  3): r(2) (1) + W(1 3) = 0 + 0 = 0
 Incoming (2  3): r(2) (2) + W(2 3) = 0 + (-1) = -1
 Incoming (4  3): r(2) (4) + W(4 3) = -1 + 2 = 1
 No change in r(3) (3) ⇒ r(3) (3) = -1
 V=4
 Incoming (1  4): r(2) (1) + W(1 4) = 0 + 0 = 0
 Incoming (2  4): r(2) (2) + W(2 4) = 0 + (-1) = -1
 Incoming (3  4): r(2) (3) + W(3 4) = -1 + 1 = 0
 No change in r(3) (4) ⇒ r(3) (4) = -1
So r(3) (V) = (-1, 0, -1, -1)

34
 Bellman-Ford algorithm
 𝑘 = 4; 𝑉 = 1, 2, 3, 4 and r(3)(V) = (-1, 0, -1, -1)  third iteration
 V=1
 Incoming (2 1): r(3) (2) + W(2 1) = 0 + 0 = 0
 Incoming (3 1): r(3) (3) + W(3 1) = -1 + 0 = -1
 Incoming (4 1): r(3) (4) + W(4 1) = -1 + 1 = 0
 No change in r(4) (1) ⇒ r(1) (3) = -1
 V=2
 Incoming (1 2): r(3) (1) + W(1 2) = -1 + 1 = 0
 Incoming (3 2): r(3) (3) + W(3 2) = -1 + 1 = 0
 Incoming (4 2): r(3) (4) + W(4 2) = -1 + 2 = 1
 No change in r(4) (2) ⇒ r(4) (2) = 0
 V=3
 Incoming (1  3): r(3) (1) + W(1 3) = -1 + 0 = -1
 Incoming (2  3): r(3) (2) + W(2 3) = 0 + (-1) = -1
 Incoming (4  3): r(3) (4) + W(4 3) = -1 + 2 = 1
 No change in r(4) (3) ⇒ r(4) (3) = -1
 V=4
 Incoming (1  4): r(3) (1) + W(1 4) = -1 + 0 = 0
 Incoming (2  4): r(3) (2) + W(2 4) = 0 + (-1) = -1
 Incoming (3  4): r(3) (3) + W(3 4) = -1 + 1 = 0
 No change in r(4) (4) ⇒ r(4) (4) = -1
So r(4) (V) = (-1, 0, -1, -1)

35
 Example with c=2
 Final retiming vector is (-1, 0, -1, -1). i.e. nodes 1, 3 and 4
are advanced by 1 clock period.
 For more convenience, we can use the fourth property of
retiming (see on Slide 8) and add 1 to all entries of r(V).

 The retiming vector now becomes (0, 1, 0, 0). i.e. node 2 is

retarded by 1 clock cycle and others are unaffected.

Retiming with Clock

Period optimization

Retiming for node 2

wr (U 

e
V )  w(U 

e
V )  r (V )  r (U )  0
36
 4.3.3 Retiming for Register Minimization
D

D
Delay
reduction
D

(a) Usage: 1 + 3 + 7 = 11 Reg (b) Usage: 1 + 2 + 4 = 7 Reg

Register Sharing  Register reduction through

When a node has multiple fan- node delay transfer from
out with different number of multiple input edges to
delays, the registers can be output edges (e.g. r(v) > 0)
shared so that only the branch  Should be done only when
with max. # of delays will be clock cycle constraint (if
needed. any) is not violated.
37
 Retiming for General DFG
Example:

(see slides 30-36) (Register Sharing)

38
 Other Applications of Retiming
Retiming for Folding (Chapter 6)
Retiming for Power Reduction (Chap. 17)
Retiming for Logic Synthesis (Beyond Scope of
This Class)
Multi-Rate/Multi-Dimensional Retiming
(Denk/Parhi, Trans. VLSI, Dec. 98, Jun.99)

39
 4.4 Conclusion
Reduction of clock period, number of registers,
and power consumption by retiming for
synchronous systems

Shortest path algorithms can be used to obtain a

retiming solution if one exists.

Retiming can also be used as a preprocessing

step for folding and for computation of round-off
noise as discussed in Chap 6 and 11,
respectively.
40
 Appendix
 Bellman-Ford algorithm finds shortest distance
 Floy-Warshall algorithm finds shortest distance
 Solving Systems of Inequalities

41
 Bellman-Ford algorithm finds shortest distance
 This is a single point shortest path algorithm.
 If there are no negative cycles (i.e. cycles having negative
weight or  w(ei)  0) then it finds shortest distance from any
arbitrarily chosen node U (called origin) to each node in the
graph.
 If there are negative cycles, then the algorithm goes in an
infinite loop.
 But after one pass of the algorithm, we can check for
presence of negative cycles.
 We initialize the distance vector r(1) (V) to initial values of the
edge weights from U to other nodes. If edge does not exist,
we will take the weight as 
 In each iteration, we try to find whether there is a shorter way
to reach node i from node U by going via another node W.
 If the edge W  V has negative weight, then it will provide a
shorter way to V from U via W.
42
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
From 2 to V directly
 Some steps are described below

1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 2)
1 2 is chosen as an original node.
First iteration:
2 4  V=1
1 2 There is one incoming edge into node 1, i.e.
4 1 weight 1.
3 New weight of r(2) (1) = r(1) (4) + W (4  1)
= 2+1
2
= 3 < r(1) (1) = 
So we will change r(2) (1) to 3 for next iteration.

43
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).

 Some steps are described below

1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 2)
1
First iteration:
2 4  V=2
1 2 There is one incoming edge into node 2, i.e.
1 2 weight -3.
3 New weight of r(2) (2) = r(1) (1) + W (1  2)
=  + (-3)
2
=  > r(1) (2) = 0
It will retain same value for r(2) (2).

44
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).

 Some steps are described below

1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 2)
1
First iteration:
2 4  V=3
1 2 There is one incoming edge into node 3, i.e.
2 3 weight 1.
3 New weight of r(2) (3) = r(1) (2) + W (2  3)
=0+1
2
=1
It will retain same value for r(2) (3).

45
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
= (, 0, 1, 2)
1 First iteration:
-3 1  V=4
It has 2 incoming edges 3  4, 2 4
2 4 3  4 : r(2)(4) = r(1) (3) + W (3  4)
1 2 =1+2
=3
3 2  4 : r(2)(4) = r(1) (2) + W (2  4)
=0 + 2
2
=2
⟹ r(2)(4) = min {3,2} = 2
Hence there is no change for r(2) (4).
 At the end of the first iteration:
r(2) (V) = ( 3, 0, 1, 2 ) 46
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 3, 0, 1, 2 )
1
-3 1
Second iteration:
2 4  V=1
1 2 There is one incoming edge into node 1, i.e.
4 1 weight 1.
3 New weight of r(3) (1) = r(2) (4) + W (4  1)
= 2+1
2
=3
It will retain same value for r(3) (1).

47
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 3, 0, 1, 2 )
1
-3 1
Second iteration:
2 4  V=2
1 2 There is one incoming edge into node 2, i.e.
1 2 weight -3.
3 New weight of r(3) (2) = r(2) (1) + W (1  2)
= 3 + (-3)
2
=0
It will retain same value for r(3) (2).

48
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 3, 0, 1, 2 )
1
-3 1
Second iteration:
2 4  V=3
1 2 There is one incoming edge into node 3, i.e.
2 3 weight 1.
3 New weight of r(3) (3) = r(2) (2) + W (2  3)
=0+1
2
=1
It will retain same value for r(3) (3).

49
 Bellman-Ford algorithm finds shortest distance
Example 1: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 3, 0, 1, 2 )
1 second iteration:
-3 1  V=4
It has 2 incoming edges 3  4, 2 4
2 4 3  4 : r(3)(4) = r(2) (3) + W (3  4)
1 2 =1+2
=3
3 2  4 : r(3)(4) = r(2) (2) + W (2  4)
=0 + 2
2 =2
We conclude that there ⟹ r(3)(4) = min {3,2} = 2
are no negative cycles Hence there is no change for r(3) (4).
and (3, 0, 1, 2) is a  At the end of the second iteration:
shortest path solution. r(3) (V) = ( 3, 0, 1, 2 ) 50
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
From 2 to V directly
This graph has a  Some steps are described below
negative cycle.
1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 1)
1 2 is chosen as an original node.
First iteration:
2 4  V=1
1 2 There is one incoming edge into node 1, i.e.
4 1 weight 1.
3 New weight of r(2) (1) = r(1) (4) + W (4  1)
= 1+1
1
= 2 < r(1) (1) = 
So we will change r(2) (1) to 2 for next iteration.

51
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).

 Some steps are described below

1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 1)
1
First iteration:
2 4  V=2
1 2 There is one incoming edge into node 2, i.e.
1 2 weight -3.
3 New weight of r(2) (2) = r(1) (1) + W (1  2)
=  + (-3)
1
=  > r(1) (2) = 0
It will retain same value for r(2) (2).

52
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).

 Some steps are described below

1 r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
-3 = (, 0, 1, 1)
1
First iteration:
2 4  V=3
1 2 There is one incoming edge into node 3, i.e.
2 3 weight 1.
3 New weight of r(2) (3) = r(1) (2) + W (2  3)
=0+1
1
=1
Hence there is no change for r(2) (3).

53
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
r(1) (V) = [W(2 1), W(2 2), W(2 3), W(2 4)]
= (, 0, 1, 1)
1 First iteration:
-3 1  V=4
It has 2 incoming edges 3  4, 2 4
2 4 3  4 : r(2)(4) = r(1) (3) + W (3  4)
1 2 =1+2
=3
3 2  4 : r(2)(4) = r(1) (2) + W (2  4)
=0 + 1
1
=1
⟹ r(2)(4) = min {3,1} = 1
Hence there is no change for r(2) (4).
 At the end of the first iteration:
r(2) (V) = ( 2, 0, 1, 1 ) 54
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 2, 0, 1, 1 )
1
-3 1
Second iteration:
2 4  V=1
1 2 There is one incoming edge into node 1, i.e.
4 1 weight 1.
3 New weight of r(3) (1) = r(2) (4) + W (4  1)
= 1+1
1
=2
It will retain same value for r(3) (1).

55
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 2, 0, 1, 1 )
1
-3 1
Second iteration:
2 4  V=2
1 2 There is one incoming edge into node 2, i.e.
1 2 weight -3.
3 New weight of r(3) (2) = r(2) (1) + W (1  2)
= 2 + (-3)
1
= -1
r(3) (2) is changed to -1

56
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the first iteration:
r(2) (V) = ( 2, 0, 1, 1 )
1
-3 1
Second iteration:
2 4  V=3
1 2 There is one incoming edge into node 3, i.e.
2 3 weight 1.
3 New weight of r(3) (3) = r(2) (2) + W (2  3)
=0+1
1
=1
It will retain same value for r(3) (3).

57
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the second iteration:
r(2) (V) = ( 2, 0, 1, 1 )
1 Second iteration:
-3 1  V=4
It has 2 incoming edges 3  4, 2 4
2 4 3  4 : r(3)(4) = r(2) (3) + W (3  4)
1 2 =1+2
=3
3 2  4 : r(3)(4) = r(2) (2) + W (2  4)
=0 + 1
1 =1
⟹ r(3)(4) = min {3,1} = 1
⟹ retain same value for r(3) (4).
 At the end of the second iteration:
r(3) (V) = ( 2, -1, 1, 1 ) 58
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the second iteration:
r(3) (V) = ( 2, -1, 1, 1 )
1
-3 1
Third iteration:
2 4  V=1
1 2 There is one incoming edge into node 1, i.e.
4 1 weight 1.
3 New weight of r(4) (1) = r(3) (4) + W (4  1)
= 1+1
1
=2
It will retain same value for r(4) (1).

59
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the second iteration:
r(3) (V) = ( 2, -1, 1, 1 )
1
-3 1
Third iteration:
2 4  V=2
1 2 There is one incoming edge into node 2, i.e.
1 2 weight -3.
3 New weight of r(4) (2) = r(3) (1) + W (1  2)
= 2 + (-3)
1
= -1
It will retain same value for r(4) (2).

60
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the second iteration:
r(3) (V) = ( 2, -1, 1, 1 )
1
-3 1
Third iteration:
2 4  V=3
1 2 There is one incoming edge into node 3, i.e.
2 3 weight 1.
3 New weight of r(4) (3) = r(3) (2) + W (2  3)
= -1 + 1
1
= 0 < r(3) (3) = 1
r(4) (3) is changed to 0

61
 Bellman-Ford algorithm finds shortest distance
Example 2: In the graph below, let node 2 be origin. Find
the shortest path from 2 to 1, 3, 4 (or V = 1, 3, 4).
 Some steps are described below
At the end of the second iteration:
r(3) (V) = ( 2, -1, 1, 1 )
1 Third iteration:
-3 1  V=4
It has 2 incoming edges 3  4, 2 4
2 4 3  4 : r(4)(4) = r(3) (3) + W (3  4)
1 2 =1+2
=3
3 2  4 : r(4)(4) = r(3) (2) + W (2  4)
= -1 + 1
1 =0
⟹ r(4)(4) = min {3,0} = 0
⟹ r(4) (4) is changed to 0
Exist negative cycle.  At the end of the third iteration:
(2412) r(4) (V) = ( 2, -1, 0, 0 ) 62
 Floy-Warshall algorithm finds shortest distance
 Consider a directed graph having 𝑛 nodes. The distance
between every pair of nodes (𝑖, 𝑗) is represented in a matrix
𝑅, which is of order 𝑛 × 𝑛
 We initialize the matrix such that element (𝑖, 𝑗) represents
distance from node 𝑖 to node 𝑗 if the edge (𝑖  𝑗) exists. It is
 otherwise. This is the first version of the matrix and we will
call this version R(1).
 The algorithm builds successive versions R(2), R(3)…R(n+1)
such that there is improvement in each iteration.

 If any diagonal element (𝑖, 𝑖) is negative in any iteration then

it indicates presence of a negative cycle.

63
Floy-Warshall algorithm finds shortest distance
 The distance from node U to node V (U  V ) is stored
in R(1) (U, V).
 In each iteration we try to find a better alternative by
taking a hop via another node K. We compare direct
distance U  V with one-hop distance (U  K + K 
V).
 If one of these branches is negative, it is possible that
the one-hop path is a better alternative.

64
 Floy-Warshall algorithm finds shortest distance
Example: In the graph below, find the shortest
distance between every pair of nodes by using
Floy-Warshall algorithm.

65
Floy-Warshall algorithm finds shortest distance
R(1)
U V

Let K = 1
 V = 1: W (11) = 
 V = 2: W (12) = -3
• Shortest Distance W (42)
= W (41) + W (12) = 1 – 3 = -2
⟹W (42) is changed from  to -2

 V = 3: W (13) = 
 V = 4: W (14) = 

66
Floy-Warshall algorithm finds shortest distance
R(1)
U V

Let K = 1
 V = 1: W (11) = 
 V = 2: W (12) = -3 R(2)
U V
• Shortest Distance W (42)
= W (41) + W (12) = 1 – 3 = -2
⟹W (42) is changed from  to -2

 V = 3: W (13) = 
 V = 4: W (14) = 

67
Floy-Warshall algorithm finds shortest distance
R(2)
U V

Let K = 2
 V = 1: W (21) = 
 V = 2: W (22) = 
 V = 3: W (23) = 1
• W (13) = W(12) + W(23)
= -3 + 1 = -2

• W (43) = W(42) + W(23)

= -2 + 1 = -1

68
Floy-Warshall algorithm finds shortest distance
R(2)
U V

Let K = 2
 V = 4: W (24) = 2
• W (14) = W(12) + W(24)
= -3 + 2 = -1

• W (44) = W(42) + W(24)

= -2 + 2 = 0

69
Floy-Warshall algorithm finds shortest distance
R(2)
U V

Let K = 2
 V = 4: W (24) = 2 R(3)
U V
• W (14) = W(12) + W(24)
= -3 + 2 = -1

• W (44) = W(42) + W(24)

= -2 + 2 = 0

70
Floy-Warshall algorithm finds shortest distance
R(3)
U V

Let K = 3
W(U⟶3)
 V = 1: W (31) = 
 V = 2: W (32) = 
 V = 3: W (33) = 
 V = 4: W (34) = 2
• W (14) = W(13) + W(34)
= -2 + 2 = 0 (unchanged)
• W (24) = W(23) + W(34)
= 1 + 2 = 3 (unchanged)
• W (44) = W(43) + W(34)
= -1 + 2 = 1 (unchanged) 71
Floy-Warshall algorithm finds shortest distance
R(3)
U V

Let K = 3
unchanged
 V = 1: W (31) =  R(4)
 V = 2: W (32) =  U V
 V = 3: W (33) = 
 V = 4: W (34) = 2
• W (14) = W(13) + W(34)
= -2 + 2 = 0 (unchanged)
• W (24) = W(23) + W(34)
= 1 + 2 = 3 (unchanged)
• W (44) = W(43) + W(34)
= -1 + 2 = 1 (unchanged) 72
Floy-Warshall algorithm finds shortest distance
R(4)
U V

Let K = 4
 V = 1: W (41) = 1
U = 1,2,3,4
• W (11) = W(14) + W(41)
= -1+ 1 = 0 1 -1
• W (21) = W(24) + W(41) K=4
2 V=1
= 2+1=3 2 1
• W (31) = W(34) + W(41) 4 1
= 2+1=3
2
3
• W (41) = W(44) + W(41) 0
= 0 + 1 = 1 (unchanged)
4
73
Floy-Warshall algorithm finds shortest distance
R(4)
U V

Let K = 4
 V = 2: W (42) = 1
U = 1,2,3,4
• W (12) = W(14) + W(42)
= -1+ (-2) = -3 (unchanged) 1 -1
• W (22) = W(24) + W(42) K=4
2 V=2
= 2 + (-2) = 0 2 -2
• W (32) = W(34) + W(42) 4 2
= 2 + (-2) = 0
2
3
• W (42) = W(44) + W(42) 0
= 0 + (-2) = -2 (unchanged)
4
74
Floy-Warshall algorithm finds shortest distance
R(4)
U V

Let K = 4
 V = 3: W (43) = -1
U = 1,2,3,4
• W (13) = W(14) + W(43)
= -1+ (-1) = -2 (unchanged) 1 -1
• W (23) = W(24) + W(43) K=4
2 V=3
= 2 + (-1) = 1 (unchanged) 2 -1
• W (33) = W(34) + W(43) 4 3
= 2 + (-1) = 1
2
3
• W (43) = W(44) + W(43) 0
= 0 + (-1) = -1 (unchanged)
4
75
Floy-Warshall algorithm finds shortest distance
R(4)
U V

Let K = 4
 V = 4: W (44) = 0
U = 1,2,3,4
⟹ unchanged 1 -1
K=4
2 V=4
2 0
4 4
2
3
0
4
76
 Floy-Warshall algorithm finds shortest distance
R(4)
U V

Same result as Bellman-

Ford algorithm on this
graph to find the shortest
Let K = 4 path from node 2 to
other nodes

 V = 4: W (44) = 0
R(5)
⟹ unchanged U V

 This is the shortest distance

matrix. As the diagonal
elements are positive, there
are no negative cycles.
77
 Solving Systems of Inequalities

Given a set of M inequalities and N variables, where

each inequality has the form ri – rj <= k for integer
values of k, can use one of shortest path algorithms to
determine if solution exists and to find one solution

Shortest path algorithms (Appendix A of Parhi book)

 Bellman-Ford
 Floyd-Warshall

78
Solving Systems of Inequalities

79
 Example: M=5, N=4

 Bellman-Ford algorithm finds shortest distance from node N + 1 to every other node.
 Floyd-Warshall algorithm finds shortest distance to every node from every other node..
80
 Bellman-Ford algorithm
 Find shortest path from origin = 5 to every other node.
 Initial value of r(1)(V) = (0, 0, 0, 0)
 𝑘 = 2; 𝑉 = 1, 2, 3, 4 and r(1)(V) = (0, 0, 0, 0)  first iteration
 V=1
 Incoming (2 1): r(1) (2) + W(2 1) = 0 + 0 = 0
 Incoming (5 1): Ignore links from home node
 No change in r(2) (1) ⇒ r(2) (1) = 0
 V=2
 No incoming Edges
 No change in r(2) (2) ⇒ r(2) (2) = 0
 V=3
 Incoming (2  3): r(1) (2) + W(2 3) = 0 + 2 = 2 > 0
 Incoming (1  3): r(1) (1) + W(1 3) = 0 + 5 = 5 > 0
 No Change r(2) (3) ⇒ r(2) (3) = 0
 V=4
 Incoming (1  4): r(1) (1) + W(1 4) = 0 + 4 = 4 > 0
 Incoming (3  4): r(1) (3) + W(3 4) = 0 + (-1) = -1 < 0
 r(2) (4) is changed from 0 to -1
So r(2) (V) = (0, 0, 0, -1)
81
 Bellman-Ford algorithm
 𝑘 = 3; 𝑉 = 1, 2, 3, 4 and r(2)(V) = (0, 0, 0, -1)  second iteration
 V=1
 Incoming (2 1): r(2) (2) + W(2 1) = 0 + 0 = 0
 Incoming (5 1): Ignore links from home node
 No change in r(3) (1) ⇒ r(3) (1) = 0
 V=2
 No incoming Edges
 No change in r(3) (2) ⇒ r(3) (2) = 0
 V=3
 Incoming (2  3): r(2) (2) + W(2 3) = 0 + 2 = 2 > 0
 Incoming (1  3): r(2) (1) + W(1 3) = 0 + 5 = 5 > 0
 No Change in r(3) (3) ⇒ r(3) (3) = 0
 V=4
 Incoming (1  4): r(2) (1) + W(1 4) = 0 + 4 = 4 > 0
 Incoming (3  4): r(2) (3) + W(3 4) = 0 + (-1) = -1
 No Change in r(3) (4) ⇒ r(3) (4) = -1
So r(3) (V) = (0, 0, 0, -1)
 In entire second iteration, there was no change. Hence in third iteration
also, there will be no change and in negative cycle test also, there will
be no change. So the final solution is r(2) (V) = (0, 0, 0, -1).
82
 Example: M=5, N=4

 Bellman-Ford algorithm finds shortest distance from node N + 1 to every other node.
 Floyd-Warshall algorithm finds shortest distance to every node from every other node..
83
 Example 4.3.1 with M=5, N=4 (2/2)

Floyd-Warshall algorithm finds shortest distance to every node from every other node.
84