Inventory Routing

Problems
Martin Savelsbergh
Goals
Introduce Inventory Routing Problems
Introduce Solution Approaches for
Inventory Routing Problems
Introduce Inventory Routing Game
Inventory Routing
Inventory Management
Vehicle Routing
Conventional Inventory
Management
Customer
monitors inventory levels
places orders
Vendor
manufactures/purchases product
assembles order
loads vehicles
routes vehicles
makes deliveries
You call We haul
Problems with Conventional
Inventory Management
Large variation in demands on
production and transportation
facilities
workload balancing
utilization of resources
unnecessary transportation
costs
urgent vs nonurgent orders
setting priorities
Vendor Managed Inventory
Customer
trusts the vendor to manage
the inventory
Vendor
monitors customers inventory
customers call/fax/e-mail
remote telemetry units
set levels to trigger call-in
controls inventory
replenishment & decides
when to deliver
how much to deliver
how to deliver
You rely We supply
Vendor Managed Inventory
VMI transfers inventory management (and
possibly ownership) from the customer to the
supplier
VMI synchronizes the supply chain through the
process of collaborative order fulfillment
Advantages of VMI
Customer
less resources for inventory
management
assurance that product will be
available when required
Vendor
more freedom in when & how to
manufacture product and make deliveries
better coordination of inventory levels at
different customers
better coordination of deliveries to
decrease transportation cost
Inventory Routing
Decide when to deliver to a customer
Decide how much to deliver to a customer
Decide on the delivery routes
Inventory Routing
Inventory Management
Vehicle Routing
Long-Term Problem
Inventory Costs
Transportation Costs
+
Single Customer (Single Link)
Inventory cost
Transportation cost
A
A
A B
Determine shipping
policies that optimize
the trade-off
between:
transp.
cost
inv. cost
Problem Description
A B
Fleet of vehicles:
- transportation capacity
- transportation cost (ABA)
1 = r
c
0 1 2 3 4 5
Problem Description
A B
Volume produced in A per unit time: v
Volume consumed in B per unit time: v
Inventory cost per unit time: h
Goal
A B
Determine shipping policies
that minimize
inventory cost + transportation cost
The Continuous Variant
Single frequency f
Continuous time between shipments t = 1/f
Single vehicle
Optimal solution
vt r
t 0
minhvt +
c
t
t

= min(
p
c
hv
,
r
v
)
Minimum Intershipment Times
A practical constraint:
Minimum intershipment times, e.g., 1 day
ZIO (Zero Inventory Ordering)
FBPS (Frequency Based Periodic Shipping)
Zero Inventory Policy
Minimum inter-shipment time
Single frequency f
Continuous time between shipments t= 1/f
A shipment is performed when the inventory
level of the products is zero
* t * t

=
v
v
h
v c
t , , 1 max min *
Frequency Based Periodic
Shipping Policies
Minimum intershipment time
One or more frequencies
Integer time between consecutive shipments
0 1 2
Single frequency
Double frequency
Best Single Frequency Policy
1/k*: best single frequency
Lemma:

( )

= 1 , max * 1
h
c
v v k k

+ =

vk
k
c
hk z
k k
Best
1
SF
min

0 1 2
Best Double Frequency Policy
1/k
1
*,1/ k
2
*: best frequencies
Lemma:
... k * k * k
... k * k
= <
=
2 2 1
1 1
1

=
<
) , ( min ), ( min min
2 1 1
1
DF
2 2 1 1 1
k k z k z z
DF
k k k
SF
k k
Best

Multiple Customers
0
1
2
3
Inventory cost
Transportation cost
Determine shipping
policies that optimize
the trade-off
between:
Inventory Routing Problem
Deterministic order-up-to level
policy
Each customer defines a minimum and a
maximum level of the inventory
The plant determines the set of delivery time
instants
Every time a customer is visited, the shipping
quantity is such that the maximum level of the
inventory is reached at the customer
Order-Up-To
Inventory
at retailer s
Maximum
level
Time
U
s
Minimum
level
L
s
Starting
level
Problem Formulation
Decisions:
For each customer s:
the set of delivery time instants
For each delivery time instant t:
the route followed by the vehicle
Objective function:
Min Inv. Plant + Inv. Customer + Routing
Key Assumption: Deliveries are instantaneous
Transportation Costs
Inventory Routing Problem
Single Plant
single facility
single product
set of n customers
set of m homogenous vehicles of
capacity Q
Inventory Routing Problem
Each customer:
storage capacity
initial inventory
product usage rate OR
probability distribution of product
usage
Inventory Routing Problem
Objective
Minimize distribution costs without
causing any stock-outs over a finite
horizon OR
Maximize the expected total discounted
value (rewards minus costs) over an
infinite horizon
Inventory Routing Problem
Extensions
Operating modes
Delivery time windows
Delivery times (fixed plus variable part)
Inventory Routing
Even simple situations are non-trivial
There are 14 possible customer combinations
A-B-C-D A-B
A-B-C A-C
A-B-D A-D
A-C-D B-C
A B-D
B C-D
C D
There are an infinite number of possible delivery
volumes
10
100
10
140
C
D
100
100
100
A B
A B C D
Daily Use 1000 3000 2000 1500
Max. Delv. 5000 3000 2000 4000
Truck capacity is 5000
Inventory Routing
10
100
10
140
C
D
100
100
100
A B
A B C D
Daily Use 1000 3000 2000 1500
Max. Delv. 5000 3000 2000 4000
Truck capacity is 5000
The natural solution
Daily schedule
trip 1: deliver 1000 to A & 3000 to B
trip 2: deliver 2000 to C & 1500 to D
420 miles per day
A better solution
Day 1 schedule
trip 1: deliver 3000 to B & 2000 to C
Day 2 schedule
trip 1: deliver 2000 to A & 3000 to B
trip 2: deliver 2000 to C & 3000 to D
380 miles per day
Complexity
Single customer problem?
Two customer problem?
Single Customer Problem
Planning horizon
Usage rate
Initial inventory
Deterministic d-day policy
v
T
(d) = max(0, d
TuI
min(C,Q)
e)c
Storage capacity
Vehicle capacity
Single Customer Problem
Stochastic d-day policy
Cost of filling up every d days over T day period:
d T : v
T
(d) =
P
d1
j=1
p
j
(v
Tj
(d) +S) + (1 p)(v
Td
(d) + c)
d > T : v
T
(d) =
P
T
j=1
p
j
(v
Tj
(d) + S)
Probability that stockout occurs on day j
Stockout cost
Probability that no stockout occurs
Delivery cost
v
T
(d) = (d) + (d)T + f(T, d)
(d) constant, f(T,d) goes to zero exponentially fast as T
(d) =
pS+(1p)c
P
d
j=1
jp
j
Single Customer Problem
An optimal constant replenishment period strategy over a
large T-day planning period will correspond to choosing d* to
minimize (d)
(d) =
pS+(1p)c
P
d
j=1
jp
j
Expected number of days
between deliveries
Expected cost of a delivery
Best d-day policy
Demand: uniformly in [1,20]
Deliver cost: 40
Stockout cost: 50
Optimal policy
Two customer problem
Deterministic d day policy:
Always individually:
Always together:
Sometimes individually, sometimes together?
What if one cannot take a full truckload?
What if the customers are close together?
How much to deliver to each of them on a combined route?
Two Customer Problem
Stochastic policy:
Storage capacity: 20
P[demand = 0] = 0.4, P[demand = 10] = 0.6
Shortage penalty: 1000 customer 1; 1005 customer 2
Vehicle capacity: 10
Individual routes: 120; Combined route: 180
Infinite horizon Markov Decision Process
Minimize expected total discounted cost
Two Customer Problem
Bounds
Customer usage during period: u
i
Customer storage capacity: C
i
Vehicle capacity: Q
Minimize total mileage D
*
subject to
Total volume delivered to customer i u
i
Maximum volume delivered per trip Q
Maximum quantity delivered to customer i min(C
i
, Q)
Bounds
Simple bounds on total mileage
Assume C
i
Q
u
i
I
Q
t
ij
Set of customers
Usage of customer i (period)
Vehicle tank capacity
Travel distance from i to j
Assume direct delivery
Two Customer Analysis
1 2
01
t
12
t
12 01 02
t t t + =
Q C >
1
Q C <
2
0
Plant
Customer
Patterns:
(Q,0)
(0,C
2
)
(Q-C
2
,C
2
)
Two Customer Analysis
Plant
Customer
01
t
12
t
02
t
Q C <
1
Q C >
2
0
1
2
Assume :
Patterns:
(C
1
,0)
(0,Q)
(C
1
,Q-C
1
)
Extra mileage
Improved Bounds
Delivery Patterns
P
j
= (d
j1
,d
j2
, ..., d
jn
) is feasible if

i I
d
ji
Q

0 d
ji
C
i
i I .
and
(P
j
) = {i I : d
ji
> 0} : Customers visited in P
j
c(P
j
) : The cost of delivery pattern P
j
- optimal TSP value
P : Set of all feasible delivery patterns
Pattern Selection LP
D
*
= min
P
j
P
c(P
j
) x
j
s.t.
P
j
P
d
ji
x
j
u
i
, i I
x
j
0
x
j
: How many times
should pattern P
j
be
used
Obstacles
Obstacle I : Infinite number of feasible
delivery patterns
Obstacle II : The calculation of the cost of
each delivery pattern involves the solution of a
traveling salesman problem
Obstacle I
Base Pattern
A feasible delivery pattern P is a base pattern
if at most one customer, say k, in (P) receives a
delivery quantity less than min(C
k
, Q), and, in that
case, the delivery quantity is Q
i (P)\k
C
i
Theorem
The base patterns are sufficient to find an optimal
solution to the Pattern Selection LP
Number of columns of Pattern Selection LP is finite
Obstacle II
Focus on upper and lower bounds on D
*
instead of D
*
itself.
Lower bound (LB
k
) :
If C
i
< Q/k, then assume C
i
= Q/k
LB
1
LB
2
D
*
If min
i I
(C
i
) Q/k, then LB
k
= D
*
|(P)| k for any base pattern P
Upper bound (UB
k
) :
At most k stops in a tour
For values k=3 and
k=4, the TSPs that
have to be solved
involve at most 4 and
5 stops, respectively,
and thus can be solved
relatively easily by
enumeration
UB
1
UB
2
D
*
If Q/min
i I
(C
i
) k, then UB
k
= D
*
|(P)| k for any base pattern P
Dominance
Do we need base pattern P ?
If z c(P),
then the base patterns with
j
> 0 collectively dominate P
Base pattern P can be eliminated from the Pattern Selection LP
Simple Dominance
Consider base pattern P with d
4
<C
4
P
C
1
C
2
C
3
d
4
d
44
= min(Q,C
4
)
C
1
C
2
C
3
0
0
0
0
d
44
Condition I
C
1
C
2
C
3
0
C
1
0
0
d
14
0
C
2
0
d
24
0
C
3
0
d
34
d
i4
= min(Q-C
i
,C
4
)
Condition II
C
1
C
2
C
3
0
C
1
0
d
124
C
2
0
C
3
d
234
C
2 0
d
134
C
1
C
3
d
ij4
= min(Q-C
i
-C
j
,C
4
)
Condition III
Instances n before after
1 136 5,015,046 3,029,980
2 157 7,665,722 4,336,466
3 169 9,086,385 5,420,907
4 147 15,180,701 8,838,137
5 157 14,471,228 8,975,615
6 194 22,575,528 16,640,122
Implementation: Sifting
Approach
Specialized solver for LPs with a large ratio of
number of columns to number of rows
A Partial LP
Subset of full
set of columns
Check reduced
costs for the
remaining columns
Optimal
solution
Instance n #of patterns #of iterations default(sec) sifting(sec)
1 136 3,029,980 5 85.66 77.09
2 157 4,336,466 6 118.37 121.34
3 169 5,420,907 6 155.29 152.56
4 147 8,838,137 5 360.20 240.83
5 157 8,975,615 6 397.33 254.81
6 194 16,640,122 6 675.98 533.56
Computational Experiments
36 plants, ~2000 customers
2500000
2900000
3300000
3700000
4100000
1 2 3 4
k
UB
LB
Inventory Routing
Schedule 1: 420 miles per day
10
100
10
140
3
4
100
100
100
1 2
(1000,3000,0,0),(0,0,2000,1500)
Schedule 2: 380 miles per day
(0,3000,2000,0)
(2000,3000,0,0), (0,0,2000,3000)
Pattern Selection LP with T=1day
Optimal Objective Value : 380
0.5 : (0,3000,2000,0)
0.5 : (2000,3000,0,0)
0.5 : (0,0,2000,3000)
Pattern Selection LP found schedule 2
and it shows no better schedule exists!
Q = 5000
Solution Approaches
Deterministic
Based on average product usage
Stochastic
Based on probability distribution of product
usage
Deterministic Solution
Approach
Two Phase Approach
Phase I: Determine which customers should
receive a delivery on each day of the planning
period and how much
Phase II: Create the precise delivery routes for
each day
Rolling horizon approach
Deterministic Solution
Approach
Two Phase Approach
Phase I: Integer program
Phase II: Insertion heuristic
Integer Program
Lower bound on the total volume that has to be delivered
to customer i by the end of day t:
Upper bound on the total volume that can be delivered
to customer i by the end of day t:
Delivery constraint:
Integer Program
Resource constraints:
Vehicle capacity
Number of vehicles
Integer Program
Vehicle capacity
Number of vehicles
Storage capacity
Integer Program
Improve the efficiency by
Route elimination
Aggregation
Insertion Heuristic
Input for next k days:
List of customers
List of recommended delivery amounts
Output for next k days for each vehicle:
Start time
Sequence of deliveries
Arrival time at each customer
Actual delivery amount at each customer
Key Issue
How to handle variable delivery quantities?
We may be able to increase delivery amounts
We may be able to decrease delivery
amounts
We may be able to postpone deliveries to
another day
Insertion Heuristic
Minimum delivery volume:
Amount suggested by
the integer program
Earliest time a delivery can be made:
Insertion Heuristic
Latest time a delivery can be made:
Maximum delivery volume:
Insertion Heuristic
For each route:
Earliest time a route can start
Latest time a route can start
Earliest time a route can end
Latest time a route can end
Sum of minimum deliveries
Sum of maximum deliveries
Insertion Heuristic
Feasibility check:
Compute minimum delivery volume. Will the
minimum delivery volume fit given the other
deliveries?
Compute earliest and latest delivery can take
place. Is late greater than early?
Compute maximum delivery volume. Is
minimum less than maximum?
Delivery Volume Optimization
Observe:
The amount that can be delivered at a customer
depends on the time at which the delivery starts
The time it takes to make the delivery depends on the
size of the delivery
There is a limit on the elapsed time of a route
Result:
It is nontrivial to determine, given a route, i.e., a
sequence of customer visits, what the maximum
amount of product is that can be delivered on this
route !!
Delivery Volume Optimization
Tank capacity or Truck capacity
Earliest delivery time
Latest delivery time
Usage rate Pump rate
Delivery Volume Optimization
There is a polynomial time algorithm that solves this
problem. The algorithm constructs a series of piecewise
linear graphs (one for each customer on the route)
representing the maximum amount of product that can
be delivered on the remainder of the route as a function
of the start time of the delivery at the customer.
Delivery Volume Optimization
Customer 1
Customer 2
Delivering a little less at
Customer 1 allows a much
larger delivery at Customer 2
Pump time +
Travel time
GRASP
The insertion heuristic is embedded in a
Greedy Randomized Adaptive Search
Heuristic (GRASP)
Stochastic IRP
Determine
inventory
levels
Assign
customers
to vehicles
Deliver at
customer &
drive back
Load to
capacity & drive
to customer
Markov Decision Process
Model
State, x
inventory levels at different customers
Action, a
Which customers to replenish
How much to deliver at each customer
How to combine customers into vehicle routes
Objective
Solving Problems Exactly
Algorithm: Policy Iteration
For each problem
Customer capacity 10 units
Customer demand 1, , 10 w.p. 0.1 each
Vehicle capacity 5 units
Direct delivery only
MDP Model: Issues
Optimality Equation
Computing optimal value function
Computing expected value
Computing optimal action
V

(x) = max
aA(x)
E[g(x, a) +V

(X
t+1
)|X
t
= x, A
t
= a]
Approximation Methods
Idea
Approximate V

with

V
Motivation
Parameterized approximation function
Examples for Basis Functions
Polynomial function
inventory level at customers
second order effects
4450
4650
4850
5050
5250
0 1 2 3 4 5 6 7 8 9 10
Inventory at customer 3 (X3)
V
a
l
u
e
opt, X2 = 0
opt, X2 = 5
opt, X2 = 10
app, X2 = 0
app, X2 = 5
app, X2 = 10
Approximating the Value
Function
Although IRP is not separable, the major
costs (including transportation) are
associated with small groups of
customers (vehicle routes)
We do not know in advance which
groups will be in each vehicle route
We can identify subsets of customers
that can possibly be in the same vehicle
route
MDP for subset of customers
State, (x
i
, k
i
)
inventory at customers vehicles
which could be allocated
Action, a
i
deliveries to customers in the subset
Transition probability
MDPs for small subsets of customers can
be solved optimally in advance
Approximating the Value
Function
In advance, optimally solve problems for subsets of
customers
On each day, partition the customers and vehicles into
subsets by solving a cardinality constrained partitioning
problem
1-customer subsets: nonlinear knapsack problem
2-customer subsets: maximum weight perfect matching
problem
Non-linear Knapsack Problem
V
e
h
i
c
l
e
s
Customers
0,0
2,0
1,1
2,1
1,0
0,1 0,3
2,3
1,3
0,2
1,2
2,2
0
V
1
(x
1
,0)
V
1
(x
1
,0)
V
1
(x
1
,0)
V
1
(
x
1
,
2
)
V
2
(
x
2
,
1
)
V
2
(x
2
,0)
V
2
(x
2
,0)
V
3
(x
3
,0)
V
3
(
x
3
,
2
)
0
0
V
3
(
x
3
,
1
)
V
3
(x
3
,0)
0
0
0
0
0
0
0
0
0
Computing Parameters
Method I
Objective Function
Looks like weighted
least squares
regression problem
Cannot be
computed for
large problems
Computing Parameters
Stochastic Approximation Algorithm

Simulate system under policy

Sample path x
0
, x
1
, , x
t
, ...
Update coefficients
Step size
Temporal difference
Eligibility vector

t
= g(x
t
, (x
t
)) +

V (x
t+1
, r
t
)

V (x
t
, r
t
)
P

t=0

t
= ,
P

t=0
(
t
)
2
<
z
t+1
= z
t
+O
r

V (x
t
, r
t
)
Convergence typically very slow
Computing Parameters
Method II
Value function for policy
Looks like weighted least squares
regression problem
Cannot be
computed for
large
problems

Computing Parameters
Kalman Filter Algorithm
Simulate system under policy
Sample path x
0
, x
1
, , x
t
, ...
Update matrices M
t
(similar to XX) & Y
t
(similar to XY)
r
t
is the solution of M
t
r
t
= Y
t

Convergence significantly faster

Computing Parameters
Stoch App vs. Kalman F + Stoch App
0
2
4
6
8
0:00 0:09 0:18 0:28 0:37 0:46 0:56 1:05 1:14 1:24 1:33
Time (hours)
P
a
r
a
m
e
t
e
r
s
Customer 1
Customer 2
Customer 1
Customer 2
S
t
o
c
h
a
s
t
i
c

A
p
p
Stochastic App
Kalman F
Estimating Expected Value
Multi-dimensional Integral
d = #dimensions = #customers
very hard to compute
Deterministic Methods
MSE = O(n
2 - 2c/d
)
Randomized Methods
MSE = O(1/n)
Deterministic methods are better when 2 - 2c/d < -1
Randomized methods are
better for large d
Choosing the Best Action
Based on sample averages of actions
Question
How large should the sample be so that we are reasonably
sure of choosing the best action?
Nelson and Matejcik (1995)
Sample size to ensure chosen alternative has value within
tolerance of best value with specified probability
Variance reduction methods
Common random numbers
Orthogonal arrays
Variance Reduction Methods
Number of Observations for Choosing Best Action
0
400
800
1200
1600
2000
1 101 201 301 401 501 601 701 801 901
Simulation Steps
N
u
m
b
e
r

o
f

O
b
s
e
r
v
a
t
i
o
n
s
OA
Random
Approximate Policy Iteration
1. Initialization. Simulate initial policy
0
and obtain parameters r

0
2. Use parameters r

t1
for policy
t
and obtain actions using
3. Simulate policy
t
and obtain parameters r

t
4. t t +1; go to Step 2
Performance Comparison
Small Instances
9.4
9.6
9.8
10
10.2
10.4
10.6
10.8
0 1 2 3 4 5 6 7 8 9 10
Inventory at customer 3 (X
3
)
V
a
l
u
e

*
1
0
-
3
Optimal value
Policy 1
Policy 2
Policy 3
CBW policy
Performance Comparison
Large Instances
Price-Direct Replenishment
A control policy based on a simple
economic mechanism for dispatching
The dispatcher receives a transfer price
d
i
V
i
from management for replenishing d
i
units of product at customer i.
The dispatcher is responsible for paying
the distribution costs c
I
, when replenishing
a set of customers I.
Price-Direct Replenishment
Net value for dispatcher
Incremental value for dispatcher
P
iI
V
i
d
i
c
I
d
i
V
i
(c
I{i}
c
I
)
Price-Directed Replenishment
Managements problem: Set V
i
so that the
dispatcher is motivated to minimize the
long-run time average replenishment costs
Price-Direct Replenishment
Management problem (single customer):
Primal Dual
mincZ
dZ = u
0 d min{C, Q}
0 Z
max uV
dV c 0 d min{C, Q}
replenishment
frequency
usage
Price-Direct Replenishment
Management problem (single customer):
Dual
max uV
dV c 0 d min{C, Q}
If V is interpreted as the transfer
price received by the dispatcher
for replenishing one unit, then
this dual program maximizes the
rate at which transfer revenue
accumulates, subject to the
constraint that the total transfer
payment cannot exceed the cost
on any replenishment
Direct Replenishment
Price directed operating policy maximizing
the net value of a replenishment
max
0dmin{C,Q}
{V

d c}
References
P. J aillet, J . Bard, L. Huang, M. Dror (2002). Delivery cost
approximations for inventory routing problems in a rolling
horizon framework. TS 36, 292-300.
A. Campbell, M. Savelsbergh (2004). A decomposition
approach for the inventory routing problem. TS 38, 488-502.
A. Campbell, M. Savelsbergh (2004). Delivery volume
optimization. TS 38, 210-223.
A. Kleywegt, V. Nori, M. Savelsbergh (2002). The stochastic
inventory routing problem with direct deliveries. TS 36, 94-
118.
A. Kleywegt, V. Nori, M. Savelsbergh (2004). Dynamic
programming approximations for a stochastic inventory
routing problem. TS 38, 42-70.
J .-H. Song, M. Savelsbergh (2006). Performance
measurement for inventory routing. TS. To appear.
References
L. Bertazzi, M.G. Speranza (2002). Continuous and discrete
shipping strategies for the single link problem. TS 36, 314-
325.
L. Bertazzi, G. Palletta, M.G. Speranza (2002). Deterministic
order-up-to level policies in an inventory routing problem. TS
36, 119-132.
L. Bertazzi, G. Palletta, M.G. Speranza (2005). Minimizing the
total cost in an integrated vendor-managed inventory system.
J H 11, 393-419.
References
D. Adelman (2003). Price-directed replenishment of subsets:
methodology and its application to inventory routing. MSOM
5, 348-371.
V. Gaur, M. Fisher (2004). A periodic inventory routing
problem at a supermarket chain. OR 52, 813-822.
A. Campbell, J . Hardin (2005). Vehicle minimization for
periodic deliveries. EJ OR 165, 668-684.
W. Bell, L. Dalberto, M. Fisher, A. Greenfield, R, J aikumar, P.
Kedia, R. Mack, P. Prutzman (1983). Improving the
distribution of industrial gases with an online computerized
routing and scheduling optimizer. Interfaces 13, 4-23.
Inventory Routing Game
http://kronos.isye.gatech.edu:8081/IRGame
Login: player1, , player20
Password: player1, , player20
Play Instance 3
Winner gets prize on Friday
Questions?