Integer Programming and Branch & Bound Method

Week 5

Operational Research Methods

Introduction
LP is useful tool for optimizing decisions LP assumes all variables are continuous In many situations, variables has to be integer e.g., People, Machines, Trucks LP with variables restricted to integers is called Integer Linear Programming or Integer Programming

Week 5

Operational Research Methods

Introduction
Example 1
Pottery making: Bowls and Vases each Bowl Clay (pounds) Time (hours) Revenue 2 3 10 each Vase 5 2 8 Available 35 20

How many of each item to make to maximize revenue? This problem can be formulated as an IP model

Week 5

Operational Research Methods

Introduction
Decision variable:
B the number of bowls to be made, V the number of vases to be made.

Objective: to maximize revenue. Constraints:

Clay used cannot be more than available amount, Time used cannot exceed time available, Numbers of bowls and vases must be nonnegative, Numbers of bowls and vases must be integer.
Week 5 Operational Research Methods 4

Introduction
IP Model of the problem: maximize Z = 10B + 8V subject to 2B + 5V 35 3B + 2V 20 B, V 0 and integer.

Week 5

Operational Research Methods

Graphical Solution
Applies to IP with two variables only Similar to LP, but considers integer constraints Steps Draw the feasible region for corresponding LP Feasible solutions are integer points in the region Get optimal LP solution point Compare objective values of integer points close to LP solution to get the optimal integer solution

Week 5

Operational Research Methods

Graphical Solution
Graphical solution to problem in Example 1 feasible region and LP solution V
10 8 Optimal solution for LP 6 4 2 Objective function line 0
Week 5

Optimal solution for IP

10

12

14

16

18

B
7

Operational Research Methods

Graphical Solution
Compare objective values of hopeful integer points.
Integer point B=2, V=6 B=3, V=5 B=4, V=4 Objective value 68 70 72*

The optimal solution is B=4, V=4. Making 4 bowls and 4 vases, max revenue 72.
Week 5 Operational Research Methods 8

Binary (0-1) Integer Programming

Often, integer variables represent logical decisions Two possible values 1 yes, 0 no

IP with all integer variables like this, is called Binary Integer Programming or 0 1 Integer Programming

Week 5

Operational Research Methods

Binary (0-1) Integer Programming

Example 2
Knapsack problem: total weight can carry is 25kg Item no. Value Weight (kg) 1 4 7 2 5 9 3 6 10 4 3 5

Decide which items to carry to maximize total value? This problem can be formulated a binary IP model Other applications have same structure as this problem
Week 5 Operational Research Methods 10

Binary (0-1) Integer Programming

Example 2 Definition of variables
x1 = 1, if item 1 is carried; otherwise, x1 = 0. x2 = 1, if item 2 is carried; otherwise, x2 = 0. x3 = 1, if item 3 is carried; otherwise, x3 = 0. x4 = 1, if item 4 is carried; otherwise, x4 = 0.

The model
maximize 4x1 + 5x2 + 6x3 + 3x4 subject to 7x1 + 9x2 + 10x3 + 5x4 25 x1, x2, x3, x4 are 0 or 1.
Week 5 Operational Research Methods 11

In-class exercise
(a) There are three one-year projects available for investment. You company has 8m and wants to select some of these projects to invest. The investment needed for these projects and their potential profit are listed in the table below. Project Potential profit Investment needed 1 4 6 2 3 3 3 2 4

Formulate an integer programming model to decide which projects to invest in order to maximize the potential profit.

Week 5

Operational Research Methods

12

Binary (0-1) Integer Programming

Example 2 Solution by enumeration
Values of x1, x2, x3, x4 0, 0, 0, 0 0, 0, 0, 1 0, 0, 1, 0 0, 0, 1, 1 0, 1, 0, 0 0, 1, 0, 1 0, 1, 1, 0 0, 1, 1, 1
Week 5

Objective value 0 3 6 9 5 8 11 14*

Values of x1, x2, x3, x4 1, 0, 0, 0 1, 0, 0, 1 1, 0, 1, 0 1, 0, 1, 1 1, 1, 0, 0 1, 1, 0, 1 1, 1, 1, 0 1, 1, 1, 1

Objective value 4 7 10 13 9 12 Infeasible infeasible

13

Operational Research Methods

In-class exercise
(b) Use enumeration to solve the model in (a): Maximize Subject to 4 x1 + 3 x2 + 2 x3 6 x1 + 3 x2 + 4 x3 8 x1, x2, x3 are binary
Objective value 0 2 3 5* Values of x1, x2, x3 1, 0, 0 1, 0, 1 1, 1, 0 1, 1, 1 Objective value 4 infeasible infeasible infeasible

Values of x1, x2, x3 0, 0, 0 0, 0, 1 0, 1, 0 0, 1, 1

Week 5

Operational Research Methods

14

More Examples of IP Problems

Example 3
Choose to rent one of the two machines for producing product A and B.
Machine M Rent per day () Cost to produce one unit of A (/unit) Cost to produce one unit of B (/unit) Efficiency of producing A (units/hour) Efficiency of producing B (units/hour) 200 5 4 20 25 Machine N 100 3 5 20 15

Week 5

Products A and B can be sold at the price of 12 and 10 respectively Work 8 hours per day. which machine to rent in order to maximize the daily profit ?
Operational Research Methods 15

More Examples of IP Problems

Example 3 Definition of variables
x1 be the units of Product A to be produced per day on Machine M; x2 be the units of Product B to be produced per day on Machine M; y1 be the units of Product A to be produced per day on Machine N; y2 be the units of Product B to be produced per day on Machine N; z1 represent whether Machine M is rent; z2 represent whether Machine N is rent;

Week 5

Operational Research Methods

16

More Examples of IP Problems

Example 3 The model
maximize 12x1 + 10x2 + 12y1 + 10y 2 5x1 4x 2 3y1 5y 2 200z1 100z 2 = 7x 1 + 6x 2 + 9y 1 + 5y 2 200z 1 100z 2 z1 + z2 = 1 (1/20)x 1 + (1/25)x 2 8z 1 0 (1/20)y 1 + (1/15)y 2 8z 2 0 x 1, x2, y 1, y 2 0. z 1, z 2 are 0 or 1.
Week 5 Operational Research Methods 17

subject to

More Examples of IP Problems

Example 4
Anna, Ben and Carlos form a group to do a simulation project together.
Data type 1 Anna Ban Carlos 3 4 5 Data type 2 7 5 7 Data type 3 4 4 4 Data type 4 8 9 7 Data type 5 5 5 6

Each type of data needs one person only. To balance the workload, no one should spend less than 5 hours or more than 10 hours Formulate an integer programming model to minimize total data collection time
Operational Research Methods 18

Week 5

More Examples of IP Problems

Example 4 Definition of variables
x 11, x 12, x 13, x 14, x 15 represent whether Anna collects data types 1, 2, 3, 4, 5 respectively; x 21, x 22, x 23, x 24, x 25 represent whether Ben collects data types 1, 2, 3, 4, 5 respectively; x 31, x 32, x 33, x 34, x 35 represent whether Carlos collects data types 1, 2, 3, 4, 5 respectively.

Week 5

Operational Research Methods

19

More Examples of IP Problems

Example 4 The model
minimize 3x11 + 7x12 + 4x13 + 8x14 + 5x15 + 4x21 + 5x22 + 4x23 + 9x24 + 5x25 + 5x31 + 7x32 + 4x33 + 7x34 + 6x35 subject to x11 + x21 + x31 = 1 -- only one person collect data type 1 x12 + x22 + x32 = 1 x13 + x23 + x33 = 1 x14 + x24 + x34 = 1 x15 + x25 + x35 = 1
Operational Research Methods 20

Week 5

More Examples of IP Problems

Example 4 The model (continued)
3x11 + 7x12 + 4x13 + 8x14 + 5x15 5 4x21 + 5x22 + 4x23 + 9x24 + 5x25 5 5x31 + 7x32 + 4x33 + 7x34 + 6x35 5 3x11 + 7x12 + 4x13 + 8x14 + 5x15 10 -- Anna spends at most than 10 hours 4x21 + 5x22 + 4x23 + 9x24 + 5x25 10 5x31 + 7x32 + 4x33 + 7x34 + 6x35 10 x11 , x12 , x13 , x14 , x15, x21, x22, x23, x24, x25, x31, x32, x33, x34, x35 are all 0 or 1.
-- Anna spends at least than 5 hours

Week 5

Operational Research Methods

21

Discussion on Solution of IP Problems

Graphical method
Only for problems with two variables

Enumeration
Only for very small problems

Round-off
Solve as LP, then round the solution to integers. Not practical for most problems: solution obtained far from optimal or even infeasible.

More systematic method: branch and bound Computer packages, e.g., Xpress, Excel solver.
Week 5 Operational Research Methods 22

Branch and Bound Method

The procedure
Step 1 Initialization: Set Z* = -, solve the LP relaxation of the whole MIP problem. If an integer solution is obtained (in this case, we save the solution and let Z* be the objective value of this solution) or the LP relaxation is infeasible, the problem is fathomed. Otherwise, the problem is classified as an active subproblem. The objective value of the LP relaxation solution is an upper bound for this subproblem. Step 2 Optimality test: If no remaining active subproblems, stop. Z* is the optimal value of objective function, and the solution corresponding to Z* is an optimal solution. If Z* is still - when the procedure stops, the IP problem is infeasible. Otherwise, if there are remaining active problems, go step 3.

Week 5

Operational Research Methods

23

Branch and Bound Method

The procedure (continued)
Step 3 Iteration:
a. Branching: select the active subproblem that created most recently (break ties according to which has the larger bound), choose the first (in the natural ordering of variables) integer variable xj with non-integer value xj' in the LP relaxation solution, and branch the subproblem into two new subproblems by adding the constraints xj[xj'] and xj[xj']+1 respectively. b. Bounding: For each new subproblem, get its upper bound by solving its LP relaxation. c. Fathoming: For each new subproblem, test whether it is fathomed by the following criteria: a subproblem is fathomed if (1) its LP relaxation is infeasible, or (2) if the optimal objective value of its LP relaxation is less than or equal to Z*, or (3) the solution of its LP relaxation is an integer solution (in this case if the objective value of this integer solution Z is larger than Z*, then save the solution, let Z* = Z, and do the fathoming test (2) again for all active subproblems using the new Z*). After the test, eliminate all the fathomed subproblems from further consideration. d. go to step 2.

Week 5

Operational Research Methods

24

Branch and Bound Method

Demonstration of B&B method using the example below (an MIP problem) maximize Z = 4x1 - 2x2 + 7x3 - x4 subject to x1 6x1 - 5x2 -x1 + 5x3 10 1 0 + 2x3 - 2x4 3 x1 + x2 - x3

xj 0, for j = 1, 2, 3, 4 xj is integer, for j = 1, 2, 3.

Week 5 Operational Research Methods 25

Branch and Bound Method

0 (5/4, 3/2, 7/4, 0) 141/4

x1 1

x 1 2

Week 5

Operational Research Methods

26

Branch and Bound Method

0 (5/4, 3/2, 7/4, 0) 141/4

x1 1

x1 2

1 (1, 6/5, 9/5, 0) 141/5

2 infeasible fathomed

Week 5

Operational Research Methods

27

Branch and Bound Method

0 (5/4, 3/2, 7/4, 0) 141/4

x1 1 1 (1, 6/5, 9/5, 0) 141/5

x1 2 2 infeasible fathomed

x2 1 3 (5/6, 1, 11/6, 0) 141/6

x2 2 4 (5/6, 2, 11/6, 0)

121/6

Week 5

Operational Research Methods

28

Branch and Bound Method

0 (5/4, 3/2, 7/4, 0) 141/4 x1 1 x1 2

1 (1, 6/5, 9/5, 0) 141/5

2 infeasible fathomed

x2 1

x2 2

3 (5/6, 1, 11/6, 0) 141/6

4 (5/6, 2, 11/6, 0) fathomed 121/6

x1 0

x1 1

5 (0, 0, 2, 1/2) fathomed Z*=131/2

6 infeasible fathomed

Week 5

Operational Research Methods

29

In-class exercise
(c) Solving the LP relaxation of the problem in (a), we got an optimal LP solution: x1 = 5/6, x2 = 1, x3 = 0, Z = 19/3. If you want to solve the IP problem using the Branch and Bound method, the above relaxed solution provides information for the first node of the branch tree. See the diagram below. There are two sub-problems branching out from this first node. Write these two sub-problems (hint: with additional constraints, the value of x1 is fixed in each of the sub-problems. So each of the sub-problem becomes a two-variable problem).
(1) Maximize Z = 3x2 + 2x3 Subject to 3x2 + 4x3 8 x2, x3 are binary and (2) Maximize Z = 4 + 3x2 + 2x3 Subject to 3x2 + 4x3 2 x2, x3 are binary

Week 5

Operational Research Methods

30

In-class exercise
(d) Solve the LP relaxation of these two sub-problems using graphical method or any other method, and continue the branch and bound procedure to solve the original IP problem. Present the complete branch tree, and give the optimal solution.
0 x1=5/6, x2=1, x3=0 Z=19/3

x1 0

x1 1

Week 5

Operational Research Methods

31

In-class exercise
0 x1=5/6, x2=1, x3=0 x1 0 Z=19/3

x1 1

1 x1=0, x2=1, x3=1 fathomed Z*=5

2 x1=1, x2=2/3, x3=0 x2 0 Z=6

x2 1

3 x1=1, x2=0, x3=1/2 fathomed Week 5 Operational Research Methods Z=5

4 infeasible fathomed 32