Solving Linear

Programming Problems
The Simplex Method

Mohammed Brahimi
ENSIA/Intelligent Systems Enginnering

October 12, 2024

Outline
Key Takeaways from Last Lecture

LP Standard Form

A Naive Algorithm

Basic and Nonbasic Variables

Protoype example

The states ofsolutions

Simplex Algorithm

Simplex: Special cases

Conclusion

2/36
Key Takeaways from Last Lecture
• If an optimal solution exists for the LP, then at least one of the corner points is
optimal.

3/36
Key Takeaways from Last Lecture
• If an optimal solution exists for the LP, then at least one of the corner points is
optimal.
• The feasible region for any LP is a convex set.

3/36
LP Standard Form
Maximize z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a21 x2 + · · · + an1 xn = b1 ,
a12 x1 + a22 x2 + · · · + an2 xn = b2 ,
..
.
a1m x1 + a2m x2 + · · · + anm xn = bm ,
xi ≥ 0 (1 ≤ i ≤ n)

But ...
Why this LP Standard Form ?!

4/36
LP Standard Form
Maximize z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a21 x2 + · · · + an1 xn = b1 ,
a12 x1 + a22 x2 + · · · + an2 xn = b2 ,
..
.
a1m x1 + a2m x2 + · · · + anm xn = bm ,
xi ≥ 0 (1 ≤ i ≤ n)

Because ...
To solve a system of equations.

5/36
How to convert to the LP Standard Form
• Introduce non-negative slack and excess variable variables, Si ≥ 0, to transform
the inequality constraints into equality constraints.
∑n ∑n
– i=1
aij xi ≤ bj , add a slack variable Sj ≥ 0 such that ( i=1
aij xi ) + Sj = bj .
∑n ∑n
– i=1
aij xi ≥ bj , add a surplus variable Sj ≥ 0 such that ( i=1
aij xi ) − Sj = bj .

6/36
How to convert to the LP Standard Form
• Introduce non-negative slack and excess variable variables, Si ≥ 0, to transform
the inequality constraints into equality constraints.
∑n ∑n
– i=1
aij xi ≤ bj , add a slack variable Sj ≥ 0 such that ( i=1
aij xi ) + Sj = bj .
∑n ∑n
– i=1
aij xi ≥ bj , add a surplus variable Sj ≥ 0 such that ( i=1
aij xi ) − Sj = bj .

6/36
How to convert to the LP Standard Form
• Introduce non-negative slack and excess variable variables, Si ≥ 0, to transform
the inequality constraints into equality constraints.
∑n ∑n
– i=1
aij xi ≤ bj , add a slack variable Sj ≥ 0 such that ( i=1
aij xi ) + Sj = bj .
∑n ∑n
– i=1
aij xi ≥ bj , add a surplus variable Sj ≥ 0 such that ( i=1
aij xi ) − Sj = bj .

• When the Slack/Surplus/Original variable is null, it means that a constraint is

satisfied with equality ( Binding constraint).

6/36
How to convert to the LP Standard Form
• Introduce non-negative slack and excess variable variables, Si ≥ 0, to transform
the inequality constraints into equality constraints.
∑n ∑n
– i=1
aij xi ≤ bj , add a slack variable Sj ≥ 0 such that ( i=1
aij xi ) + Sj = bj .
∑n ∑n
– i=1
aij xi ≥ bj , add a surplus variable Sj ≥ 0 such that ( i=1
aij xi ) − Sj = bj .

• When the Slack/Surplus/Original variable is null, it means that a constraint is

satisfied with equality ( Binding constraint).

• Introduce two non-negative artificial variables, Si

+
≥ 0 and S−
i ≥ 0, to transform
unrestricted variable xi (no sign restriction).
−
– xi i − Si
= S+

6/36
How to convert to the LP Standard Form
• Introduce non-negative slack and excess variable variables, Si ≥ 0, to transform
the inequality constraints into equality constraints.
∑n ∑n
– i=1
aij xi ≤ bj , add a slack variable Sj ≥ 0 such that ( i=1
aij xi ) + Sj = bj .
∑n ∑n
– i=1
aij xi ≥ bj , add a surplus variable Sj ≥ 0 such that ( i=1
aij xi ) − Sj = bj .

• When the Slack/Surplus/Original variable is null, it means that a constraint is

satisfied with equality ( Binding constraint).

• Introduce two non-negative artificial variables, Si

+
≥ 0 and S−
i ≥ 0, to transform
unrestricted variable xi (no sign restriction).
−
– xi i − Si
= S+

• All the original variables xi and the added variables Si are positive.

6/36
LP Standard Form
Maximize z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a21 x2 + · · · + an1 xn = b1 ,
a12 x1 + a22 x2 + · · · + an2 xn = b2 ,
..
.
a1m x1 + a2m x2 + · · · + anm xn = bm ,
xi ≥ 0 (1 ≤ i ≤ n)

But ...
More variables than equations (m ≪ n).

7/36
LP Standard Form
Maximize z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a21 x2 + · · · + an1 xn = b1 ,
a12 x1 + a22 x2 + · · · + an2 xn = b2 ,
..
.
a1m x1 + a2m x2 + · · · + anm xn = bm ,
xi ≥ 0 (1 ≤ i ≤ n)

But ...
Take m variables and solve the system.

7/36
Solving LP problem: A Naive Algorithm
• Generate all feasible corner points
– Determine all the intersection points between constraints.
– Test whether it is feasible.

• Use the objective function to determine which corner point is the optimal solution.

But ...
What is the number of intersection points for a LP problem with m
constraints and n variables?

8/36
Solving LP problem: A Naive Algorithm
• Generate all feasible corner points
– Determine all the intersection points between constraints.
– Test whether it is feasible.

• Use the objective function to determine which corner point is the optimal solution.

But ...
(n) n!
m
= m!(n−m)!

8/36
Solving LP problem: A Naive Algorithm
• Generate all feasible corner points
– Determine all the intersection points between constraints.
– Test whether it is feasible.

• Use the objective function to determine which corner point is the optimal solution.

But ...
(30)
10 = 30045015 !!!

8/36
Basic and Nonbasic Variables
• A Basic Solution is obtained by setting (n − m) variables to 0 and solving the
remaining system of m variables.
• The remaining variables are the Basic variables, and the removed ones are the
Non-basic variables.
• The non basic variables set to zero represent the fully satisfied constraints.
• A Basic Feasible Solution (BFS) is a basic solution that satisfies all of the
constraints.

Key Idea
A corner point in the feasible region of an LP is a Basic
Feasible Solution (BFS).

9/36
Basic Variables Nonbasic Variables
n
∑
x1 = b1 + a1ixi
i=m+1
...
∑
n

xm = bm + amixi
i=m+1
Basic Variables Nonbasic Variables
n
∑
x1 = b1 + a1ixi
i=m+1
... Assign to 0

∑
n

xm = bm + amixi
i=m+1
 Basic Variables Nonbasic Variables
n
∑
x1 = b1 + a1ixi
i=m+1
Assign to b’s ... Assign to 0

∑
n

xm = bm + amixi
i=m+1

10/36
Example: Basic Feasible solutions (BFS’s)

Maximize x+y
subject to x + 2y ≤6
2x + y ≤ 6

11/36
 Example: Basic Feasible Solutions (BFS’s)

Maximize z =x+y E
subject to x + 2y ≤ 6
2x + y ≤ 6

A D F

12/36
Example: Basic Feasible Solutions (BFS’s)

• Variables {x, y, S1 , S2 }
Maximize z =x+y
subject to x + 2y + S1 = 6
2x + y + S2 = 6
x, y, S1 , S2 ≥ 0

13/36
Example: Basic Feasible Solutions (BFS’s)

• Variables {x, y, S1 , S2 }
Maximize z =x+y
subject to x + 2y + S1 = 6
• Number of variables n =4
2x + y + S2 = 6 • Number of constraints m =2
x, y, S1 , S2 ≥ 0

13/36
Example: Basic Feasible Solutions (BFS’s)

• Variables {x, y, S1 , S2 }
Maximize z =x+y
subject to x + 2y + S1 = 6
• Number of variables n =4
2x + y + S2 = 6 • Number of constraints m =2
x, y, S1 , S2 ≥ 0 (n)
• Number of Basic solutions m
(4)
• 2
= 6 Basic Solutions

13/36
Example: Basic Feasible Solutions (BFS’s)

• Basic solutions
– A, B, C, D, E, F

B
• Feasible Basic Solutions
– A, B, C, D
C

A D F

14/36
Example: Basic Feasible Solutions (BFS’s)

• Basic solutions
– A, B, C, D, E, F

B
• Feasible Basic Solutions
– A, B, C, D
C
• How to find these FBS’s ?!
A D F

14/36
 Example: BFS (S1, S2) - A -

Maximize z =x+y E

subject to S1 = 6 − x − 2y
S2 = 6 − 2x − y
x, y, S1 , S2 ≥ 0
B

A D F

15/36
 Example: BFS (y, S2)- B -

1 1
Maximize z = 3 + x − S1 E
2 2
1 1
subject to y = 3 − x − S1
2 2
3 1
S2 = 3 − x + S1 B
2 2
x, y, S1 , S2 ≥ 0 C

A D F

16/36
 Example: BFS (x, y)- C -

1 1
Maximize z = 4 − S1 − S2 E
3 3
1 2
subject to x = 2 + S1 − S2
3 3
2 1
y = 2 − S1 + S2 B
3 3
x, y, S1 , S2 ≥ 0 C

A D F

17/36
 Example: BFS (x, S1)- D -

1 1
Maximize z = 3 + y − S2 E
2 2
1 1
subject to x = 3 − y − S2
2 2
3 1
S1 = 3 − y + S2 B
2 2
x, y, S1 , S2 ≥ 0 C

A D F

18/36
 Example: Infeasible solution (y, S1)- E -

Maximize z = 6 − x − s2 E

subject to y = 6 − 2x − S2
S1 = −6 + 3x + 2S2
x, y, S1 , S2 ≥ 0
B

A D F

19/36
 Example: Infeasible solution (x, S2)- F -

Maximize z = 6 − y − s1 E

subject to x = 6 − 2y − S1
S2 = −6 − 3y + 2S1
x, y, S1 , S2 ≥ 0
B

A D F

20/36
Example: Summary

Nonbasic (zero) Basic Basic Corner Feasible ? Objective

variables variables solution

x,y S1 , S2 (0,0) A Yes 0

x, S1 y, S2 (0,3) B Yes 3
S1 , S2 x, y (2,2) C Yes 4 (0ptimum)
y, S2 x, S1 (3,0) D Yes 3
x, S2 y, S1 (0,6) E No
6
y, S1 x, S2 (6,0) F No
6

21/36
State 1: Basic Feasible Solution

1 1
Maximize z = 3 + x − s1
2 2
1 1
subject to y = 3 − x − S1
2 2
3 1
S2 = 3 − x + S1
2 2
x, y, S1, S2 ≥ 0

22/36
State 1: Basic Feasible Solution

1 1
Maximize z = 3 + x − s1
2 2
1 1
subject to y = 3 − x − S1
2 2
3 1
S2 = 3 − x + S1
2 2
x, y, S1, S2 ≥ 0

22/36
State 2:Infeasible Solution

Maximize z = 6 − x − s2
subject to y = 6 − 2x − S2
S1 = −6 + 3x + 2S2
x, y, S1, S2 ≥ 0

23/36
State 2:Infeasible Solution

Maximize z = 6 − x − s2
subject to y = 6 − 2x − S2
S1 = -6 + 3x + 2S2
x, y, S1, S2 ≥ 0

23/36
State 3:Optimal Basic Feasible Solution

1 1
Maximize z = 4 − S1 − S2
3 3
1 2
subject to x = 2 + S1 − S2
3 3
2 1
y = 2 − S1 + S2
3 3
x, y, S1, S2 ≥ 0

24/36
State 3:Optimal Basic Feasible Solution

1 1
Maximize z = 4 − S1 − S2
3 3
1 2
subject to x = 2 + S1 − S2
3 3
2 1
y = 2 − S1 + S2
3 3
x, y, S1, S2 ≥ 0

24/36
Simplex Algorithm
• Convert to the standard form
∑
n

Maximize z = b0 + ci xi • While (The BFS is not optimal)

i=m+1
– Move to a new Basic Feasible Solution
∑
n

subject to x1 = b1 + a1i xi
• Output the optimal solution.
i=m+1

..
.
∑
n

xm = bm + ami xi
i=m+1

25/36
Simplex Algorithm
• While (∃Ci > 0)
∑
n

Maximize z = b0 + ci xi – Select a non-basic variable with a positive

coefficient: Entering variable
i=m+1

∑
n
– Introduce this variable in the basis by
subject to x1 = b1 + a1i xi removing a basic variable: Leaving
i=m+1 variable
..
. – Perform Gaussian elimination
∑n

xm = bm + ami xi • Output the optimal solution.

i=m+1

25/36
Simplex Algorithm

1: Initialize a feasible basis and corresponding basic feasible solution.

2: while the objective function has positive coefficients do

3: Choose a non-basic variable with a positive coefficient as the entering variable.

4: Choose a basic variable to leave the basis using the minimum ratio test.

5: Update the basis by replacing the leaving variable with the entering variable.

6: Recalculate the basic feasible solution.

7: end while

8: Output the optimal solution.

26/36
The minimum ratio test
Maximize z =x+y
subject to S1 = 6 − x − 2y
S2 = 6 − 2x − y
x, y, S1 , S2 ≥0

How to choose the leaving variable?

• We must maintain feasibility.

27/36
The minimum ratio test
Maximize z =x+y
subject to S1 = 6 − x − 2y
S2 = 6 − 2x − y
x, y, S1 , S2 ≥0

Chose the leaving variable based on the ration −bai

i
(ai ≤ 0)
• x is the Entering variable (y also can be selected).
• S2 is the Leaving variable.
– Ratio(S1 ) = 61 = 6.
– Ratio(S2 )= 26 = 3.

• We only consider constraints with Negative coefficients (ai ≤ 0) for the entering
variable, so the ratio is necessarily positive.

27/36
The minimum ratio test
Maximize z =x+y
subject to S1 = 6 − x − 2y
S2 = 6 − 2x − y
x, y, S1 , S2 ≥0

Old BFS is
• Basic Variables = {S1 , S2 }
• Non Basic variables = {x, y}
New BFS is
• Basic Variables = {x, S1 }
• Non Basic variables = {y, s2 }

27/36
 Example: BFS (S1, S2) - A -

Maximize z =x+y E

subject to S1 = 6 − x − 2y
S2 = 6 − 2x − y
x, y, S1 , S2 ≥ 0
B

A D F

28/36
 Example: BFS (x, S1)- A → D -

1 1
Maximize z = 3 + y − S2 E
2 2
1 1
subject to x = 3 − y − S2
2 2
3 1
S1 = 3 − y + S2 B
2 2
x, y, S1 , S2 ≥ 0 C

A D F

29/36
 Example: BFS (x, y)- A → D → C -

1 1
Maximize z = 4 − S1 − S2 E
3 3
1 2
subject to x = 2 + S1 − S2
3 3
2 1
y = 2 − S1 + S2 B
3 3
x, y, S1 , S2 ≥ 0 C

A D F

30/36
Simplex Tableau Method

Maximize z = −(−x − y) Base z x y S1 S2 b

subject to S1 + x + 2y = 6 1 -1 -1 0 0 0
S2 + 2x + y = 6
S1 0 1 2 1 0 6
x, y, S1 , S2 ≥0
S2 0 2 1 0 1 6

• The simplex should be updated to consider the sign modifications (optimality test,
how to choose leaving/entering variables, minimum ratio test).

31/36
Simplex Tableau Method

Base z x y S1 S2 b Ratio

1 -1 -1 0 0 0
6
S1 0 1 2 1 0 6 1 =6
6
S2 0 2 1 0 1 6 3 =2

• Entering variable: x
• Leaving variable: S2

32/36
Simplex Tableau Method

Base z x y S1 S2 b Ratio

1 0 − 21 0 1
2 3

S1 0 0 3
2 1 − 12 3 3 × 2
3 =2
x 0 1 1
2 0 1
2 3 3× 2
1 =6

• Entering variable: y
• Leaving variable: S1

33/36
Simplex Tableau Method

Base z x y S1 S2 b Ratio
1 1
1 0 0 3 3 4

y 0 0 1 2
3 − 31 2
x 0 1 0 − 31 2
3 2

• All the objective coefficients are positive.

• Optimal Basic Feasible Solution detected

34/36
Simplex: Special cases
• Unboundedness: occurs when the objective function can be increased indefinitely
without violating any of the constraints.

• Infeasibility: occurs when there is no feasible solution that satisfies all of the
constraints.

• Degeneracy: occurs when one or more basic variables become zero during the
iteration process.

35/36
Conclusion
• Simplex method is an efficient algorithm for finding optimal solutions
to LP problems by navigating through the corner points of the
feasible region.

• It iteratively moves from one Basic Feasible Solution (BFS) to a better

neighborhood BFS until the optimal BFS is reached.

• By detecting the optimal BFS, the simplex method provides the

optimal values of the decision variables and the objective function.

36/36