Practical Guide to the

Simplex Method of Linear Programming

Marcel Oliver

Revised: September 28, 2020

1 The basic steps of the simplex algorithm

Step 1: Write the linear programming problem in standard
form
Linear programming (the name is historical, a more descriptive term would
be linear optimization) refers to the problem of optimizing a linear objective
function of several variables subject to a set of linear equality or inequality
constraints.
Every linear programming problem can be written in the following stan-
dard form.

Minimize ζ = cT x (1a)

subject to

Ax = b , (1b)
x ≥ 0. (1c)

Here x ∈ Rn is a vector of n unknowns, A ∈ M (m × n) with n typically

much larger than m, c ∈ Rn the coefficient vector of the objective function,
and the expression x ≥ 0 signifies xi ≥ 0 for i = 1, . . . , n. For simplicity, we
assume that rank A = m, i.e., that the rows of A are linearly independent.
Turning a problem into standard form involves the following steps.

(i) Turn Maximization into minimization and write inequalities in stan-

dard order.
This step is obvious. Multiply expressions, where appropriate, by −1.

1
 (ii) Introduce slack variables to turn inequality constraints into equality
constraints with nonnegative unknowns.
Any inequality of the form a1 x1 + · · · + an xn ≤ c can be replaced by
a1 x1 + · · · + an xn + s = c with s ≥ 0.
(iii) Replace variables which are not sign-constrained by differences.
Any real number x can be written as the difference of nonnegative
numbers x = u − v with u, v ≥ 0.
Consider the following example.
Maximize z = x1 + 2 x2 + 3 x3 (2a)
subject to
x1 + x2 − x3 = 1 , (2b)
−2 x1 + x2 + 2 x3 ≥ −5 , (2c)
x1 − x2 ≤ 4 , (2d)
x2 + x3 ≤ 5 , (2e)
x1 ≥ 0 , (2f)
x2 ≥ 0 . (2g)
Written in standard form, the problem becomes
minimize ζ = −x1 − 2 x2 − 3 u + 3 v (3a)
subject to
x1 + x2 − u + v = 1 , (3b)
2 x1 − x2 − 2 u + 2 v + s 1 = 5 , (3c)
x1 − x2 + s 2 = 4 , (3d)
x2 + u − v + s 3 = 5 , (3e)
x1 , x2 , u, v, s1 , s2 , s3 ≥ 0 . (3f)

Step 2: Write the coefficients of the problem into a simplex

tableau
The coefficients of the linear system are collected in an augmented matrix
as known from Gaussian elimination for systems of linear equations; the
coefficients of the objective function are written in a separate bottom row
with a zero in the right hand column.
For our example, the initial tableau reads:

2
 x1 x2 u v s1 s2 s3
1 1 −1 1 0 0 0 1
2 −1 −2 2 1 0 0 5
1 −1 0 0 0 1 0 4
0 1 1 −1 0 0 1 5
−1 −2 −3 3 0 0 0 0
In the following steps, the variables will be divided into m basic variables
and n − m non-basic variables. We will act on the tableau by the rules of
Gaussian elimination, where the pivots are always chosen from the columns
corresponding to the basic variables.
Before proceeding, we need to choose an initial set of basic variables
which corresponds to a point in the feasible region of the linear program-
ming problem. Such a choice may be non-obvious, but we shall defer this
discussion for now. In our example, x1 and s1 , . . . , s3 shall be chosen as the
initial basic variables, indicated by gray columns in the tableau above.

Step 3: Gaussian elimination

For a given set of basic variables, we use Gaussian elimination to reduce
the corresponding columns to a permutation of the identity matrix. This
amounts to solving Ax = b in such a way that the values of the nonbasic
variables are zero and the values for the basic variables are explicitly given
by the entries in the right hand column of the fully reduced matrix. In
addition, we eliminate the coefficients of the objective function below each
pivot.
Our initial tableau is thus reduced to
x1 x2 u v s1 s2 s3
1 1 −1 1 0 0 0 1
0 −3 0 0 1 0 0 3
0 −2 1 −1 0 1 0 3
0 1 1 −1 0 0 1 5
0 −1 −4 4 0 0 0 1
The solution expressed by the tableau is only admissible if all basic variables
are non-negative, i.e., if the right hand column of the reduced tableau is free
of negative entries. This is the case in this example. At the initial stage,
however, negative entries may come up; this indicates that different initial
basic variables should have been chosen. At later stages in the process, the

3
selection rules for the basic variables will guarantee that an initially feasible
tableau will remain feasible throughout the process.

Step 4: Choose new basic variables

If, at this stage, the objective function row has at least one negative entry,
the cost can be lowered by making the corresponding variable basic. This
new basic variable is called the entering variable. Correspondingly, one
formerly basic variable has then to become nonbasic, this variable is called
the leaving variable. We use the following standard selection rules.

(i) The entering variable shall correspond to the column which has the
most negative entry in the cost function row. If all cost function
coefficients are non-negative, the cost cannot be lowered and we have
reached an optimum. The algorithm then terminates.

(ii) Once the entering variable is determined, the leaving variable shall be
chosen as follows. Compute for each row the ratio of its right hand
coefficient to the corresponding coefficient in the entering variable col-
umn. Select the row with the smallest finite positive ratio. The leaving
variable is then determined by the column which currently owns the
pivot in this row. If all coefficients in the entering variable column
are non-positive, the cost can be lowered indefinitely, i.e., the linear
programming problem does not have a finite solution. The algorithm
then also terminates.

If entering and leaving variable can be found, go to Step 3 and iterate.

Note that choosing the most negative coefficient in rule (i) is only a
heuristic for choosing a direction of fast decrease of the objective function.
Rule (ii) ensures that the new set of basic variables remains feasible.
Let us see how this applies to our problem. The previous tableau holds
the most negative cost function coefficient in column 3, thus u shall be the
entering variable (marked in boldface). The smallest positive ratio of right
hand column to entering variable column is in row 3, as 13 < 51 . The pivot
in this row points to s2 as the leaving variable. Thus, after going through
the Gaussian elimination once more, we arrive at

4
 x1 x2 u v s1 s2 s3
1 −1 0 0 0 1 0 4
0 −3 0 0 1 0 0 3
0 −2 1 −1 0 1 0 3
0 3 0 0 0 −1 1 2
0 −9 0 0 0 4 0 13
At this point, the new entering variable is x2 corresponding to the only
negative entry in the last row, the leaving variable is s3 . After Gaussian
elimination, we find

x1 x2 u v s1 s2 s3
2 1 14
1 0 0 0 0 3 3 3
0 0 0 0 1 −1 1 5
1 2 13
0 0 1 −1 0 3 3 3
0 1 0 0 0 − 31 1
3
2
3
0 0 0 0 0 1 3 19
Since there is no more negative entry in the last row, the cost cannot be low-
ered by choosing a different set of basic variables; the termination condition
applies.

Step 4: Read off the solution

The solution represented by the final tableau has all nonbasic variables set to
zero, while the values for the basic variables can be can be read off the right
hand column. The bottom right corner gives the negative of the objective
function.
In our example, the solution reads x1 = 14 2 13
3 , x2 = 3 , x3 = u = 3 , s1 = 5,
v = s2 = s3 = 0, which corresponds to ζ = −19, which can be independently
checked by plugging the solution back into the objective function.
As a further check, we note that the solution must satisfy (2b), (2d),
and (2e) with equality and (2c) with a slack of 5. This can also be checked
by direct computation.

2 Initialization
For some problem it is not obvious which set of variables form a feasible
initial set of basic variables. For large problems, a trial-and-error approach

5
 n


is prohibitively expensive due the rapid growth of m , the number of possi-
bilities to choose m basic variables out of a total of n variables, as m and n
become large. This problem can be overcome by adding a set of m artificial
variables which form a trivial set of basic variables and which are penalized
by a large coefficients ω in the objective function. This penalty will cause
the artificial variables to become nonbasic as the algorithm proceeds.
We explain the method by example. For the problem

minimize z = x1 + 2 x2 + 2 x3 (4a)

subject to

x1 + x2 + 2 x3 + x4 = 5 , (4b)
x1 + x2 + x3 − x4 = 5 , (4c)
x1 + 2 x2 + 2 x3 − x4 = 6 , (4d)
x ≥ 0, (4e)

we set up a simplex tableau with three artificial variables which are initially
basic:
a1 a2 a3 x1 x2 x3 x4
1 0 0 1 1 2 1 5
0 1 0 1 1 1 −1 5
0 0 1 1 2 2 −1 6
ω ω ω 1 2 2 0 0
We proceed as before by first eliminating the nonzero entries below the
pivots:

a1 a2 a3 x1 x2 x3 x4
1 0 0 1 1 2 1 5
0 1 0 1 1 1 −1 5
0 0 1 1 2 2 −1 6
0 0 0 1 − 3ω 2 − 4ω 2 − 5ω ω −16ω
Since, for ω large, 2 − 5ω is the most negative coefficient in the objective
function row, x3 will be entering and, since 52 < 62 < 51 , a1 will be leaving.
The Gaussian elimination step then yields

6
 a1 a2 a3 x1 x2 x3 x4
1 1 1 1 5
2 0 0 2 2
1 2 2
1
− 12 1 0 1
2 2
0 − 32 5
2
−1 0 1 0 1 0 −2 1
−1 + 52 ω 0 0 − 12 ω 1 − 32 ω 0 −1 + 72 ω −5 − 72 ω
Now x2 is entering, a3 is leaving, and we obtain

a1 a2 a3 x1 x2 x3 x4
1
1 0 − 12 2
0 1 3
2 2
1
0 1 − 12 2
0 0 − 12 2
−1 0 1 0 1 0 −2 1
ω 0 −1 + 23 ω − 12 ω 0 0 1 + 12 ω −6 − 2ω
The new entering variable is x1 while the criterion for the leaving variable
is tied between a2 and x3 . Since we want the artificial variable to become
nonbasic, we take a2 to be leaving. (Choosing x3 as the leaving variable
would lead us to the same solution, albeit after a few extra steps.) We
obtain
a1 a2 a3 x1 x2 x3 x4
1 −1 0 0 0 1 2 0
0 2 −1 1 0 0 −1 4
−1 0 1 0 1 0 −2 1
ω ω −1 + ω 0 0 0 1 −6
The termination condition is now satisfied, and we see that the solution is
z = 6 with x1 = 4, x2 = 1, x3 = 0, x4 = 0.
We close with two remarks.

• When using a computer to perform the simplex algorithm numerically,

ω should be chosen large (one or two orders of magnitude larger than
any of the other coefficients in the problem) but not too large (to avoid
loss of significant digits in floating point arithmetic).

• If not all artificial variables become nonbasic, ω must be increased. If

this happens for any value of ω, the feasible region is empty.

• In the final tableau, the penalty parameter ω can only appear in arti-
ficial variable columns.

7
3 Duality
The concept of duality is best motivated by an example. Consider the
following transportation problem. Some good is available at location A at
no cost and may be transported to locations B, C, and D according to the
following directed graph:

4 B
1
4
!
A 3
= D
5
2 
*
C

On each of the unidirectional channels, the unit cost of transportation is cj

for j = 1, . . . , 5. At each of the vertices bα units of the good are sold, where
α = B, C, D. How can the transport be done most efficiently?
A first, and arguably most obvious way of quantifying efficiency would be
to state the question as a minimization problem for the total cost of trans-
portation. If xj denotes the amount of good transported through channel j,
we arrive at the following linear programming problem:

minimize c1 x1 + · · · + c5 x5 (5a)

subject to

x1 − x3 − x4 = bB , (5b)
x2 + x3 − x5 = bC , (5c)
x4 + x5 = bD . (5d)

The three equality constraints state that nothing gets lost at nodes B, C,
and D except what is sold.
There is, however, a second, seemingly equivalent way of characterizing
efficiency of transportation. Instead of looking at minimizing the cost of
transportation, we seek to maximize the income from selling the good. Let-
ting yα denote the unit price of the good at node α = A, . . . , D with yA = 0
by assumption, the associated linear programming problem is the following:

maximize yB bB + yC bC + yD bD (6a)

8
subject to

yB − yA ≤ c1 , (6b)
yC − yA ≤ c2 , (6c)
yC − yB ≤ c3 , (6d)
yD − yB ≤ c4 , (6e)
yD − yC ≤ c5 . (6f)

The inequality constraints encode that, in a free market, we can only main-
tain a price difference that does not exceed the cost of transportation. If we
charged a higher price, then “some local guy” would immediately be able to
undercut our price by buying from us at one end of the channel, using the
channel at the same fixed channel cost, then selling at a price lower than
ours at the high-price end of the channel.
Setting
     
x1 yB 1 0 −1 −1 0
x =  ...  , y = yC  , and A = 0 1 1 0 −1 , (7)
     

x5 yD 0 0 0 1 1

we can write (5) as the abstract primal problem

minimize cT x (8a)
subject to Ax = b, x ≥ 0 . (8b)

Likewise, (6) can be written as the dual problem

maximize y T b (9a)
subject to y T A ≤ cT . (9b)

We shall prove in the following that the minimal cost and the maximal
income coincide, i.e., that the two problems are equivalent.
Let us first remark this problem is easily solved without the simplex al-
gorithm: clearly, we should transport all goods sold at a particular location
through the cheapest channel to that location. Thus, we might perform a
simple search for the cheapest channel, something which can be done effi-
ciently by combinatorial algorithms such as Dijkstra’s algorithm [2]. The ad-
vantage of the linear programming perspective is that additional constraints
such as channel capacity limits can be easily added. For the purpose of

9
understanding the relationship between the primal and the dual problem,
and for understanding the significance of the dual formulation, the simple
present setting is entirely adequate.
The unknowns x in the primal formulation of the problem not only iden-
tify the vertex of the feasible region at which the optimum is reached, but
they also act as sensitivity parameters with regard to small changes in the
cost coefficients c. Indeed, when the linear programming problem is nonde-
generate, i.e. has a unique optimal solution, changing the cost coefficients
from c to c + ∆c with |∆c| sufficiently small will not make the optimal
vertex jump to another corner of the feasible region, as the cost depends
continuously on c. Thus, the corresponding change in cost is ∆cT x. If xi
is nonbasic, the cost will not react at all to small changes in ci , whereas if
xi is large, then the cost will be sensitive to changes in ci . This informa-
tion is often important because it gives an indication where to best spend
resources if the parameters of the problem—in the example above, the cost
of transportation—are to be improved.
Likewise, the solution vector y to the dual problem provides the sensi-
tivity of the total income to small changes in b. Here, b is representing the
number of sales at the various vertices of the network; if the channels were
capacity constrained, the channel limits were also represented as compo-
nents of b. Thus, the dual problem is providing the answer to the question
“if I were to invest in raising sales, where should I direct this investment to
achieve the maximum increase in income?”
The following theorems provide a mathematically precise statement on
the equivalence of primal and dual problem.
Theorem 1 (Weak duality). Assume that x is a feasible vector for the
primal problem (8) and y is a feasible vector for the dual problem (9). Then
(i) y T b ≤ cT x;

(ii) if (i) holds with equality, then x and y are optimal for their respective
linear programming problems;

(iii) the primal problem does not have a finite minimum if and only if the
feasible region of the dual problem is empty; vice versa, the dual prob-
lem does not have a finite maximum if and only if the feasible region
of the primal problem is empty.
The proof is simple and shall be left as an exercise.
To proceed, we say that x is a basic feasible solution of Ax = b, x ≥ 0
if it has at most m nonzero components. We say that it is nondegenerate

10
if it has exactly m nonzero components. If, in the course of performing the
simplex algorithm, we hit a degenerate basic feasible solution, it is possible
that the objective function row in the simplex tableau contains negative
coefficients, yet the cost cannot be lowered because the corresponding basic
variable is already zero. This can lead to cycling and thus non-termination
of the algorithm. We shall not consider the degenerate case further.
When x is a nondegenerate solution to the primal problem (8), i.e., x
is nondegenerate basic feasible and also optimal, then we can be assured
that the simplex method terminates with all coefficients in the objective
function row nonnegative. (If they were not, we could immediately perform
at least one more step of the algorithm with strict decrease in the cost.) In
this situation, we can use the simplex algorithm as described to prove the
following stronger form of the duality theorem.

Theorem 2 (Strong duality). The primal problem (8) has an optimal solu-
tion x if and only if the dual problem (9) has an optimal solution y; in this
case y T b = cT x.

Proof. We only give a proof in the case when the solution to the primal
problem is non-degenerate. It is based on a careful examination of the
termination condition of the simplex algorithm. Assume that x solves the
primal problem. Without loss of generality, we can reorder the variables
such that the first m variables are basic, i.e.
!
xB
x= (10)
0

and that the final simplex tableau reads

 !
I R  xB

. (11)
0T r T  −z


The last row represents the objective function coefficients and z denotes
the optimal value of the objective function. We note that the termination
condition of the simplex algorithm reads r ≥ 0. We now partition the initial
matrix A and the coefficients of the objective function c into their basic and
nonbasic components, writing
!
  cB
A= B N and c= . (12)
cN

11
Finally, it can be shown that the elementary row operations used in the
Gaussian elimination steps of the simplex algorithm can be written as mul-
tiplication by a matrix from the left, which we also partition into components
compatible with the block matrix structure of (11), so that the transforma-
tion from the initial to the final tableau can be written as
!  !  !
M v B N b M B + vcTB M N + vcTN  M b
 
= . (13)
uT α cTB cTN  0 uT B + αcTB uT N + αcTN  uT b
 

We now compare the right hand side of (13) with (11) to determine the
coefficients of the left hand matrix. First, we note that in the simplex
algorithm, none of the Gaussian elimination steps on the equality constraints
depend on the objective function coefficients (other than the path taken from
initial to final tableau, which is not at issue here). This immediately implies
that v = 0. Second, we observe that nowhere in the simplex algorithm do
we ever rescale the objective function row. This immediately implies that
α = 1. This leaves us with the following set of matrix equalities:
MB = I , (14a)
M b = xB , (14b)
T
u B + cTB T
=0 , (14c)
uT N + cTN = rT . (14d)
so that M = B −1 and uT = −cTB B −1 . We now claim that
y T = cTB B −1 (15)
solves the dual problem. We compute
 
y T A = cTB B −1 B N
 
= cTB cTB B −1 N
 
= cTB cTN − r T
 
≤ cTB cTN = cT . (16)

This shows that y is feasible for the dual problem. Moreover,

y T b = cTB B −1 b = cTB xB = cT x . (17)
Thus, by weak duality, y is also optimal for the dual problem.
The reverse implication of the theorem follows from the above by noting
that the bi-dual is identical with the primal problem.

12
References
[1] G.B. Dantzig, M.N. Thapa, “Linear Programming,” Springer, New York,
1997.

[2] “Dijkstra’s algorithm.” Wikipedia, The Free Encyclopedia. April 4, 2012,

06:13 UTC.
http://en.wikipedia.org/w/index.php?title=Dijkstra%27s_algorithm&oldid=485462965

[3] D. Gale, Linear Programming and the simplex method, Notices of the
AMS 54 (2007), 364–369.
http://www.ams.org/notices/200703/fea-gale.pdf

[4] P. Pedregal, “Introduction to optimization,” Springer, New York, 2004.

13