?

LINEAR PROGRAMMING
Subject: LINEAR PROGRAMMING Credits: 4

SYLLABUS

Basic Mathematics and Optimization

Optimization in more than two variables, Optimization in two variables, Inequalities and Convex Sets, Basic
Algebra

Simplex Method and Duality

Duality Theorems, Primal and Dual, Simplex Method, Standard Form and Solutions

Basic Mathematics and Optimization

The Assignment Problem, Computational Method for the Transportation Problem, An Initial Basic Feasible ,
Solution of the Transportation Problem, The Transportation Problem.

Basic Mathematics and Optimization

Games and Linear Programming, Graphical Method and Dominance, Games with Mixed Strategies, Games
with Pure Strategies

Suggested Readings:

1. S. S. Rao; Optimization; Wiley Eastern India

2. H. A. Taha; Operation Research; Macmillan India
3. G. Hadley; Linear Programming; Addison-Wesley
4. E.K.P. Chong and S.H. Zak; Optimization; Wiley Interscience
 Linear Programming
Optimization
In this section we are going to look at optimization problems. In optimization problems we are
looking for the largest value or the smallest value that a function can take. We saw how to solve
one kind of optimization problem in the Absolute Extrema section where we found the largest
and smallest value that a function would take on an interval.

In this section we are going to look at another type of optimization problem. Here we will be
looking for the largest or smallest value of a function subject to some kind of constraint. The
constraint will be some condition (that can usually be described by some equation) that must
absolutely, positively be true no matter what our solution is. On occasion, the constraint will not
be easily described by an equation, but in these problems it will be easy to deal with as we‘ll see.

This section is generally one of the more difficult for students taking a Calculus course. One of
the main reasons for this is that a subtle change of wording can completely change the problem.
There is also the problem of identifying the quantity that we‘ll be optimizing and the quantity
that is the constraint and writing down equations for each.

The first step in all of these problems should be to very carefully read the problem. Once you‘ve
done that the next step is to identify the quantity to be optimized and the constraint.

In identifying the constraint remember that the constraint is the quantity that must true regardless
of the solution. In almost every one of the problems we‘ll be looking at here one quantity will be
clearly indicated as having a fixed value and so must be the constraint. Once you‘ve got that
identified the quantity to be optimized should be fairly simple to get. It is however easy to
confuse the two if you just skim the problem so make sure you carefully read the problem first!

Let‘s start the section off with a simple problem to illustrate the kinds of issues we will be
dealing with here.

Example 1 We need to enclose a field with a fence. We have 500 feet of fencing material and a
building is on one side of the field and so won‘t need any fencing. Determine the dimensions of
the field that will enclose the largest area.

Solution
In all of these problems we will have two functions. The first is the function that we are actually
trying to optimize and the second will be the constraint. Sketching the situation will often help
us to arrive at these equations so let‘s do that.

In this problem we want to maximize the area of a field and we know that will use 500 ft of
fencing material. So, the area will be the function we are trying to optimize and the amount of
fencing is the constraint. The two equations for these are,

Okay, we know how to find the largest or smallest value of a function provided it‘s only got a
single variable. The area function (as well as the constraint) has two variables in it and so what
we know about finding absolute extrema won‘t work. However, if we solve the constraint for
one of the two variables we can substitute this into the area and we will then have a function of a
single variable.

So, let‘s solve the constraint for x. Note that we could have just as easily solved for y but that
would have led to fractions and so, in this case, solving for x will probably be best.

Substituting this into the area function gives a function of y.

Now we want to find the largest value this will have on the interval [0,250]. Note that the
interval corresponds to taking (i.e. no sides to the fence) and (i.e. only two sides
and no width, also if there are two sides each must be 250 ft to use the whole 500ft).

Note that the endpoints of the interval won‘t make any sense from a physical standpoint if we
actually want to enclose some area because they would both give zero area. They do, however,
give us a set of limits on y and so the Extreme Value Theorem tells us that we will have a
maximum value of the area somewhere between the two endpoints. Having these limits will also
mean that we can use the process we discussed in the Finding Absolute Extrema section earlier
in the chapter to find the maximum value of the area.

So, recall that the maximum value of a continuous function (which we‘ve got here) on a closed
interval (which we also have here) will occur at critical points and/or end points. As we‘ve
already pointed out the end points in this case will give zero area and so don‘t make any sense.
That means our only option will be the critical points.

So let‘s get the derivative and find the critical points.

Setting this equal to zero and solving gives a lone critical point of . Plugging this into
the area gives an area of 31250 ft2. So according to the method from Absolute Extrema section
this must be the largest possible area, since the area at either endpoint is zero.

Finally, let‘s not forget to get the value of x and then we‘ll have the dimensions since this is what
the problem statement asked for. We can get the x by plugging in our y into the constraint.
 The dimensions of the field that will give the largest area, subject to the fact that we used
exactly 500 ft of fencing material, are 250 x 125.

Don‘t forget to actually read the problem and give the answer that was asked for. These types of
problems can take a fair amount of time/effort to solve and it‘s not hard to sometimes forget
what the problem was actually asking for.

In the previous problem we used the method from the Finding Absolute Extrema section to find
the maximum value of the function we wanted to optimize. However, as we‘ll see in later
examples it will not always be easy to find endpoints. Also, even if we can find the endpoints we
will see that sometimes dealing with the endpoints may not be easy either. Not only that, but this
method requires that the function we‘re optimizing be continuous on the interval we‘re looking
at, including the endpoints, and that may not always be the case.

So, before proceeding with any more examples let‘s spend a little time discussing some methods
for determining if our solution is in fact the absolute minimum/maximum value that we‘re
looking for. In some examples all of these will work while in others one or more won‘t be all
that useful. However, we will always need to use some method for making sure that our answer
is in fact that optimal value that we‘re after.

Method 1 : Use the method used in Finding Absolute Extrema.

This is the method used in the first example above. Recall that in order to use this method the
range of possible optimal values, let‘s call it I, must have finite endpoints. Also, the function
we‘re optimizing (once it‘s down to a single variable) must be continuous on I, including the
endpoints. If these conditions are met then we know that the optimal value, either the maximum
or minimum depending on the problem, will occur at either the endpoints of the range or at a
critical point that is inside the range of possible solutions.

There are two main issues that will often prevent this method from being used however. First,
not every problem will actually have a range of possible solutions that have finite endpoints at
both ends. We‘ll see at least one example of this as we work through the remaining examples.
Also, many of the functions we‘ll be optimizing will not be continuous once we reduce them
down to a single variable and this will prevent us from using this method.

Method 2 : Use a variant of the First Derivative Test.

In this method we also will need a range of possible optimal values, I. However, in this case,
unlike the previous method the endpoints do not need to be finite. Also, we will need to require
that the function be continuous on the interior I and we will only need the function to be
continuous at the end points if the endpoint is finite and the function actually exists at the
endpoint. We‘ll see several problems where the function we‘re optimizing doesn‘t actually exist
at one of the endpoints. This will not prevent this method from being used.
Let‘s suppose that is a critical point of the function we‘re trying to optimize, . We
already know from the First Derivative Test that if \ immediately to the left of
(i.e. the function is increasing immediately to the left) and if immediately to the
right of (i.e. the function is decreasing immediately to the right) then will be a
relative maximum for .

Now, this does not mean that the absolute maximum of will occur at . However,
suppose that we knew a little bit more information. Suppose that in fact we knew that
for all x in I such that . Likewise, suppose that we knew that for
all x in I such that . In this case we know that to the left of , provided we stay in I of
course, the function is always increasing and to the right of , again staying in I, we are
always decreasing. In this case we can say that the absolute maximum of in I will occur
at .

Similarly, if we know that to the left of the function is always decreasing and to the right
of the function is always increasing then the absolute minimum of in I will occur
at .

Before we give a summary of this method let‘s discuss the continuity requirement a little.
Nowhere in the above discussion did the continuity requirement apparently come into play. We
require that the function we‘re optimizing to be continuous inI to prevent the following situation.

In this case, a relative maximum of the function clearly occurs at . Also, the function is
always decreasing to the right and is always increasing to the left. However, because of the
discontinuity at , we can clearly see that and so the absolute maximum of the
function does not occur at . Had the discontinuity at not been there this would not
have happened and the absolute maximum would have occurred at .

Here is a summary of this method.

First Derivative Test for Absolute Extrema

Let I be the interval of all possible optimal values of and further suppose that is
continuous on I , except possibly at the endpoints. Finally suppose that is a critical point
of and that c is in the interval I. If we restrictx to values from I (i.e. we only consider
possible optimal values of the function) then,

1. If for all and if for all then will be the

absolute maximum value of on the interval I.
2. If for all and if for all then will be the absolute
minimum value of on the interval I.

Method 3 : Use the second derivative.

There are actually two ways to use the second derivative to help us identify the optimal value of
a function and both use theSecond Derivative Test to one extent or another.

The first way to use the second derivative doesn‘t actually help us to identify the optimal value.
What it does do is allow us to potentially exclude values and knowing this can simplify our work
somewhat and so is not a bad thing to do.

Suppose that we are looking for the absolute maximum of a function and after finding the critical
points we find that we have multiple critical points. Let‘s also suppose that we run all of them
through the second derivative test and determine that some of them are in fact relative minimums
of the function. Since we are after the absolute maximum we know that a maximum (of any
kind) can‘t occur at relative minimums and so we immediately know that we can exclude these
points from further consideration. We could do a similar check if we were looking for the
absolute minimum. Doing this may not seem like all that great of a thing to do, but it can, on
occasion, lead to a nice reduction in the amount of work that we need to do in later steps.

The second way of using the second derivative to identify the optimal value of a function is in
fact very similar to the second method above. In fact we will have the same requirements for
this method as we did in that method. We need an interval of possible optimal values, I and the
endpoint(s) may or may not be finite. We‘ll also need to require that the function, be
continuous everywhere in I except possibly at the endpoints as above.

Now, suppose that is a critical point and that . The second derivative test tells us
that must be a relative minimum of the function. Suppose however that we also knew
that for all x in I. In this case we would know that the function was concave up in all
of I and that would in turn mean that the absolute minimum of in I would in fact have to be
at .
Likewise i is a critical point and for all x in I then we would know that the
function was concave down inI and that the absolute maximum of in I would have to be
at .

Here is a summary of this method.

Second Derivative Test for Absolute Extrema

Let I be the range of all possible optimal values of and further suppose that is
continuous on I , except possibly at the endpoints. Finally suppose that is a critical point
of and that c is in the interval I. Then,

1. If for all x in I then will be the absolute minimum value of on

the interval I.
2. If for all x in I then will be the absolute maximum value of on the
interval I.

Before proceeding with some more examples we need to once again acknowledge that not every
method discussed above will work for every problem and that, in some problems, more than one
method will work. There are also problems where we may need to use a combination of these
methods to identify the optimal value. Each problem will be different and we‘ll need to see what
we‘ve got once we get the critical points before we decide which method might be best to use.

Okay, let‘s work some more examples.

Example 2 We want to construct a box whose base length is 3 times the base width. The
material used to build the top and bottom cost $10/ft2 and the material used to build the sides cost
$6/ft2. If the box must have a volume of 50ft3determine the dimensions that will minimize the
cost to build the box.

Solution
First, a quick figure (probably not to scale…).

We want to minimize the cost of the materials subject to the constraint that the volume must be
50ft3. Note as well that the cost for each side is just the area of that side times the appropriate
cost.

The two functions we‘ll be working with here this time are,

As with the first example, we will solve the constraint for one of the variables and plug this into
the cost. It will definitely be easier to solve the constraint for h so let‘s do that.

Plugging this into the cost gives,

Now, let‘s get the first and second (we‘ll be needing this later…) derivatives,

So, it looks like we‘ve got two critical points here. The first is obvious, , and it‘s also just
as obvious that this will not be the correct value. We are building a box here and w is the box‘s
width and so since it makes no sense to talk about a box with zero width we will ignore this
critical point. This does not mean however that you should just get into the habit of ignoring
zero when it occurs. There are other types of problems where it will be a valid point that we will
need to consider.

The next critical point will come from determining where the numerator is zero.

So, once we throw out , we‘ve got a single critical point and we now have to verify that this
is in fact the value that will give the absolute minimum cost.

In this case we can‘t use Method 1 from above. First, the function is not continuous at one of the
endpoints, , of our interval of possible values. Secondly, there is no theoretical upper limit
to the width that will give a box with volume of 50 ft3. If w is very large then we would just
need to make h very small.

The second method listed above would work here, but that‘s going to involve some calculations,
not difficult calculations, but more work nonetheless.

The third method however, will work quickly and simply here. First, we know that whatever the
value of w that we get it will have to be positive and we can see second derivative above that
provided we will have and so in the interval of possible optimal values the cost
function will always be concave up and so must give the absolute minimum cost.

All we need to do now is to find the remaining dimensions.

Also, even though it was not asked for, the minimum cost is : .

Example 3 We want to construct a box with a square base and we only have 10 m2 of material
to use in construction of the box. Assuming that all the material is used in the construction
process determine the maximum volume that the box can have.

Solution
This example is in many ways the exact opposite of the previous example. In this case we want
to optimize the volume and the constraint this time is the amount of material used. We don‘t
have a cost here, but if you think about it the cost is nothing more than the amount of material
used times a cost and so the amount of material and cost are pretty much tied together. If you
can do one you can do the other as well. Note as well that the amount of material used is really
just the surface area of the box.

As always, let‘s start off with a quick sketch of the box.

Now, as mentioned above we want to maximize the volume and the amount of material is the
constraint so here are the equations we‘ll need.

We‘ll solve the constraint for h and plug this into the equation for the volume.

Here are the first and second derivatives of the volume function.

Note as well here that provided , which we know from a physical standpoint will be true,
then the volume function will be concave down and so if we get a single critical point then we
know that it will have to be the value that gives the absolute maximum.

Setting the first derivative equal to zero and solving gives us the two critical points,

In this case we can exclude the negative critical point since we are dealing with a length of a box
and we know that these must be positive. Do not however get into the habit of just excluding
any negative critical point. There are problems where negative critical points are perfectly valid
possible solutions.

Now, as noted above we got a single critical point, 1.2910, and so this must be the value that
gives the maximum volume and since the maximum volume is all that was asked for in the
problem statement the answer is then : .

Note that we could also have noted here that if then and likewise
if then and so if we are to the left of the critical point the volume is always
increasing and if we are to the right of the critical point the volume is always decreasing and so
by the Method 2 above we can also see that the single critical point must give the absolute
maximum of the volume.

Finally, even though these weren‘t asked for here are the dimension of the box that gives the
maximum volume.

So, it looks like in this case we actually have a perfect cube.

In the last two examples we‘ve seen that many of these optimization problems can be done in
both directions so to speak. In both examples we have essentially the same two equations:
volume and surface area. However, in Example 2 the volume was the constraint and the cost
(which is directly related to the surface area) was the function we were trying to optimize. In
Example 3, on the other hand, we were trying to optimize the volume and the surface area was
the constraint.

It is important to not get so locked into one way of doing these problems that we can‘t do it in
the opposite direction as needed as well. This is one of the more common mistakes that students
make with these kinds of problems. They see one problem and then try to make every other
problem that seems to be the same conform to that one solution even if the problem needs to be
worked differently. Keep an open mind with these problems and make sure that you understand
what is being optimized and what the constraint is before you jump into the solution.

Also, as seen in the last example we used two different methods of verifying that we did get the
optimal value. Do not get too locked into one method of doing this verification that you forget
about the other methods.

Let‘s work some another example that this time doesn‘t involve a rectangle or box.

Example 4 A manufacturer needs to make a cylindrical can that will hold 1.5 liters of liquid.
Determine the dimensions of the can that will minimize the amount of material used in its
construction.

Solution
In this problem the constraint is the volume and we want to minimize the amount of material
used. This means that what we want to minimize is the surface area of the can and we‘ll need to
include both the walls of the can as well as the top and bottom ―c
aps‖. Here is a quick sketch to
get us started off.
We‘ll need the surface area of this can and that will be the surface area of the walls of the can
(which is really just a cylinder) and the area of the top and bottom caps (which are just disks, and
don‘t forget that there are two of them).

Note that if you think of a cylinder of height h and radius r as just a bunch of disks/circles of
radius r stacked on top of each other the equations for the surface area and volume are pretty
simple to remember. The volume is just the area of each of the disks times the height. Similarly,
the surface area is just the circumference of the each circle times the height. The equations for
the volume and surface area of a cylinder are then,

Next, we‘re also going to need the required volume in a better set of units than liters. Since we
want length measurements for the radius and height we‘ll need to use the fact that 1 Liter = 1000
cm3 to convert the 1.5 liters into 1500 cm3. This will in turn give a radius and height in terms of
centimeters.

Here are the equations that we‘ll need for this problem and don‘t forget that there two caps and
so we‘ll need the area from each.

In this case it looks like our best option is to solve the constraint for h and plug this into the area
function.

Notice that this formula will only make sense from a physical standpoint if
which is a good thing as it is not defined at .

Next, let‘s get the first derivative.

From this we can see that we have two critical points : and

. The first critical point doesn‘t make sense from a physical standpoint and so
we can ignore that one.

So we only have a single critical point to deal with here and notice that 6.2035 is the only value
for which the derivative will be zero and hence the only place (with of course) that the
derivative may change sign. It‘s not difficult to check that if then and
likewise if then . The variant of the First Derivative Test above then tells us
that the absolute minimum value of the area (for ) must occur at .

All we need to do this is determine height of the can and we‘ll be done.

Therefore if the manufacturer makes the can with a radius of 6.2035 cm and a height of 12.4070
cm the least amount of material will be used to make the can.

As an interesting side problem and extension to the above example you might want to show that
for a given volume, L, the minimum material will be used if regardless of the volume of
the can.

In the examples to this point we‘ve put in quite a bit of discussion in the solution. In the
remaining problems we won‘t be putting in quite as much discussion and leave it to you to fill in
any missing details.

Example 5 We have a piece of cardboard that is 14 inches by 10 inches and we‘re going to cut
out the corners as shown below and fold up the sides to form a box, also shown below.
Determine the height of the box that will give a maximum volume.

.
Solution
In this example, for the first time, we‘ve run into a problem where the constraint doesn‘t really have an equation.
is simply the size of the piece of cardboard and has already been factored into the figure above. This will happen
and so don‘t get excited about it when it does. This just means that we have one less equation to worry about. In
want to maximize the volume. Here is the volume, in terms of h and its first derivative.

Setting the first derivative equal to zero and solving gives the following two critical points,

We now have an apparent problem. We have two critical points and we‘ll need to determine which one is the va
this case, this is easier than it looks. Go back to the figure in the problem statement and notice that we can quite
limits on h. The smallest h can be is even though this doesn‘t make much sense as we won‘t get a box in th
from the 10 inch side we can see that the largest h can be is although again, this doesn‘t make much sense p

So, knowing that whatever h is it must be in the range we can see that the second critical point is outsid
so the only critical point that we need to worry about is 1.9183.

Finally, since the volume is defined and continuous on all we need to do is plug in the critical points and
the volume to determine which gives the largest volume. Here are those function evaluations.

So, if we take we get a maximum volume.

Example 6 A printer need to make a poster that will have a total area of 200 in2 and will have 1
inch margins on the sides, a 2 inch margin on the top and a 1.5 inch margin on the bottom. What
dimensions will give the largest printed area?

Solution
This problem is a little different from the previous problems. Both the constraint and the
function we are going to optimize are areas. The constraint is that the overall area of the poster
must be 200 in2 while we want to optimize the printed area (i.e. the area of the poster with the
margins taken out).

Here is a sketch of the poster and we can see that once we‘ve taken the margins into account the
width of the printed area is and the height of the printer area is .
Here are the equations that we‘ll be working with.

Solving the constraint for h and plugging into the equation for the printed area gives,

The first and second derivatives are,

From the first derivative we have the following three critical points.

However, since we‘re dealing with the dimensions of a piece of paper we know that we must
have and so only 10.6904 will make sense.

Also notice that provided the second derivative will always be negative and so in the range
of possible optimal values of the width the area function is always concave down and so we
know that the maximum printed area will be at .

The height of the paper that gives the maximum printed area is then,

We‘ve worked quite a few examples to this point and we have quite a few more to work.
However this section has gotten quite lengthy so let‘s continue our examples in the next section.
This is being done mostly because these notes are also being presented on the web and this will
help to keep the load times on the pages down somewhat.
 1. Optimization in more than two variables

Optimizing a Function of Several Variables

Using Partial Derivatives

Local maximums and local minimums of a function of one variable, f(x), are found at critical
points (points where f'(x) = 0 or f'(x) is undefined) and at endpoints, as shown in the figure
below. Thus, one way to find local maximums and minimums is by looking at critical points and
endpoints.

The same idea works for a function of two or more variables except that a critical point is a point
at which all the partial derivatives are zero or undefined.

Example:

Suppose that a manufacturer wants to build a box that will hold one cubic foot of sand. The box
will be in the shape of a rectangular prism as shown in the figure below. The manufacturer wants
to minimize the cost of materials for the box by minimizing the total surface area of all six sides.
Notice that the total surface area is

S(x, y) = 2xy + 2xh + 2yh

Since the total volume is one cubic foot

xyh = 1

and

h = 1/xy

we can write

The two partial derivatives are

So, to find the critical points we must solve the two equations

or
Thus,

x=y=h=1

and the box should be a cube.

1. Suppose that a manufacturer wants to build a box with an open top that will hold one c
ubic foot of sand. She wants to minimize the cost of materials by minimizing the total
surface area of the box. What dimensions should the box have?

2.

3.
4. One cubic foot. The box should be a cube one foot on each side.
5. There are three critical points -- at (-1, 0), (0, 0), and (1, 0).
 The critical point at (0, 0) looks like a saddle. The other two critical points are
minimums.

Suppose that a manufacturer wants to build a box with an open top that will hold one cubic foot
of sand. She wants to minimize the cost of materials. Because sand is so heavy the bottom of the
box will cost four times as much per square foot as the sides. What dimensions should the box
have?

Problem 1: You decide to build a box that has the shape of a rectangular prism with a volume of
1000 cubic centimeters. Find the dimensions x, y and z of the box so that the total surface area of
all 6 faces of the box is minimum.

Solution to Problem 1:

The total area A of all six faces of the prism is given by.

A = 2xy + 2yz + 2zx

The volume of the box is given; hence

xyz = 1000

Solve the above for z.

z = 1000 / (xy)

Substitute z in the expression of the area A to obtain.

A(x,y) = 2xy + 2y * 1000 / (xy) + 2x * 1000 / (xy) = 2xy + 2000 / x + 2000 / y

We now need to find x and y that minimize the area A. We first need to find the critical
points and then test the second partial derivatives. The first order partial derivatives of A are
given by
Ax(x,y) = 2y - 2000 / (x2)

Ay(x,y) = 2x - 2000 / (y2)

The critical points are found by setting Ax(x,y) = 0 and Ay(x,y) = 0 and solving the system
obtained. Which gives

2y - 2000 / (x2) = 0 which gives 2yx2 = 2000

2x - 2000 / (y2) = 0 which gives 2xy2 = 2000

Solve the above to obtain

x = 10 and y = 10

We now need to find the second order partial derivatives.

Axx(x,y) = 4000/(x3)

Ayy(x,y) = 4000/(y3)

Axy(x,y) = 2

We now need to test the values of Axx, Ayy and Axy at the point (10,10) in order to use the
theorem on minima and maxima of functions with 2 variables.

D = Axx(10,10) Ayy(10,10) - Axy2(10,10) = 4 * 4 - 4 = 12

D is positive and Axx(10,10) = 4 is positive and therefore the area A is minimum for

x = 10 cm

y = 10 cm

z = 1000/(xy) = 10 cm.

Problem 2: Find the dimensions of a six-faced box that has the shape of a rectangular prism with
the largest possible volume that you can make with 12 squared meters of cardboard.

Solution to Problem 2:

Using all available cardboard to make the box, the total area A of all six faces of the prism is
given by.

A = 2xy + 2yz + 2zx = 12

The volume V of the box is given by

V = xyz

Solve the equation 2xy + 2yz + 2zx = 12 for z

z = (6 - xy) / (x + y)

Substitute z in the expression of the volume V to obtain.

V(x,y) = xy (6 - xy) / (x + y)

Let us find the critical points by first finding the first order partial derivatives

Vx(x,y) = -y2( x2 + 2xy - 6 ) / (x + y) 2

Vy(x,y) = -x2( y2 + 2xy - 6 ) / (x + y) 2

We now solve the system of equations given by Vx = 0 and Vy = 0. One obvious solution is given
by the point (0,0) but is not physically possible. Other solutions are found by setting

x2 + 2xy - 6 = 0

and

y2 + 2xy - 6 = 0

Subtracting the equations term by term we obtain

x2 - y2 = 0

Solve to obtain

x = y and x = - y

The solution x = -y is not valid for this problem since both x and y are dimensions and cannot be
negative. Use x = y in the equation x2 + 2xy - 6 = 0, we obtain

x2 + 2x2 - 6 = 0

Solve for x
x = sqrt(2)

Find y

y = x = sqrt(2)

Let us now find the second order partial derivatives

Vxx(x,y) = -2y2( y2 + 6 ) / (x + y) 3

Vyy(x,y) = -2x2( x2 + 6 ) / (x + y) 3

Vxy(x,y) = -2xy(x2 + 3xy + y2 - 6 ) / (x + y) 3

We now need the values of Axx, Ayy and Axy to find the value of D = Vxx(sqrt(2),sqrt(2))
Vyy(sqrt(2),sqrt(2)) - Vxy2(sqrt(2),sqrt(2)) in order to use the theorem on minima and maxima of
functions with 2 variables.

D = Vxx(sqrt(2),sqrt(2)) Vyy(sqrt(2),sqrt(2)) - Vxy2(sqrt(2),sqrt(2)) = 5/2

D is positive and Vxx(sqrt(2),sqrt(2)) = -sqrt(2) is negative and therefore the volume V is

maximum for

x = sqrt(2) meters

y = sqrt(2) meters

z = (6- xy) / (x + y) = sqrt(2) meters.

Problem 3: Find the distance from the point (1,2,-1) to the plane given by the equation x - y + z =
3.

Solution to Problem 3:

One way to find the distance from a point to a plane is to take a point (x,y,z) on the plane; find
the distance between this point and the given point and minimize it. Because the distance
involves the square root, it would be better to minimize the square of the distance. Let the square
of the distance between the given point and the point (x,y,z) on the plane be f.

f(x,y,z) = (x - 1)2 + (y - 2)2 + (z + 1)2

We now solve the equation of the plane for z to obtain

z=3-x+y
Substitute z in f by 3 - x + y in f.

F(x,y) = (x - 1)2 + (y - 2)2 + (- x + y + 4)2

We now find the first order partial derivatives

Fx(x,y) = 2(x - 1) + 2(-1)(-x + y + 4)

Fy(x,y) = 2(y - 2) + 2(-x + y + 4)

We now need to find the critical points by setting the first partial derivatives equal to zero.

2(x - 1) + 2(-1)(-x + y + 4) = 0 and 2(y - 2) + 2(-x + y + 4) = 0

We now solve the system of equations to obtain

(8/3 , 1/3 , 2/3)

We now calculate the second order derivatives

Fxx(x,y) = 4

Fyy(x,y) = 4

Fxy(x,y) = -2

We now need to find the sign of D = Fxx(8/3,1/3) Fyy(8/3,1/3) - Fxy2(8/3,1/3) in order to use the
theorem on minima and maxima of functions with 2 variables

D = 4 * 4 - (-2)2 = 12

Since D is positive and Fxx is positive, F has a minimum at the point (8/3,1/3) which correspond
to a point on the plane given by

(8/3,-1/3,2/3)

The distance d between the given point and the plane is given by

d = sqrt [ (1 - 8/3)2 + (2 - 1/3)2 + (-1 - 2/3)2 ]

= 5 / sqrt(3)

2.Optimization in two variables

Problem 1: You decide to build a box that has the shape of a rectangular prism with a volume of
1000 cubic centimeters. Find the dimensions x, y and z of the box so that the total surface area of
all 6 faces of the box is minimum .

Solution to Problem 1:

The total area A of all six faces of the prism is given by.

A = 2xy + 2yz + 2zx

The volume of the box is given; hence

xyz = 1000

Solve the above for z.

z = 1000 / (xy)

Substitute z in the expression of the area A to obtain.

A(x,y) = 2xy + 2y * 1000 / (xy) + 2x * 1000 / (xy) = 2xy + 2000 / x + 2000 / y

We now need to find x and y that minimize the area A. We first need to find the critical
points and then test the second partial derivatives. The first order partial derivatives of A are
given by

Ax(x,y) = 2y - 2000 / (x2)

Ay(x,y) = 2x - 2000 / (y2)

The critical points are found by setting Ax(x,y) = 0 and Ay(x,y) = 0 and solving the system
obtained. Which gives

2y - 2000 / (x2) = 0 which gives 2yx2 = 2000

2x - 2000 / (y2) = 0 which gives 2xy2 = 2000

Solve the above to obtain

x = 10 and y = 10

We now need to find the second order partial derivatives.

Axx(x,y) = 4000/(x3)

Ayy(x,y) = 4000/(y3)

Axy(x,y) = 2

We now need to test the values of Axx, Ayy and Axy at the point (10,10) in order to use the
theorem on minima and maxima of functions with 2 variables.

D = Axx(10,10) Ayy(10,10) - Axy2(10,10) = 4 * 4 - 4 = 12

D is positive and Axx(10,10) = 4 is positive and therefore the area A is minimum for

x = 10 cm

y = 10 cm

z = 1000/(xy) = 10 cm.

Problem 2: Find the dimensions of a six-faced box that has the shape of a rectangular prism with
the largest possible volume that you can make with 12 squared meters of cardboard.

Solution to Problem 2:
Using all available cardboard to make the box, the total area A of all six faces of the prism is
given by.

A = 2xy + 2yz + 2zx = 12

The volume V of the box is given by

V = xyz

Solve the equation 2xy + 2yz + 2zx = 12 for z

z = (6 - xy) / (x + y)

Substitute z in the expression of the volume V to obtain.

V(x,y) = xy (6 - xy) / (x + y)

Let us find the critical points by first finding the first order partial derivatives

Vx(x,y) = -y2( x2 + 2xy - 6 ) / (x + y) 2

Vy(x,y) = -x2( y2 + 2xy - 6 ) / (x + y) 2

We now solve the system of equations given by Vx = 0 and Vy = 0. One obvious solution is given
by the point (0,0) but is not physically possible. Other solutions are found by setting

x2 + 2xy - 6 = 0

and

y2 + 2xy - 6 = 0

Subtracting the equations term by term we obtain

x2 - y2 = 0

Solve to obtain

x = y and x = - y

The solution x = -y is not valid for this problem since both x and y are dimensions and cannot be
negative. Use x = y in the equation x2 + 2xy - 6 = 0, we obtain

x2 + 2x2 - 6 = 0

Solve for x
x = sqrt(2)

Find y

y = x = sqrt(2)

Let us now find the second order partial derivatives

Vxx(x,y) = -2y2( y2 + 6 ) / (x + y) 3

Vyy(x,y) = -2x2( x2 + 6 ) / (x + y) 3

Vxy(x,y) = -2xy(x2 + 3xy + y2 - 6 ) / (x + y) 3

We now need the values of Axx, Ayy and Axy to find the value of D = Vxx(sqrt(2),sqrt(2))
Vyy(sqrt(2),sqrt(2)) - Vxy2(sqrt(2),sqrt(2)) in order to use the theorem on minima and maxima of
functions with 2 variables.

D = Vxx(sqrt(2),sqrt(2)) Vyy(sqrt(2),sqrt(2)) - Vxy2(sqrt(2),sqrt(2)) = 5/2

D is positive and Vxx(sqrt(2),sqrt(2)) = -sqrt(2) is negative and therefore the volume V is

maximum for

x = sqrt(2) meters

y = sqrt(2) meters

z = (6- xy) / (x + y) = sqrt(2) meters.

Problem 3: Find the distance from the point (1,2,-1) to the plane given by the equation x - y + z =
3.

Solution to Problem 3:

One way to find the distance from a point to a plane is to take a point (x,y,z) on the plane; find
the distance between this point and the given point and minimize it. Because the distance
involves the square root, it would be better to minimize the square of the distance. Let the square
of the distance between the given point and the point (x,y,z) on the plane be f.

f(x,y,z) = (x - 1)2 + (y - 2)2 + (z + 1)2

We now solve the equation of the plane for z to obtain

z=3-x+y

Substitute z in f by 3 - x + y in f.

F(x,y) = (x - 1)2 + (y - 2)2 + (- x + y + 4)2

We now find the first order partial derivatives

Fx(x,y) = 2(x - 1) + 2(-1)(-x + y + 4)

Fy(x,y) = 2(y - 2) + 2(-x + y + 4)

We now need to find the critical points by setting the first partial derivatives equal to zero.

2(x - 1) + 2(-1)(-x + y + 4) = 0 and 2(y - 2) + 2(-x + y + 4) = 0

We now solve the system of equations to obtain

(8/3 , 1/3 , 2/3)

We now calculate the second order derivatives

Fxx(x,y) = 4

Fyy(x,y) = 4

Fxy(x,y) = -2

We now need to find the sign of D = Fxx(8/3,1/3) Fyy(8/3,1/3) - Fxy2(8/3,1/3) in order to use the
theorem on minima and maxima of functions with 2 variables

D = 4 * 4 - (-2)2 = 12

Since D is positive and Fxx is positive, F has a minimum at the point (8/3,1/3) which correspond
to a point on the plane given by

(8/3,-1/3,2/3)

The distance d between the given point and the plane is given by

d = sqrt [ (1 - 8/3)2 + (2 - 1/3)2 + (-1 - 2/3)2 ]

= 5 / sqrt(3)
 3. Inequalities and Convex Sets

Let K be a bounded, closed, convex set in the euclidean plane

having diameter d , width w , inradius r , and circumradius

R . We show that

(w-2r)d 5 2V3 r2

and

w(2R-d) 5 V3 (2-\/3)i?2

where both these inequalities are best possible.Let if tie a bounded, closed, convex set in the
euclidean plane. We

denote the area, perimeter, diameter, (minimal) width, inradius, and

circumradius of K by A, p, d, W, r , and S respectively. There are

many known inequalities amongst the quantities A, p, d, w, r , and R

(see, for example, [/], [2]). The two inequalities established in the

present paper appear to be new. THEOREM 1. (w-2r)d 5 2\^3 r2

, with equality when and only when K

is an equilateral triangle of side length 2sFi r .

Proof. We observe that a largest circle inscribed in -K must either

contain two boundary points of K which are ends of a diameter of the

circle, or else it contains three boundary points il, V, W of X which

form the vertices of an acute angled triangle (see, for example, [3]).

In the first case, w = 2r , and the theorem is trivially true. In the second case, the tangents to the
circle at U, V, W form a triangle,

hXYZ say. Since & is convex, K is contained in AWZ . In fact,

since we are interested in maximizing the width and diameter of K , we may

take K to be

We notice that for a triangle, the diameter is the length of a longest

side, and the width is the altitude to that side. Thus

wd = 2A = pr

and

{w-2r)d = r{p-2d) .

Hence it is sufficient for us to maximize p - 2d for a fixed value of

YxZ

Using the notation in the diagram, let us assume that x 2: y > z .

Now

p - 2d = {x+y+z) - 2x

=y+s-x

= 2x' .

Since r = x' tan(£/2) , and r is fixed, the maximum value of x'

will be assumed when (X is as small as possible, subject to the

constraint x 2 y 2 z . This occurs when IX = TT/3 (= IJ = /Z) ; that is,

when and only when t\XYZ is equilateral. For this equilateral triangle,Inequalities for convex
sets 133

d = 2V3 r , w = 32* , (w-2r)d = 2\/3 r .

Hence for any convex set X ,

(w-2r)d <2i§r 2

as required.
 THEOREM 2. u(2ff-d) £ V3{2-V3)R2

,with equality when and only when K

is a Reuleaux triangle of width V3 R .

Proof. It is known [3] that if K has circumradius R , then

V3 R £ d £ 2fl . Also, w £ d , so for any d ,

(2R-d)w £ (2R-d)d .

Now /(d) = (2R-d)d is a decreasing function of d , and so takes its

maximum value for d = 1/3 if . Hence

(2R-d)w £ (2-\/3)l/3 i?2

For equality here we require a set K having w = d ; that is, K must

be a set of constant width. Finally, it is known, [3], that the only set

of constant width which satisfies d = V3 R is the Reuleaux triangle of

width V3 R . This completes the proof.

Linear and nonlinear variational inequality problems over a polyhedral convex set are analyzed
parametrically. Robinson‘s notion of strong regularity, as a criterion for the solution set to be a
singleton depending Lipschitz continuously on the parameters, is characterized in terms of a new
―critical face‖ condition and in other ways. The consequences for complementarity problems are
worked out as a special case. Application is also made to standard nonlinear programming
problems with parameters that include the canonical perturbations. In that framework a new
characterization of strong regularity is obtained for the variational inequality associated with the
Karush-Kuhn-Tucker conditions.

Introduction to Inequalities

Inequality tells you about the relative size of two values.

Mathematics is not always about "equals"! Sometimes you only know that something is bigger or
smaller

Example: Alex and Billy have a race, and Billy wins!

What do we know?
We don't know how fast they ran, but we do know that Billy was faster than Alex:
Billy was faster than Alex
We can write that down like this:
b>a
(Where "b" means how fast Billy was, ">" means "greater than", and "a" means how fast Alex
was)

We call things like that inequalities (because they are not "equal")

Greater or Less Than

ss
Symbol Words Example Use

> greater than 5>2

< less than 7<9

They are easy to remember: the "small" end always points to the smaller number, like this:

Greater Than Symbol: BIG > small

Example: Alex plays in the under 15s soccer. How old is Alex?
We don't know exactly how old Alex is, because it doesn't say "equals"
But we do know "less than 15", so we can write:
Age < 15
The small end points to "Age" because the age is smaller than 15.

... Or Equal To!

You can also have inequalities that include "equals", like:

 Symbol Words Example Use

≥ greater than or equal to x≥1

≤ less than or equal to y≤3

Example: you must be 13 or older to watch a movie.

The "inequality" is between your age and the age of 13.
Your age must be "greater than or equal to 13", which would be written:
Age ≥ 13
Symbol Words Example

> greater than x+3>2

< less than 7x < 28

≥ greater than or equal to 5≥x-1

≤ less than or equal to 2y + 1 ≤ 7

Solving

Our aim is to have x (or whatever the variable is) on its own on the left of the inequality sign:

Something like: x<5

or: y ≥ 11

We call that "solved".

How to Solve

Solving inequalities is very like solving equations ... you do most of the same things ...
 ... but you must also pay attention to the direction of the inequality.

Direction: Which way the arrow "points"

Some things you do will change the direction!

< would become >
> would become <
≤ would become ≥
≥ would become ≤

Safe Things To Do

These are things you can do without affecting the direction of the inequality:

Add (or subtract) a number from both sides

Multiply (or divide) both sides by a positive number

Simplify a side
Example: 3x < 7+3
You can simplify 7+3 without affecting the inequality:
3x < 10

But these things will change the direction of the inequality ("<" becomes ">" for example):

Multiply (or divide) both sides by a negative number

Swapping left and right hand sides

Example: 2y+7 < 12
When you swap the left and right hand sides, you must also change the direction of the
inequality:
12 > 2y+7

Here are the details:

Adding or Subtracting a Value

 We can often solve inequalities by adding (or subtracting) a number from both sides (just as
in Introduction to Algebra), like this:

Solve: x + 3 < 7
If we subtract 3 from both sides, we get:
x+3-3<7-3
x<4
And that is our solution: x < 4
In other words, x can be any value less than 4.

What did we do?

We went from this: x+3 < 7

To this: x<4

And that works well for adding and subtracting, because if you add (or subtract) the same
amount from both sides, it does not affect the inequality

Example: Alex has more coins than Billy. If both Alex and Billy get three more coins each, Alex
will still have more coins than Billy.

What If I Solve It, But "x" Is On The Right?

No matter, just swap sides, but reverse the sign so it still "points at" the correct value!

Example: 12 < x + 5
If we subtract 5 from both sides, we get:
12 - 5 < x + 5 - 5
7<x
That is a solution!
But it is normal to put "x" on the left hand side ...
... so let us flip sides (and the inequality sign!):
x>7
Do you see how the inequality sign still "points at" the smaller value (7) ?
And that is our solution: x > 7

Note: "x" can be on the right, but people usually like to see it on the left hand side.
 Multiplying or Dividing by a Value

Another thing we do is multiply or divide both sides by a value (just as in Algebra -

Multiplying).

But we need to be a bit more careful (as you will see).

Positive Values

Everything is fine if you want to multiply or divide by a positive number:

Solve: 3y < 15
If we divide both sides by 3 we get:
3y/3 < 15/3
y<5
And that is our solution: y < 5

Negative Values

When you multiply or divide by a negative number

you have to reverse the inequality.

Why?
Well, just look at the number line!
For example, from 3 to 7 is an increase,
but from -3 to -7 is a decrease.

-7 < -3 7>3

See how the inequality sign reverses (from < to >) ?

Let us try an example:

Solve: -2y < -8

Let us divide both sides by -2 ... and reverse the inequality!
 -2y < -8
-2y/-2 > -8/-2
y>4
And that is the correct solution: y > 4

(Note that I reversed the inequality on the same line I divided by the negative number.)

So, just remember:

When multiplying or dividing by a negative number, reverse the inequality

Multiplying or Dividing by Variables

Here is another (tricky!) example:

Solve: bx < 3b
It seems easy just to divide both sides by b, which would give us:
x<3
... but wait ... if b is negative we need to reverse the inequality like this:
x>3
But we don't know if b is positive or negative, so we can't answer this one!

To help you understand, imagine replacing b with 1 or -1 in that example:

if b is 1, then the answer is simply x < 3

but if b is -1, then you would be solving -x < -3, and the answer would be x > 3

So:

Do not try dividing by a variable to solve an inequality (unless you know the variable is always
positive, or always negative).

A Bigger Example
Solve: (x-3)/2 < -5
First, let us clear out the "/2" by multiplying both sides by 2.
Because you are multiplying by a positive number, the inequalities will not change.
(x-3)/2 ×2 < -5 ×2
(x-3) < -10
Now add 3 to both sides:
x-3 + 3 < -10 + 3
x < -7
And that is our solution: x < -7
 Two Inequalities At Once!

How could you solve something where there are two inequalities at once?

Solve:
-2 < (6-2x)/3 < 4
First, let us clear out the "/3" by multiplying each part by 3:
Because you are multiplying by a positive number, the inequalities will not change.
-6 < 6-2x < 12
Now subtract 6 from each part:
-12 < -2x < 6
Now multiply each part by -(1/2).
Because you are multiplying by a negative number, the inequalities change direction.
6 > x > -3
And that is the solution!
But to be neat it is better to have the smaller number on the left, larger on the right. So let us swap
them over (and make sure the inequalities point correctly):
-3 < x < 6

Summary

Many simple inequalities can be solved by adding, subtracting, multiplying or dividing

both sides until you are left with the variable on its own.

But these things will change direction of the inequality:

Multiplying or dividing both sides by a negative number

Swapping left and right hand sides

Don't multiply or divide by a variable (unless you know it is always positive or

always negative)

Basic Rules

In this section, you will learn how so solve inequalities. "Solving'' an inequality means finding
all of its solutions. A "solution'' of an inequality is a number which when substituted for the
variable makes the inequality a true statement.

Here is an example: Consider the inequality

When we substitute 8 for x, the inequality becomes 8-2 > 5. Thus, x=8 is a solution of the
inequality. On the other hand, substituting -2 for x yields the false statement (-2)-2 > 5. Thus x =
-2 is NOT a solution of the inequality. Inequalities usually have many solutions.

As in the case of solving equations, there are certain manipulations of the inequality which do
not change the solutions. Here is a list of "permissible'' manipulations:

Rule 1. Adding/subtracting the same number on both sides.

Example: The inequality x-2>5 has the same solutions as the inequality x > 7. (The
second inequality was obtained from the first one by adding 2 on both sides.)

Rule 2. Switching sides and changing the orientation of the inequality sign.

Example: The inequality 5-x> 4 has the same solutions as the inequality 4 < 5 - x. (We
have switched sides and turned the ``>'' into a ``<'').

Last, but not least, the operation which is at the source of all the trouble with inequalities:

Rule 3a. Multiplying/dividing by the same POSITIVE number on both sides.

Rule 3b. Multiplying/dividing by the same NEGATIVE number on both sides AND
changing the orientation of the inequality sign.

Examples: This sounds harmless enough. The inequality has the same solutions
as the inequality . (We divided by +2 on both sides).

The inequality -2x > 4 has the same solutions as the inequality x< -2. (We divided by (-2)
on both sides and switched ">'' to "<''.)

But Rule 3 prohibits fancier moves: The inequality DOES NOT have the same
solutions as the inequality x > 1. (We were planning on dividing both sides by x, but we
can't, because we do not know at this point whether x will be positive or negative!) In
fact, it is easy to check that x = -2 solves the first inequality, but does not solve the
second inequality.

Only ``easy'' inequalities are solved using these three rules; most inequalities are solved by using
different techniques.

Let's solve some inequalities:

Example 1:
Consider the inequality

The basic strategy for inequalities and equations is the same: isolate x on one side, and put the
"other stuff" on the other side. Following this strategy, let's move +5 to the right side. We
accomplish this by subtracting 5 on both sides (Rule 1) to obtain

after simplification we obtain

Once we divide by +2 on both sides (Rule 3a), we have succeeded in isolating x on the left:

or simplified,

All real numbers less than 1 solve the inequality. We say that the "set of solutions'' of the
inequality consists of all real numbers less than 1. In interval notation, the set of solutions is the

interval .

Example 2:
Find all solutions of the inequality

Let's start by moving the ``5'' to the right side by subtracting 5 on both sides (Rule 1):

or simplified,

How do we get rid of the ``-'' sign in front of x? Just multiply by (-1) on both sides (Rule 3b),
changing " " to " " along the way:
or simplified

All real numbers greater than or equal to -1 satisfy the inequality. The set of solutions of the

inequality is the interval .

Example 3:
Solve the inequality

Let us simplify first:

There is more than one route to proceed; let's take this one: subtract 2x on both sides (Rule 1).

and simplify:

Next, subtract 9 on both sides (Rule 1):

simplify to obtain

Then, divide by 4 (Rule 3a):

and simplify again:

It looks nicer, if we switch sides (Rule 2).

In interval notation, the set of solutions looks like this: .

Exercise 1:
Find all solutions of the inequality

Solution.
We want to isolate x on the left: subtract 2 from both sides (Rule 1) to obtain

i.e.,

then, divide by 3 on both sides (Rule 3a):

so

The set of solutions consists of all real numbers exceeding 1.

Exercise 2:
Solve the inequality

Solution.
Start by subtracting 5 on both sides (Rule 1):
to obtain

then divide by (-3) on both sides, turning " " around (Rule 3b):

or simplified:

The set of solutions consists of all real numbers greater than or equal to 2/3.

Solving linear inequalities is very similar to solving linear equations, except for one small but
important detail: you flip the inequality sign whenever you multiply or divide the inequality by a
negative. The easiest way to show this is with some examples:

1) The only difference between the

linear equation "x + 3 = 2" and this
linear inequality is that I have a "less
than" sign, instead of an "equals"
sign. The solution method is exactly
the same: subtract 3 from either side.
Graphically, the solution is:
Note that the solution to a "less than,
but not equal to" inequality is
graphed with a parentheses (or else
an open dot) at the endpoint,
indicating that the endpoint is not
included within the solution.
2) The only difference between the
linear equation "2 – x = 0" and this
linear inequality is the "greater than"
sign in place of an "equals" sign.

Note that "x" in the solution does not

"have" to be on the left. However, it
Graphically, the solution is: is often easier to picture what the
solution means with the variable on
 the left. Don't be afraid to rearrange
things to suit your taste.
3) The only difference between the
linear equation "4x + 6 = 3x – 5" and
this inequality is the "less than or
equal to" sign in place of a plain
"equals" sign. The solution method is
exactly the same.

Note that the solution to a "less than

Graphically, the solution is: or equal to" inequality is graphed
with a square bracket (or else a
closed dot) at the endpoint,
indicating that the endpoint is
included within the solution.
4) The solution method here is to divide
both sides by a positive two.

Copyright © Elizabeth Stapel

1999-2011 All Rights Reserved
Graphically, the solution is:

5) This is the special case noted above.

When I divided by the negative two,
I had to flip the inequality sign.

Graphically, the solution is:

The rule for example 5 above often seems unreasonable to students the first time they see it. But
think about inequalities with numbers in there, instead of variables. You know that the number
four is larger than the number two: 4 > 2. Multiplying through this inequality by –1, we get –4 <
–2, which the number line shows is true:
If we hadn't flipped the inequality, we would have ended up with "–4 > –2", which clearly isn't
true.

The previous inequalities are called "linear" inequalities because we are dealing with linear
expressions like "x – 2" ("x > 2" is just "x – 2 > 0", before you finished solving it). When we
have an inequality with "x2" as the highest-degree term, it is called a "quadratic inequality". The
method of solution is more complicated.

Solve x2 – 3x + 2 > 0

First, I have to find the x-intercepts of

the associated quadratic, because the
intercepts are where y = x2 – 3x + 2
isequal to zero. Graphically, an
inequality like this is asking me to find
where the graph is above or below
the x-axis. It is simplest to find where
it actually crossesthe x-axis, so I'll start
there.

Factoring, I get x2 – 3x + 2 = (x – 2)
(x – 1) = 0, so x = 1 or x = 2. Then the
graph crosses the x-axis at 1 and 2, and
the number line is divided into the
intervals (negative infinity, 1), (1, 2),
and (2, positive infinity). Between the x-intercepts, the graph is either above the axis (and
thus positive, or greater than zero), or else below the axis (and thus negative, or less than
zero).

There are two different algebraic ways of checking for this positivity or negativity on the
intervals. I'll show both.

1) Test-point method. The intervals between the x-intercepts are (negative

infinity, 1), (1, 2), and (2, positive infinity). I will pick a point (any point) inside each
interval. I will calculate the value of y at that point. Whatever the sign on that value is,
that is the sign for that entire interval.

For (negative infinity, 1), let's say I choose x = 0; then y = 0 – 0 + 2 = 2, which is

positive. This says that y is positive on the whole interval of (negative infinity, 1), and
this interval is thus part of the solution (since I'm looking for a "greater than zero"
solution).

For the interval (1, 2), I'll pick, say, x = 1.5; then y = (1.5)2 – 3(1.5) + 2 = 2.25 – 4.5 + 2 =
4.25 – 4.5 = –0.25, which is negative. Then y is negative on this entire interval, and this
interval is then not part of the solution.
For the interval (2, positive infinity), I'll pick, say, x = 3; then y = (3)2 – 3(3) + 2 = 9 – 9 +
2 = 2, which is positive, and this interval is then part of the solution. Then the complete
solution for the inequality is x < 1 and x > 2. This solution is stated variously as:

inequality notation

interval, or set, notation

number line with

parentheses
(brackets are used
for closed intervals)

number line with open dots

(closed dots are used
for closed intervals)

The particular solution format you use will depend on your text, your teacher, and your
taste. Each format is equally valid. Copyright © Elizabeth Stapel 1999-2011 All Rights
Reserved

2) Factor method. Factoring, I get y = x2 – 3x + 2 = (x – 2)(x – 1). Now I will consider

each of these factors separately.

The factor x – 1 is positive for x > 1; similarly, x – 2 is positive for x > 2. Thinking back
to when I first learned about negative numbers, I know that (plus)×(plus) = (plus),
(minus)×(minus) = (plus), and (minus)×(plus) = (minus). So, to compute the sign
on y = x2 – 3x + 2, I only really need to know the signs on the factors. Then I can apply
what I know about multiplying negatives.

First, I set up a grid,

showing the factors
and the number line.

Now I mark the

intervals where each
factor is positive.
 Where the factors
aren't positive, they
must be negative.

Now I multiply up
the columns, to
compute the sign
of y on each interval.

Then the solution of x2 – 3x + 2 > 0 are the two intervals with the "plus" signs:

(negative infinity, 1) and (2, positive infinity).

Solve –2x2 + 5x + 12 < 0.

First I find the zeroes, which are the endpoints of the intervals: y = –2x2 + 5x + 12 =
(–2x – 3)(x – 4) = 0 for x = –3/2 and x = 4. So the endpoints of the intervals will be at –
3
/2 and4. The intervals are between the endpoints, so the intervals are (negative infinity, –
3
/2], [–3/2, 4], and [4, positive infinity). (Note that I use brackets for the endpoints in "or
equal to" inequalities, instead of parentheses, because the endpoints will be included in
the final solution.)

To find the intervals where y is negative by the Test-Point Method, I just pick a point in
each interval. I can use points such as x = –2, x = 0, and x = 5.

To find the intervals where y is negative by the Factor Method, I just solve each factor: –
2x – 3 is positive for –2x – 3 > 0, –3 > 2x, –3/2 > x, or x < –3/2; and x – 4 is positive
for x – 4 > 0,
x > 4. Then I fill out the grid:

Then the solution to this inequality is all x's in

(negative infinity, –3/2 ] and [4, positive infinity).

General Polynomial Inequalities

Solve x5 + 3x4 – 23x3 – 51x2 + 94x + 120 > 0.

First, I factor to find the zeroes:

x5 + 3x4 – 23x3 – 51x2 + 94x + 120

= (x + 5)(x + 3)(x + 1)(x – 2)(x – 4) = 0

...so x = –5, –3, –1, 2, and 4 are the zeroes of this polynomial. (Review how to solve
polynomials, if you're not sure how to get this solution.)

To solve by the Test-Point Method, I would pick a sample point in each interval, the
intervals being (negative infinity, –5), (–5, –3), (–3, –1), (–1, 2), (2, 4), and (4, positive
infinity). As you can see, if your polynomial or rational function has many factors, the
Test-Point Method can become quite time-consuming.

To solve by the Factor Method, I would solve each factor for its positivity: x + 5 >
0 for x > –5; x+ 3 > 0 for x > –3; x + 1 > 0 for x > –1; x – 2 > 0 for x > 2; and x – 4 >
0 for x > 4. Then I draw the grid:

...and fill it in:

...and solve:

Then the solution (remembering to include the endpoints, because this is an "or equal to"
inequality) is the set of x-values in the intervals [–5, –3], [–1, 2], and [4, positive
infinity].
As you can see, if your polynomial or rational function has many factors, the Factor Method can
be much faster.

Rational Inequalities

Solve x/(x – 3) < 2. ADVERTISEMENT

First off, I have to remember that I

can't begin solving until I have the
inequality in "= 0" format.

Now I need to convert to a common

denominator:

...and then I can simplify: Copyright © Elizabeth Stapel 1999-2011 All Rights Reserved

The two factors are –x + 6 and x – 3. Note that x cannot equal 3, or else I would be
dividing by zero, which is not allowed. The first factor, –x + 6, equals zero when x = 6.
The other factor, x – 3, equals zero when x = 3. Now, x cannot actually equal 3, so this
endpoint will not be included in any solution interval (even though this is an "or equal to"
inequality), but I need the value in order to figure out what my intervals are. In this case,
my intervals are (negative infinity, 3), (3, 6], and [6, positive infinity). Note the use of
brackets to indicate that 6 can be included in the solution, but that 3 cannot.

Using the Test-Point Method, I would pick a point in each interval and test for the sign on
the result. I could use, say, x = 0, x = 4, and x = 7.
 Using the Factor Method, I solve each factor: –x + 6 > 0 for –x > –6, or x < 6; x – 3 >
0 forx > 3. Then I do the grid:

...fill in the signs on the factors:

...and solve for the sign on the rational function:

So the solution is all x's in the intervals (negative infinity, 3) and [6, positive infinity).

There is another way to solve inequalities. You still have to find the zeroes (x-intercepts) first,
but then you graph the function, and just look: wherever the graph is above the x-axis, the
function is positive; wherever it is below the axis, the function is negative. For instance, for
the first quadratic exercise, y =x2 – 3x + 2 > 0, we found the zeroes at x = 1 and x = 2. Now look
at the graph:

On the graph, the solution is obvious: you would take the two intervals (but not the interval
endpoints) where the line is above the x-axis.

Or that huge polynomial we did at the top of this page: x5 + 3x4 – 23x3 – 51x2 + 94x + 120 > 0.
We found the zeroes of the polynomial, being x = –5, x = –3, x = –1, x = 2, and x = 4. Now look
at the graph:
On the picture, the solution is obvious: take the three intervals (together with the interval
endpoints) where the line is above the x-axis.

As you can probably guess, a graphing calculator can save you a lot of time on these inequalities
— ifyou understand what you're doing. You should still show your work and reasoning, but don't
be shy about using the pictures to confirm the algebra.

Convex Sets

Between a and b that does not belong to S. The point of this restatement is to include the
empty set within the definition of convexity. The definition also includes singleton sets where a
and b have to be the same point and thus the line between a and b is the same point. A convex set
is a set of elements from a vector space such that all the points on the straight line line between
any two points of the set are also contained in the set. If a and b are points in a vector space the
points on the straight line between a and b are given by

x = λa + (1-λ)b for all λ from 0 to 1.

The above definition can be restated as: A set S is convex if for any two points a and b
belonging to S there are no points on the line between a and b that are not members of S.
Another restatement of the definition is: A set S is convex if there are no points a and b in S such
that there is a point on the line

With the inclusion of the empty set as a convex set then it is true that:

The intersection of any two convex sets is a convex set

The proof of this theorem is by contradiction. Suppose for convex sets S and T there are
elements a and b such that a and b both belong to S∩T, i.e., a belongs to S and T and b belongs
to S and T and there is a point c on the straight line between a and b that does not belong to S∩T.
This would mean that c does not belong to one of the sets S or T or both. For whichever set c
does not belong to this is a contradiction of that set's convexity, contrary to assumption. Thus no
such c and a and b can exist and hence S∩T is convex.
Separating planes and Support Planes

The significance of convex sets in economics is some theorems on the existence of separating
planes and support planes for any convex set. The nature of these planes, more properly
hyperplanes, will be explained later.

On a vector space there are linear functionals which map the vector space into the real
numbers; i.e., f: V->R such that f(x+y)=f(x)+f(y). These linear functionals form a vector space,
called the dual space to the original vector space. A hyperplane is the set points of the vector
space that map into the same real value; i.e., x such that f(x)=b. The hyperplane has associated
with it two open half spaces; i.e., the set of points such that f(x)<b and the set of points such that
f(x)>b. There are also to closed half spaces associated with a hyperplane; i.e., the set of points
such that f(x)≤b and the set of points such that f(x)≥b.

Separating Hyperplane

Let C be a nonempty convex set of a vector space V and y any point of V that does not belong
to C. There exists a hyperplane g(x)=b such that y is in the hyperplane and C is a subset of one of
the two open half spaces associated with the hyperplane; i.e., for all x belonging to C either
g(x)<b or for all x belonging to C g(x)>b.

Supporting hyperplane

If y is a boundary point of a closed, nonempty convex set C then there exists a supporting
hyperplane h(x)=b such that y is in the hyperplane, h(y)=b, and all of C lies entirely in one of the
two closed half spaces associated with the hyperplane; i.e., either for all x in C, h(x)≤b or for all
x in C, h(x)≥b.

A convex set is a set of points such that, given any two points A, B in that set, the line
AB joining them lies entirely within that set.
Intuitively, this means that the set is connected (so that you can pass between any two
points without leaving the set) and has no dents in its perimeter. Sections of the
perimeter may be straight lines.

1. It is assumed that the concept of a line joining two points is defined. In a vector
space over the reals, it is the set {λA+(1-λ)B}, 0 < λ < 1}. It will be assumed that
we are dealing with vector spaces over the reals unless the contrary is stated
explicitly.
2. By convention, the empty set and all sets consisting of a single point are
regarded as convex. Since we cannot find two distinct points in such sets, we
cannot say that the line joining two such points isn't contained in the set.

Theorem
If X is a convex set and x1 ... xk are any points in it, then
 where all and
is also in X.
Proof: If k=2, the theorem is true by the definition of a convex set.
Now proceed by induction. Assume the theorem is true for k=m. If k=m+1, we can formulate the
definition of x as a linear combination of two points, one of which is itself a linear combination
of the first m points and the other is the (m+1)th point. By the induction hypothesis, the first
point must be in X; therefore, from the definition, s is in X.
Convex

A set in Euclidean space is convex set if it contains all the line segments connecting any pair
of its points. If theset does not contain all the line segments, it is called concave.

A convex set is always star convex, implying pathwise-connected, which in turn

implies connected.

What do you know about the topological properties of convex sets and the Lebesgue Measure
on Rn?

Let C be your convex set, and assume without loss of generality(1) that it contains zero as an
interior point and is bounded.
The question boils down to showing that ∂C has measure zero(2), which can be shown by
squeezing the boundary between the interior C∘, and a slightly expanded version of the
interior, 11−ϵ C∘.
Let p∈∂C. Since 0 is an interior point, by convexity the point q:=(1−ϵ )p lies in the interior of
the cone K:={sp+(1−s)x:x∈Br(0)}, and therefore q∈C∘. But then p=11−ϵ q∈11−ϵ C∘.

Thus
∂C⊂11−ϵ C∘.
Since for any set the boundary and the interior are disjoint,
∂C⊂11−ϵ C∘∖C∘.
Since the interior of a convex set is convex(3) and C∘ contains zero, C∘ is contained in it's
dilation:
C∘⊂11−ϵ C∘.

Finally, since we have assumed C∘ is bounded, the measure of the boundary,

λ(∂C)≤λ(11−ϵ C∘∖C∘)=(11−ϵ )nλ(C∘)−λ(C∘),
can be made as small as desired by taking ϵ →0.

Tying up loose ends:

(1):

If the set is not bounded, cut it off with a countable collection of successively larger
balls. Since the countable union of measurable sets is measurable, this suffices.

If the set C contains some interior point, translate the set so that the interior point is at
zero. Since the Lebesgue measure is translation invariant, this suffices.
If the set C contains no interior points, then all it's point must lie within
a n−1 dimensional plane, otherwise C would contain a n-tetrahedron (simplex), and a
simplex contains interior points. Thus C would lie within a measure zero set and the
result is trivial.
(2):

The boundary, closure, and interior of a set are always closed, closed, and open
respectively, so they are always measurable.
 If ∂C has measure zero, then ∂C∩C is measurable and has measure zero by
completeness of the Lebesgue measure.
Once you have measurability of ∂C∩C, you have measurability of C since,
C=(∂C∩C)∪C∘.

4.Basic Algebra

Algebra - Basic Definitions

It may help you to read Introduction to Algebra first

What is an Equation
An equation says that two things are equal. It will have an equals sign "=" like this:

x + 2 = 6
That equations says: what is on the left (x + 2) is equal to what is on the right (6)

So an equation is like a statement "this equals that"

Parts of an Equation
So people can talk about equations, there are names for different parts (better than saying
"that thingy there"!)

Here we have an equation that says 4x - 7 equals 5, and all its parts:

A Variable is a symbol for a number we don't

know yet. It is usually a letter like x or y.

A number on its own is called a Constant.

A Coefficient is a number used to multiply a

variable (4x means 4 times x, so 4 is a
coefficient)

An Operator is a symbol (such as +, ×, etc)

that represents an operation (ie you want to
do something with the values).
 A Term is either a single number or a
variable, or numbers and variables multiplied
together.

An Expression is a group of terms (the

terms are separated by + or - signs)

So, now we can say things like "that expression has only two terms", or "the second term is
a constant", or even "are you sure the coefficient is really 4?"

Exponents

The exponent (such as the 2 in x2) says how many times to use the
value in a multiplication.

Examples:
82 = 8 × 8 = 64
y3 = y × y × y
y2z = y × y × z

Exponents make it easier to write and use many multiplications

Example: y4z2 is easier than y × y × y × y × z × z, or even yyyyzz

Polynomial
Example of a Polynomial: 3x2 + x - 2

A polynomial can have constants, variables and the exponents 0,1,2,3,...

But you never have division by a variable.

 Monomial, Binomial, Trinomial
There are special names for polynomials with 1, 2 or 3 terms:

Like Terms
Like Terms are terms whose variables (and their exponents such as the 2 in x2) are the
same.

In other words, terms that are "like" each other. (Note: the coefficients can be different)

Example:
(1/3)xy2 -2xy2 6xy2

Are all like terms because the variables are all xy2

Basics of Algebra

Algebra is a division of mathematics designed to help solve certain types of problems quicker
and easier. Algebra is based on the concept of unknown values called variables, unlike arithmetic
which is based entirely on known number values.
This lesson introduces an important algebraic concept known as the Equation. The idea is that an
equation represents a scale such as the one shown on the right. Instead of keeping the scale
balanced with weights, numbers, or constants are used. These numbers are called constants
because they constantly have the same value. For example the number 47 always represents 47
units or 47 multiplied by an unknown number. It never represents another value.

The equation may also be balanced by a device called a variable. A variable is an an unknown
number represented by any letter in the alphabet (often x). The value of each variable must
remain the same in each problem.

Several symbols are used to relate all of the variables and constants together. These symbols are
listed and explained below.

? Multiply

* Multiply

/ Divide

+ Add or Positive

- Subtract or Negative

( ) Calculate what is inside of the parentheses first. (also called grouping symbols)

The diagram on the right shows a basic equation. This equation is similar to problems which you
may have done in ordinary mathematics such as:

__ + 16 = 30

You could easily guess that __ equals 14 or do 30 - 16 to find that __ equals 14.

In this problem __ stood for an unknown number; in an equation we use variables, or any letter
in the alphabet.
When written algebraically the problem would be:
x + 16 = 30

and the answer should be written:

x = 14

Solving Equations

These equations can be solved relatively easy and without any formal method. But, as you use
equations to solve more complex problems, you will want an easier way to solve them.

Pretend you have a scale like the one shown. On the right side there are 45 pennies and on the
left side are 23 pennies and an unknown amount of pennies. The scale is balanced, therefore, we
know that there must be an equal amount of weight on each side.

As long as the same operation (addition, subtraction, multiplication, etc.) is done to both sides of
the scale, it will remain balanced. To find the unknown amount of pennies of the left side,
remove 23 pennies from each side of the scale. This action keeps the scale balanced and isolates
the unknown amount. Since the weight(amount of pennies) on both sides of the scale are still
equal and the unknown amount is alone, we now know that the unknown amount of pennies on
the left side is the same as the remaining amount (22 pennies) on the right side.

Because an equation represents a scale, it can also be manipulated like one. The diagram below
shows a simple equation and the steps to solving it.

Initial Equation / Problem x + 23 = 45

Subtract 23 from each side x + 23 - 23 = 45 - 23

Result / Answer x = 22

The diagram below shows a more complex equation. This equation has both a constant and a
variable on each side. Again, to solve this you must keep both sides of the equation equal;
perform the same operation on each side to get the variable "x" alone. The steps to solving the
equation are shown below.
 Initial Equation / Problem: x + 23 = 2x + 45

Subtract x from each side x - x + 23 = 2x - x + 45

Result 23 = x + 45

Subtract 45 from each side 23 - 45 = x + 45 - 45

Result -22 = x

Answer x = -22

Take a look at the equation below. As you can see, after the variable is subtracted
from the left and the constants are subtracted from the right, you are still left with 2x
on one side.

Initial Equation / Problem x + 23 = 3x + 45

Subtract x from each side x - x + 23 = 3x - x + 45

Result 23 = 2x + 45

Subtract 45 from each side 23 - 45 = 2x + 45 - 45

Result -22 = 2x

Switch the left and right sides of the

2x = -22
equation

This means that the unknown number multiplied by two, equals -22. To find the value of x, use
the process "dividing by the coefficient" described on the next page.
Identifying and Using Coefficients

The coefficient of a variable is the number which the variable is being multiplied by. In this
equation, 2 is the coefficient of x because 2x is present in the equation. Some additional
examples of coefficients:

Term Coefficient of x

2x 2

0.24x 0.24

x 1

-x -1

Note that in the last two examples, the following rules are applied

If the variable has no visible coefficient, then it has an implied coefficient of 1.

If the variable only has a negative sign, then it has an implied coefficient of -1.

Continue to the next page to see how we use the coefficient of the variable x in the equation, 2,
to find the value of x.

Using Division

Recall beginning to solve the equation "x + 23 = 3x + 45". Applying addition and subtraction
gave

(from previous page) 2x = -22

But our end goal is to determine what x is, not what 2x is! Imagine that three investors own an
equal share in the company Example.Com. The total worth of Example.com is $300,000. To
determine what the share of each investor is, simply divide the total investment by 3:

$300,000 / 3 = $100,000

Thus, each investor has a $100,000 stake in Example.com. We apply the same idea to finding the
value of x. However, instead of dividing by the number of investors, we divide by the coefficient
of the variable. Since we determined that the coefficient of x is 2, we divide each side of the
equation by 2:
 After dividing by 2 1x = -11

Finally rewritten as x = -11

Continue to the next page for additional resources and tools for help with basic equations.

Basic Rules of Algebra

There are basic properties in math that apply to all real numbers. When working with variables
in algebra, these properties still apply. We will apply most of the following properties to solve
various Algebraic problems.
Algebraic Properties

Let a, b, and c be real numbers, variables, or algebraic expressions.

Commutative Property of Addition

We can add numbers in any order.

Commmutative Property of Multiplication

We can also multiply numbers in any order.

Associative Property of Addition

We can group numbers in a sum any way we want and get the same answer.

Associative Property of Multiplication

We can group numbers in a product any way we want and get the same answer.
Distributive Property

When we are adding and multiplying with a parenthesis, we can distribute the multiplication
through the addition.

For an in depth discussion, see Distributive Property

Additive Identity Property

If we add 0 to any number, we will end up with the same number.

Multiplicative Identity Property

If we multiply 1 to any number, we will end up with the same number.

Additive Inverse Property

If we adda number by the opposite of itself, we will end up with 0.

Multiplicative Inverse Property

If we multiply a number by its reciprocal, we will end up with 1.

Keep in mind that subtraction is also considered addition, but with a negative number. Similarly,
divison can be thought of as inverse multiplication, but with a restriction that the denominator
cannot be equal to 0.
Properties of Negation

We must be careful not to make arithmetic mistakes when dealing with negative signs and
subtraction.

Properties of Equality

Add c to each side

Multiply both sides by c

Subtract c from both sides

Divide both sides by c

Properties of Zero

0 added or subtracted to anything equals itself

0 multiplied by anything equals 0

0 divided by anything equals 0

We cannot divide by 0
Zero Product Property

If the product of two or more things equals 0, at least one of the values must be 0
Properties and Operations of Fractions

Let a, b, c and d be real numbers, variables, or algebraic expressions such that b and d do not
equal 0.
Equivalent Fractions

cross multiply
Rules of Signs

the negative can go anywhere in the fraction and two negatives equal a positive
Generate Equivalent Fractions

multiplying the top and bottom by the same thing keeps the fraction the same value
Add/Subtract with Like Denominators

if the denominators are the same, add or subtract the top of the fraction
Add/Subtract with Unlike Denominators

find a common denominator

Multiply Fractions

top times the top and bottom times the bottom

Divide Fractions

2 Simplex Method And Duality

Every primal problem, regardless of its type (canonical or standard), dual problem can be
found by using the following converting table.
 Unfortunately, it remains unclear how exactly should I use this table.
There was an example
Consider the following linear programm
max 8x1+3x2
x1−6x2≥2
5x1+7x2=−4
x1≤0
x2≥0
applying the results of the table, we can immediatly write down the dual
min 2w1−4w2
w1+5w2≤8
−6w1+7w2≥3
w1≤0
w2unrestricted
Please, show me how can I use this table and get this result. And in you opininon what's
the best technique for finding dual problem: converting to one of the form (canonical or
standartd and find the dual), memorize the table, or something else.

1 Answer
One quick way to think about it is that you're transposing the entire
problem. Imagine placing the primal coefficients in a matrix, with the objective
at the bottom. This gives P=⎡⎣158−6732−4⎤⎦. Transposing
yields PT=D=⎡⎣1−6257−483⎤⎦. Thus the dual (coefficients and variables only)
looks like
max or min subject to 2w1−4w2w1+5w2 ? 8−6w1+7w2 ? 3.

Now we have to figure out whether we are maximizing or minimizing, what

should go where the ?'s are, and whether the dual variables are nonnegative,
nonpositive, or unrestricted. The table tells us all of this.

Since the primal problem is maximizing, the dual is minimizing. Everything else
can be read by pairing rows with columns in the transposition: Row x in P goes
with Column x in D, and Column y in P goes with Row y in D. Then the table
says...
1. Row 1 in P is a ≥ constraint in a maximization problem ⇒ The variable
associated with Column 1 in D, w1, has w1≤0.
2. Row 2 in P is a = constraint ⇒ w2 is unrestricted.
3. Column 1 in P is for x1, and x1≤0 ⇒ The constraint associated with Row 1
in D is a ≤constraint.
4. Column 2 in P, with x2≥0 ⇒ The second constraint in the dual is
a ≥ constraint.
Thus we get the complete form of the dual
minsubject to 2w1−4w2w1+5w2≤8−6w1+7w2≥3w1≤0w2 unrestricted.

For remembering how to do this, I prefer something called the "SOB"

method instead of memorizing the table. "SOB" here stands for "sensible,"
"odd," or "bizarre." The SOB table is the following:
Variables Constraints, Maximizing Constraints, Minimizing
Sensible ≥ 0 ≤ ≥
Odd Unrestricted = =
Bizarre ≤ 0 ≥ ≤
Hopefully it makes sense why these are "sensible," "odd," and "bizarre," at least
relatively speaking.

The idea then is that sensible maps to sensible, odd maps to odd, and bizarre
maps to bizarre when you're switching from the primal to the dual. Let's take
the example problem. It's "bizarre" to have a ≥ constraint in a maximization
problem, so the variable in the dual associated with the first constraint, w1, must
have the "bizarre" nonpositivity restriction w1≤0. It's "odd" to have an equality
constraint, and so the variable in the dual associated with the second
constraint, w2, must have the "odd" unrestricted property. Then, it's "bizarre" to
have a variable x1 with a nonpositivity restriction, so the constraint in the dual
associated with x1 must have the "bizarre" ≤constraint for a minimization
problem. Finally, it's "sensible" to have a variable x2 with a nonnegativity
restriction, so the constraint in the dual associated with x2 must have the
"sensible" ≥ constraint for a minimization problem.
I've found that after a few practice examples my students generally internalize
the SOB method well enough that they can construct duals without needing to
 memorize anything.

Linear programming problems with trapezoidal fuzzy variables (FVLP) have recently attracted
some interest. Some methods have been developed for solving these problems by introducing
and solving certain auxiliary problems. Here, we apply a linear ranking function to order
trapezoidal fuzzy numbers. Then, we establish the dual problem of the linear programming
problem with trapezoidal fuzzy variables and hence deduce some duality results. In particular,
we prove that the auxiliary problem is indeed the dual of the FVLP problem. Having established
the dual problem, the results will then follow as natural extensions of duality results for linear
programming problems with crisp data. Finally, using the results, we develop a new dual
algorithm for solving the FVLP problem directly, making use of the primal simplex tableau. This
algorithm will be useful for sensitivity (or post optimality) analysis when using primal simplex
tableaus.

Dualit Theorems

duality theorem [dü′al·əd·ē ‚thir·əm]

(mathematics)
A theorem which asserts that for a given n-dimensional space, the (n-p) dimensional homology
group is isomorphic to a p-dimensional cohomology group for each p = 0, …, n, provided certain
conditions are met.
Let G be either a compact group or a discrete group, let X be its character group, and let G′ be the
character group of X ; then there is an isomorphism of G onto G′ so that the groups G and G′may
be identified.
If either of two dual linear-programming problems has a solution, then so does the other.

Duality
Duality in algebraic geometry. Duality between the different cohomology spaces on algebraic
varieties.

Cohomology of coherent sheaves. Let be a non-singular algebraic variety of dimension over

an algebraically closed field and let be a locally free sheaf on . Serre's duality theorem
states that the finite-dimensional cohomology (vector) spaces
and are mutually dual. Here is the sheaf of germs of regular
differential forms of degree on , and is the locally free sheaf dual to
. If is the invertible sheaf corresponding to a divisor on , this theorem
establishes the equality
where is the canonical divisor on . If , a relation equivalent to the above was found as
early as the 19th century. There exists a generalization of Serre's theorem to the case of
cohomology of arbitrary coherent sheaves on complete algebraic varieties [1], [4]. In particular,
if the variety is a Cohen–Macaulay subvariety (e.g. a locally complete intersection) of
codimension in a non-singular projective variety , there is duality between the -
space and the space of global Ext's

where is a coherent sheaf on , (Grothendieck's dualizing

sheaf), while . Here, the sheaf is invertible if and only if is a Gorenstein
scheme (cf. Gorenstein ring).

Etale cohomology. Let be a complete connected non-singular algebraic variety of

dimension over an algebraically closed field ; let be an integer which is relatively prime to
the characteristic of the field ; let be a locally free (in the étale topology) sheaf of -
modules on ; and let be the sheaf of -th power roots of unity. Then there exists a non-
degenerate pairing of -modules [6]:

A more general duality theorem concerns smooth, but not necessarily complete, varieties [5].
There exists a non-degenerate pairing of -modules

where cohomology with compact support is found on the left-hand side. If the field is the
algebraic closure of a field , and , then the Galois
group acts on and the preceding pairing is a pairing of -
modules.

Poincaré's duality theorem is an analogue of the first of the theorems given for -adic
cohomology: There exists a non-degenerate pairing of -modules

where is the Tate sheaf, which is non-canonically isomorphic to the sheaf (cf. -adic
cohomology). Hence the isomorphism of -spaces
and, in particular, the equality of the Betti numbers

As in the case of cohomology of coherent sheaves, these results can be generalized to the relative
case of a proper scheme morphism, formulated in the language of derived categories [6].

Other cohomology theories. Analogues of Poincaré's theorem are valid in the theory of
crystalline cohomology [7], and de Rham cohomology over a field of characteristic zero [8]. In
number-theoretic applications cohomology of sheaves on the flat Grothendieck topology of
arithmetic schemes are important. Duality theorems [9] are applicable to special cases of such
cohomology theories.s

2.Primal and Dual

Primal/Dual Problems

Associated with every LP problem, which will be referred to a primal problem, there is a
corresponding dual problem. See Section 3.5.1 in the text. In a certain way, the dual problem is
a transposition of the primal problem. This is meant to imply that if the primal problem is a
minimization problem the dual problem will be a maximization one. If the primal problem has n-
variables, the dual problem will have n- constraints. If the primal problem has m-constraints, the
dual problem will have m-variables. For the discussion of duality, the standard primal problem is
generally defined as a maximization problem with <= inequalities (though the standard problem
in the LP programming discussion was a minimization problem see 3.10 - 3.12 and the
constraints were equalities).

The standard primal problem (normal maximum problem) in this section is defined as:

Maximize
Subject to:

Side Constraints
The dual of the above problem (normal minimum problem) is defined as:

Minimize

Subject to:

Side Constraints

The primal/dual pair enjoy the following properties in problem formulation:

1. The number of dual variables is the same as the number of primal constraints
2. The number of dual constraints is the same as the number of primal variables
3. The coefficient matrix A of the primal problem is transposed to provide the coefficient
matrix of the dual problem
4. The inequalities are reversed in direction
5. The maximization problem of the primal problem becomes a minimization problem in the
dual problem
6. The cost coefficients of the primal problem become the right hand sides of the dual
problem. The right hand side values of the primal become the cost coefficients in the dual
problem
7. The primal and dual variables both satisfy the non negativity condition.

For more details see Example 3.5 in the text.

The primal/dual problem also share many relationships in their solution:

1. The primal and dual problem have the same value for the optimal objective function. This is
however true only if the problems have optimal solutions.

2. If x is any feasible solution to the primal problem, and y is any feasible solution to the dual
problem, then w(y) >= z(x). This feature provides an estimate of the bounds of the dual optimal
value if a feasible primal solution is known. This results also holds for the reverse case when a
feasible dual is known.

3. If the primal problem is unbounded, the dual problem is infeasible. The unbounded problem
implies that the objective function value can be pushed to infinite limits. This happens if the
feasible domain is not closed. Of course practical considerations will limit the objective function
value to corresponding limits on the design variables. The inverse relationship holds too. If the
dual problem is unbounded, the primal is infeasible.
4. If the ith primal constraint is an equality constraint, the ith dual variable is unrestricted in
sign. The reverse holds too. If the primal variable is unrestricted in sign, then the dual constraint
is an equality.

5. Obtaining primal solution from dual solution:

(i) If the ith dual constraint is a strict inequality, then the ith primal variable is non-basic (for
optimum solutions only).
(ii) if the ith dual variable is basic, then the ith primal constraint is a strict equality.

6. Recovering primal solution from final dual Table: When the primal and dual problems are
in the standard form, the value of the ith primal variable equals the reduced coefficient (that is
the coefficient in the last row of the final table) of the slack/surplus variable associated with the
ith dual constraint.

7. If a ith dual variable is non-basic, the value of its reduced cost coefficient is the value of the
slack/surplus variable of the corresponding primal constraint.

8. Obtaining Dual Solution from Primal Solution: When the primal and dual problems are in
the standard form, the value of the ith dual variable equals the reduced coefficient (that is the
coefficient in the last row of the final table) of the slack/surplus variable associated with the ith
primal constraint.

9. If a ith primal variable is non-basic, the value of its reduced cost coefficient is the value of
the slack/surplus variable of the corresponding dual constraint.

Primal and dual solution to linear programming

Lets say we are given a primal linear programming problem:
minimize subject to: cTxAxx≥≥b0
The dual problem is defined as:
maximize subject to: bTyATyy≤≥c0
According to the duality theorem cTx≥bTy for every feasible solution x and y, and in addition when x an
solutions to the primal and the dual task then cTx=bTy
So if we define linear programming task with following constraints:
AxxATyybTy≥≥≤≥≥b0c0cTx
Then any feasible solution to this task should be an optimal solution to the primal and the dual task, bec
might be satisfied only if x and y are optimal.
The question is why this approach is not used?

I see three potential reasons:

1) I've made somewhere mistake and it doesn't make any sense.
2) It is often the case when primal or dual problem is infeasible. I've seen such examples, but in all of th
solution was unbounded, is it the only case when exactly one of the primal and dual problem is infeasibl
3) Finding any feasible solution might be hard. The so called Phase 1 of simplex method can be used to
solution. I couldn't find the complexity of this phase, is it exponential just like the simplex algorithm? T
what is the fastest method to determine whether there exist any feasible solution? This solution doesn't h

2 Answers

There are three things that have to be satisfied in zlinear programming problem to be
optimal:

1. The primal solution must be feasible.

2. The dual solution must be feasible.
3. Complementary slackness must be satisfied. (Remember that primal variables are
paired with dual slack variables and dual variables are paired with primal slack
variables. Complementary slackness is the requirement that, for each of these
pairs, at least one variable must be zero.)
The primal simplex method (after Phase I) keeps (1) and (3) always true and searches
for a solution that satisfies (2). The dual simplex method (again, after Phase I), keeps
(2) and (3) always true and searches for a solution that satisfies (1).

3.Simplex Method
The simplex method is a method for solving problems in linear programming. This
method, invented by George Dantzig in 1947, tests adjacent vertices of the feasible set
(which is a polytope) in sequence so that at each new vertex the objective function
improves or is unchanged. The simplex method is very efficient in practice, generally
taking to iterations at most (where is the number of equality constraints),
and converging in expectedpolynomial time for certain distributions of random inputs
(Nocedal and Wright 1999, Forsgren 2002). However, its worst-case complexity is
exponential, as can be demonstrated with carefully constructed examples (Klee and
Minty 1972).

A different type of methods for linear programming problems are interior point
methods, whose complexity is polynomial for both average and worst case. These
methods construct a sequence of strictly feasible points (i.e., lying in the interior of
the polytope but never on its boundary) that converges to the solution. Research on
interior point methods was spurred by a paper from Karmarkar (1984). In practice, one
of the best interior-point methods is the predictor-corrector method of Mehrotra
(1992), which is competitive with the simplex method, particularly for large-scale
problems.
Use of this system is pretty intuitive: Press "Example" to see an example of a linear
programming problem already set up. Then modify the example or enter your own linear
programming problem in the space below using the same format as the example, and press
"Solve."

Notes

Do not use commas in large numbers. For instance, enter 100,000 as 100000.
The right-hand side of each constraint must be non-negative, so multiply through by-1
first if necesssary.
The utility is quite flexible with input. For instance, the following format will also be
accepted (inequalities separated by commas):

Maximize p = x+y subject to x+y <= 2, 3x+y >= 4

Decimal mode displays all the tableaus (and results) as decimals, rounded to the number
of significant digits you select (up to 13, depending on your processor and browser).
Fraction mode converts all decimals to fractions and displays all the tableaus (and
solutions) as fractions.
Integer Mode eliminates decimals and fractions in all the tableaus (using the method
described in the simplex method tutorial) and displays the solution as fractions.
Mac users: you can use the inequality symbols "option+<" and "option+>" instead of
"<=" and ">=" if you like (although some browsers may have difficulties with this).
Solution Display Some browsers (including some versions of Internet Explorer) use a
proportional width font (like Geneva or Times) in text boxes. This will cause the display
of solutions to appear a little messy. You can remedy this by changing the "Sans Serif"
font in your browser preferences to "Courier" or some other fixed-width font, and then
reloading the page.
The Simplex Method is "a systematic procedure for generating and testing candidate
vertex solutions to a linear program." (Gill, Murray, and Wright, p. 337) It begins at an
arbitrary corner of the solution set. At each iteration, the Simplex Method selects the
variable that will produce the largest change towards the minimum (or maximum)
solution. That variable replaces one of its compatriots that is most severely restricting it,
thus moving the Simplex Method to a different corner of the solution set and closer to the
final solution. In addition, the Simplex Method can determine if no solution actually
exists. Note that the algorithm is greedy since it selects the best choice at each iteration
without needing information from previous or future iterations.
The Simplex Method solves a linear program of the form described in Figure 3. Here, the
coefficients represent the respective weights, or costs, of the variables . The
minimized statement is similarly called the cost of the solution. The coefficients of the
system of equations are represented by , and any constant values in the system of
equations are combined on the right-hand side of the inequality in the variables .
Combined, these statements represent a linear program, to which we seek a solution of
minimum cost.

Figure 3

A Linear Program

Solving this linear program involves solutions of the set of equations. If no solution to the
set of equations is yet known, slack variables , adding no cost to the
solution, are introduced. The initial basic feasible solution (BFS) will be the solution of
the linear program where the following holds:

Once a solution to the linear program has been found, successive improvements are made
to the solution. In particular, one of the nonbasic variables (with a value of zero) is

chosen to be increased so that the value of the cost function, , decreases. That
variable is then increased, maintaining the equality of all the equations while keeping the
other nonbasic variables at zero, until one of the basic (nonzero) variables is reduced to
zero and thus removed from the basis. At this point, a new solution has been determined
at a different corner of the solution set.
The process is then repeated with a new variable becoming basic as another becomes
nonbasic. Eventually, one of three things will happen. First, a solution may occur where
no nonbasic variable will decrease the cost, in which case the current solution is the
optimal solution. Second, a non-basic variable might increase to infinity without causing
a basic-variable to become zero, resulting in an unbounded solution. Finally, no solution
may actually exist and the Simplex Method must abort. As is common for research in
linear programming, the possibility that the Simplex Method might return to a previously
visited corner will not be considered here.
The primary data structure used by the Simplex Method is "sometimes called a
dictionary, since the values of the basic variables may be computed (‗looked up‘) by
choosing values for the nonbasic variables." (Gill, Murray, and Wright, p. 337)
Dictionaries contain a representation of the set of equations appropriately adjusted to the
current basis. The use of dictionaries provide an intuitive understanding of why each
variable enters and leaves the basis. The drawback to dictionaries, however, is the
necessary step of updating them which can be time-consuming. Computer
implementation is possible, but a version of the Simplex Method has evolved with a more
efficient matrix-oriented approach to the same problem. This new implementation
became known as the Revised Simplex Method.

The Revised Simplex Method

The Revised Simplex Method describes linear programs as matrix entities and presents
the Simplex Method as a series of linear algebra computations. Instead of spending time
updating dictionaries at the end of each iteration, the Revised Simplex Method does its
heavy calculation at the beginning of each iteration, resulting in much less at the end. As
Chvátal observed,
"Each iteration of the revised simplex method may or may not take less
time than the corresponding iteration of the standard simplex method. The
outcome of this comparison depends not only on the particular
implementation of the the revised simplex method but also on the nature
of the data. ...on the large and sparse [linear programming] problems
solved in applications, the revised simplex method works faster than the
standard simplex method. This is the reason why modern computer
programs for solving [linear programming] problems always use some
form of the revised simplex method." (pp. 97-98)

First of all, a linear program needs to be expressed in terms of matrices. for instance, the
linear program stated previously:

can instead be recast in the form:

where:
 The steps of the Simplex Method also need to be expressed in the matrix format
of the Revised Simplex Method. The basis matrix, B, consists of the column
entries of A corresponding to the coefficients of the variables currently in the
basis. That is if is the fourth entry of the basis, then is
the fourth column of B. (Note that B is therefore an matrix.) The non-
basic columns of A constitute a similar though likely not square, matrix referred
to here as V.

Simplex Method Revised Simplex Method

Determine the current basis, d.

Choose to enter the basis

based on the greatest cost
contribution.

If cannot decrease the If , d is optimal solution.

cost, d is optimal solution.

Determine that first exits the

basis (becomes zero) as
increases.

If can decrease without If for all i, the solution is

causing another variable to leave unbounded.
the basis, the solution is
unbounded.
Update dictionary. Update .

The physical limitations of a computer can become a factor. Round-off error and
significant digit loss are common problems in matrix manipulations, especially when the
entries of a matrix have widely differing orders of magnitude. Such matrices are usually
not well-conditioned. Matrices that are even close to ill-conditioned can cause severe
round-off errors in the finite memory of a computer. Implementation of the Revised
Simplex Method becomes more than simply programming the algorithm; it also becomes
a task in numerical stability.
Algorithmic analysis of the Revised Simplex Method shows that the costliest step is the
inversion of the basis, taking multiplications and additions, a total
of floating-point (real number) calculations. Many variants of the Revised

Simplex Method have been designed to reduce this -time algorithm as well as
improve its accuracy.

The Bartels-Golub Method

The first variant of the Revised Simplex Method was the Bartels-Golub Method which
mildly improved running time and storage space and significantly improved numerical
accuracy. The Bartels-Golub Method avoids the problem of matrix-inversion by simply
avoiding matrix-inversion altogether. Instead, the basis is decomposed into upper- (U)
and lower- (L) triangular factors, which only takes about computations to
determine. In addition, LU-decomposition of a sparse basis produces sparse factors,
preserving some space lost to dense inverse-bases. Best of all, the factors can be
permuted via pivoting to decrease the round-off errors due to a near-singular basis.
The actual implementation of the Bartels-Golub Method closely parallels the Revised
Simplex Method, utilizing the L and U factors where the inverse basis was required
before. The increase in speed was impressive, but the improvement in accuracy was
dramatic, occasionally solving problems that the Revised Simplex Method simply could
not do before.

Revised Simplex Method Bartels-Golub Method

If , d is optimal If , d is optimal
solution. solution.

If for all i, the If for all i, the

solution is unbounded. solution is unbounded.

Update . Update and .

Where the Revised Simplex Method called for the inverse of the basis to be multiplied,
the Bartels-Golub substitutes two backsolves involving the upper and lower triangular
basis factors. When a variable is selected to enter the basis, the Bartels-Golub Method
maintains that L not be changed, therefore requiring that U must make the changes
necessary to accomodate the new variable. Since the variable brings an entire column of
coefficients into the basis, an entire column of U will change when the new variable
enters, creating a "spike" in the upper triangular formation.
The various implementations and variations of the Bartels-Golub generally diverge with
the next step: reduction of the spiked upper triangular matrix back to an upper-triangular
matrix. As Chvátal noted, "some device is used to facilitate their solutions and is updated
at the end of each iteration." (p. 105) The most common methods attempt to amortize the
cost of refactorization to reduce the average asymptotic complexity of the Revised
Simplex Method. These methods have met with varying levels of success prompting this
study on their effectiveness.
The simplex algorithm is used to solve linear programming problems when the graphical method
cannot be used - on computers or when there are more than 2 variables to plot.

How the simplex algorithm works

The simplex algorithm takes the equations of the constraints and solves them simultaneously to
find the nodes. It then works out whether that node maximises the objective function.
Slack variables

In order to solve the simultaneous equations, the constraints must be in a format without
inequalilities. Therefore slack variables are used, to stand for an arbitrary number to "make up
the difference" between one side and the other. The letters s and t are used for these.
Example
would become
would become
Formulating the tableau

Formulating the tableau

The equations are then put into a computer-readable form - a tableau. This is a table with one
row for each equation and a column for each variable. The coefficients of all the variables are
listed in the cells.

The top row is always the equation for the objective function. The next 2 rows are the equations
of the constraints.

Rearrange the equations so that they all have the constants on the right hand side. Then enter the
equations into the tableau.

The objective function is made to equal P if it is to be maximised, and -P it is to be minimised

(e.g. maximise would become and would be entered on the tableau
as ).
The Pivot

Column chosen; now choosing pivot.

When you have formulated the tableau, choose any column which has a negative number in the
top row. Our aim now is to manipulate the equations to eliminate this chosen variable from 2 of
the equations. To do this, first choose a pivot - divide the k (constant) values for each equation
by their respective values in the selected columns. The pivot is the value of the selected column
of whichever equation gives the smallest answer to this calculation.

Solving the equations

Solving the equations

Divide the pivot row by the pivot to get 1 in the selected column. Add a multiple of the pivot eqn
to the other equations to get the pivot column to equal 0. If the top row has no negative elements,
stop. If it still contains negative elements, find a new pivot from the newest 3 equations and
repeat the procedure.
 Interpreting result

Find the maximum by interpreting the tableau as in the picture. P equals the k value in the top
row; non-zero elements in the top row equal zero. Use the other two equations to find the
remaining values.

4. Standard Form and Solutions

Standard Form For Linear Equations
This page assume that the reader understands the following concepts:

1. The coordinate plane.

2. Graphing lines on the coordinate plane.
3. Slope and y-intercept for linear equations.
4. Slope-intercept form for linear equations, y = m x + b.
5. Point-Slope form for linear equations, y – y1 = m ( x – x1).
6. Equations of parallel lines have the same slope.
7. Equations of perpendicular lines have either
o one being horizontal and the other vertical, or
o slopes that multiply to give -1

This page will build upon all of this through an examination of the standard form for linear
equations:
Ax + By = C

The Standard Form for a linear equation in two variables, x and y, is usually given as

Ax + By = C

where, if at all possible, A, B, and C are integers, and A is non-negative, and, A, B, and C have
no common factors other than 1. If we have a linear equation in slope-intercept form,

y=mx+b
we can change that equation into Standard Form. To do this we need to express the slope and the
ordinate of the the y-intercept in rational number form, that is, as the quotient of two integers.
For the kinds of problems that we usually find in math classes, this is not much of a demand. The
slope is defined to be the change in ydivided by the change in x. Thus, if we express the slope
as ychange/xchange, we will have met our first demand. The ordinate of the y-intercept usually
follows the same scheme, so we can express that value, the "b" in y=mx+b, as the quotient of
two integers, which we will call b_num and b_den. This means that our slope-intercept form

y=mx+b

can be rewritten as

y = (ychange/xchange) x + b_num/b_den

If we multiply both sides of the equation by the least common multiple of xchange" and "b_den",
the resulting equation will have no fractions. It will appear as

Dy = Ex + F

where D, E, and F are integers. Then, we add – Ex to both sides of the equations to get – Ex +
Dy = F To get this into standard form we want the coefficient of x to be non-negative. If – E is
actually negative, then we can multiply both sides of the equation be – 1. In either case, we end
up with an equation that has the standard form,

Ax + By = C

where, if at all possible, A, B, and C are integers, and A is non-negative, and, A, B, and C have
no common factors other than 1. Let us look at a few examples.

The first example will be the most complex. We start with

y = (5/6)x + 7/4

Our first step is to multiply both sides by the least common multiple of 6 and 4, namely 12.

12y = 12( (5/6)x + 7/4 )

12y = (12)(5/6)x + (12)(7/4)

12y = 10x + 21

Now we have removed all fractions, we bring the x term to the left side.
–
10x + y = 21

And, because the coeeficient of x is negative, we multiply both sides by – 1.

 (– 1)(– 10x +12y) = (– 1)(21)

This produces the desired result, the standard form

10x + – 12y = – 21

Another example would be to start with

y = (1/3)x + 5/6

The least common multiple of 3 and 6 is 6, so we multiply both sides by 6 to clear the fractions

6y = 6 ( (1/3)x + 5/6)

6y = (6)( (1/3)x + (6)(5/6)

6y = 2x + 5

We add – 2x to both sides to give

–
2x + 6y = + 5

And we make the leading coefficient of x be positive by multiplying both sides by – 1.

(– 1)(– 2x + 6y ) = (– 1)(5)

(– 1)(– 2x) + (– 1)(6)y ) = – 5

2x + – 6y = – 5

which is in standard form.

A third example is the slope-intercept form

y = – 2x + 8

Here, we have no need to clear fractions. We can move the x term to the left side by adding 2x to
both sides.

2x +y = 8

And this is already in standard form.

We have seen that we can transform slope-intercept form equations into standard form equations.
But why should we want to do this? There are a number of reasons. First, standard form allows
us to write the equations for vertical lines, which is not possible in slope-intercept form.
Remember that vertical lines have an undefined slope (which is why we can not write them in
slope-intercept form). However, the vertical line through the point (4,7) has the standard form
equation

1x+0y=4

which we could write in the even more simple form

x=4

[Note that the horizontal line through the point (4,7) has the slope-intercept form y=0x+7, and
the standard form 0x+1y=7. This example demonstrates why we ask for the leading coefficient
of x to be "non-negative" instead of asking for it to be "positive". For horizontal lines, that
coefficient of x must be zero.]

A second reason for putting equations into standard form is that it allows us to employ a
technique for solving systems of linear equations. This topic will not be covered until later in the
course so we do not need standard form at this point. However it will become quite useful later.

A third reason to use standard form is that it simplifies finding parallel and perpendicular lines.
Let us look at the typical parallel line problem. Find the equation of the line that is parallel to the
line 3x+4y=17 and that contains the point (2,8). The usual approach to this problem is to find
the slope of the given line and then to use that slope along with the given point in the point-slope
form for a linear equation. However, if we look at the standard form of a linear equation,

Ax + By = C

and we move the Ax term to the other side

By = – A x + C

and we divide both sides by B, assuming B is not zero, we get

y = (– A/B) x + C/B

which is the slope-intercept form. From that form we see that the slope is – A/B. Any line parallel
to the given line must have that same slope. Of course, the only values affecting the slope
are A and B from the original standard form. Therefore, as long as A and B do not change, any
line that has a standard form of

Ax + By = H

will be parallel to the line

Ax + By = C
If we return to the original problem, "Find the equation of the line that is parallel to the line
3x+4y=17 and that contains the point (2,8)" we can see that the answer must look like

3x + 4y = H

and we just need to find the value of H. Of course, we also know that the point (2,8) must make
the equation true, so

3(2) + 48 = H

must be true. But this means that we have

6 + 32 = H

or

38 = H

Because we know the value of H, we have the complete answer

3x + 4y = 38

Any time we are given a standard form linear equations and we are asked to find the equation of
a parallel line through a given point, we know that the answer will look just like the original
equation, but it will have a different constant value. We can find that value by making the
equation work when we substitute the intitial point into the equations. "Find the equation of the
line that is parallel to the line 2x+– 5y=– 19 and that contains the point (4,– 7)" The answer must
look line

2x + – 5y = H

and (4,– 7) must make it true. Therefore,

2(4) + – 5(– 7) = H

8 + 35 = H

43 = H

and that gives us the solution

2x + – 5y = 43

We have seen that parallel lines in the standard form

Ax + By = C
have the values of A and B stay the same. What do we know about perpendicular lines? Let us
look at two lines, in standard form

Ax + By = C

Bx + – Ay = D

The slope of the first is – A/B and the slope of the second is B/A. If we multiply these together
we get (– AB)/(AB) or – 1. These lines must be perpendicular. If we take our usual "perpendicular
line problem", "Find the equation of the line that is perpendicular to the line 3x+4y=17 and that
contains the point (2,8)" we know that the answer will have to look like

4x + – 3y = D

where we have switched the position of the "A" and "B" coefficients, and we have changed the
sign of one of them. Furthermore, we know that the point (2,8) will have to make this new
equation true, so

4(2) + – 3(8) = D

8 + – 24 = D
–
16 = D

Therefore, the answer is the standard form linear equation

4x + – 3y = – 16

"Find the equation of the line that is perpendicular to the line 2x+– 5y=– 19 and that contains the
point (4,– 7)." The answer must look line

5x + 2y = D

we just reverse the positions of the "A" and "B" values and we reverse the sign of one of them,
and (4,– 7) must make it true. Therefore,

5(4) + 2(– 7) = D

20 + – 14 = D

6=D

and that gives us the solution

5x + 4y = 6
From the presentation above we can see that we can do the "usual" parallel and perpendicular
line problems in just a few steps without ever finding the slope of the original line and without
using the point-slope form of a linear equation.

For completeness we should check our methods with horizontal and vertical lines. The standard
form for a horizontal line is 0x+1y=C. Another horizontal line, one parallel to the first, will still
have the form 0x+1y=D. Therefore, our rule for finding parallel lines will still work. We just
leave the "A" and "B" values the same and find a new value for "D" by substituting the
coordinates of the external point.

The standard form for a vertical line is 1x+0y=C. Another vertical line, one parallel to the first,
will still have the form 1x+0y=D. Therefore, our rule for finding parallel lines will still work. We
just leave the "A" and "B" values the same and find a new value for "D" by substituting the
coordinates of the external point.

If we start with a horizontal line in the form 0x+1y=C, and we reverse the "A" and "B" values,
and reverse the sign of one of them, we get 1x+0y=D, which is the general form of a vertical
line. This corresponds to our method for finding perpendicular lines.

Basic Mathematics And Optimization

The optimization algorithms we have discussed so far are all unconstrained problems. The next
three sections deal with constrained problems. Many of the constrained problems are derived
from theoretical models where the solution is found by finding the configuration where a
certain quantity reaches a maximum or a minimum.

In order to facilitate working with such models, the Extreme Optimizaiton Numerical
Libraries for .NET includes a framework for defining such models that allows for easier
bookkeeping and managing the results.

Components of an Optimization Model

All classes that implement optimization problems with constraints inherit from an abstract base
class, OptimizationModel.

An optimization model has three main components:

1. An objective function. This is the function that needs to be optimized.

2. A collection of decision variables. The solution to the optimization problem is the set of
values of the decision variables for which the objective function reaches its optimal
value.
3. A collection of constraints that restrict the values of the decision variables.
The OptimizationModel class provides a common API for defining and accessing variables and
constraints, as well as other properties of each model.

We will now discuss each of these components in more detail.

Types of Optimization Models

Optimization problems can be classified in terms of the nature of the objective function and the
nature of the constraints. Special forms of the objective function and the constraints give rise to
specialized algorithms that are more efficient. From this point of view, there are four types of
optimization problems, of increasing complexity.

An Unconstrained optimization problem is an optimization problem where the objective

function can be of any kind (linear or nonlinear) and there are no constraints. These types of
problems are handled by the classes discussed in the earlier sections.

A linear program is an optimization problem with an objective function that is linear in the
variables, and all constraints are also linear. Linear programs are implemented by
the LinearProgram class.

A quadratic program is an optimization problem with an objective function that is quadratic in

the variables (i.e. it may contain squares and cross products of the decision variables), and all
constraints are linear. A quadratic program with no squares or cross products in the objective
function is a linear program. Quadratic programs are implemented by
the QuadraticProgramclass.

A nonlinear program is an optimization problem with an objective function that is an arbitrary

nonlinear function of the decision variables, and the constraints can be linear or nonlinear.
Nonlinear programs are implemented by the NonlinearProgram class.

Decision Variables

Finding the optimal values of the decision variables is the goal of solving an optimization
model. In the optimization framework, variables are implemented by
the DecisionVariable class.

Each variable has a Name, which may be generated automatically.

The LowerBound and UpperBound properties specify lower and upper bounds for the values
the variable can take. After the solution of the model has been computed, the Value property
returns the value of the variable in the optimal solution.

DecisionVariable objects are created by calling one of the overloads of the optimization
model's AddVariable method. In some cases, they may also be created automatically.
An optimization model's variables can be accessed through its Variables property. Individual
variables can be accessed by name or by position. The variable's Position property returns the
variable's index in the collection.

Specific models may have specialized versions of the decision variables. For example, both
linear and quadratic programs use variables of type LinearProgramVariable, which inherits
from DecisionVariable and has an extra Cost property that represents the coefficient of the
variable in the linear portion of the objective function.

Constraints

Constraints limit the possible values for the decision variables in an optimization model. There
are several types of constraints. The classes that implement them all inherit from
the Constraint class.

Each constraint also has a Name, which may again be generated automatically.
The LowerBound and UpperBound properties specify lower and upper bounds for the value of
the constraint. After the solution of the model has been computed, the Valueproperty returns
the value of the constraint in the optimal solution.

There are two types of constraints: linear and nonlinear.

Linear constraints express that a linear combination of the decision variables must lie within a
certain range. They are implemented by the LinearConstraint class. The coefficients can be
accessed through the Gradient property. LinearConstraintobjects are created by calling one of
the overloads of the optimization model's AddLinearConstraint method. In some cases, they
may also be created automatically.

Nonlinear constraints express that the value of some arbitrary function of the decision variables
must lie within a certain range. They are implemented by the NonlinearConstraint class. The
constraint function can be accessed through the ConstraintFunction. The gradient of this
function, which is needed during the optimization process, is the FastConstraintGradient If it is
not supplied, a numerical approximation is used. NonlinearConstraint objects are created by
calling one of the overloads of the optimization model's AddNonlinearConstraint method.

An optimization model's constraints can be accessed through its Constraints property.

Individual constraints can be accessed by name or by position. The variable's Position property
returns the constraint's index in the collection.

Question 1c:
An enclosure at a reserve is divided into two rectangular cages.

One side of the enclosure is a solid wall.

There is a fence around the rest of the enclosure and another between the male and female cages.
The total length of the fences is 275 m.
5 male birds are kept in the smaller cage and 8 female birds in the larger cage.
The males have an average area of 250 sq.m each in their cage.
The area of the female cage is to be maximised.

Find the average area for each of the female birds in their cage.

Question 2d:
The diagram below shows part of the graph of the function y=x2,x>0.
The shaded region between the curve and the X-axis, and between x=m and x=m+2, has an area

of 516 sq.units. Find the value of m.

Explain the choice of the solution.

2 Answers

From the picture above, the total length of the fence is 3a+b+c. We are given that
this is 275. Hence, we the first constraint
 3a+b+c=275.

We are further given that the 5 males have an average area of 250 in their cage.
This gives us that
Ac/5=250⟹ac=1250.

We now want to maximize the average area for each of the female birds in their
cage i.e. we want to maximize ab5 subject to these constraints. Maximizing ab5 is
equivalent to maximizing ab. Hence, the optimization problem is
Maximize ab subject to 3a+b+c=275 and ac=1250.

Eliminating b using the first constraint gives us b=275−3a−c. Hence the function
we want to maximize becomes
275a−3a2−ac

subject to ac=1250. Plugging in the value of ac, we get that we want to maximize

275a−3a2−1250.

The maximum for this function can be found easily by completing the squares (or)
by differentiation and the maximum occurs at
a=275/6

giving a maximum of
60625/12.

Hence, the average area for each of the female birds in their cage is 60625/96.

1. The Assignment Problem

Introduction to Optimization
Everyone who studied calculus knows that an extremum of a smooth function is reached at a
stationary point where its gradient vanishes. Some may also remember the Weierstrass theorem
which proclaims that the minimum and the maximum of a function in a closed finite domain do
exist. Does this mean that the problem is solved?

A small thing remains: To actually find that maximum. This problem is the subject of
the optimization theory that deals with algorithms for search of the extremum. More precisely,
we are looking for an algorithm to approach a proximity of this maximum and we are allowed to
evaluate the function (to measure it) in a finite number of points. Below, some links to
mathematical societies and group in optimization are placed that testify how popular the
optimization theory is today: Many hundreds of groups are intensively working on it.
Optimization and Modeling
 The modelingof the optimizing process is conducted along with the optimization. Inaccuracy
of the model is emphasized in optimization problem, since optimization usually brings the
control parameters to the edge, where a model may fail to accurately describe the prototype. For
example, when a linearized model is optimized, the optimum often corresponds to infinite value
of the linearized control. (Click here to see an example) On the other hand, the roughness of the
model should not be viewed as a negative factor, since the simplicity of a model is as important
as the accuracy. Recall the joke about the most accurate geographical map: It is done in the 1:1
scale.

 Unlike the models of a physical phenomena, an optimization models critically depend on

designer's will. Firstly, different aspects of the process are emphasized or neglected depending
on the optimization goal. Secondly, it is not easy to set the goal and the specific constrains for
optimization.

Naturally, one wants to produce more goods, with lowest cost and highest quality. To optimize
the production, one either may constrain by some level the cost and the quality and maximize the
quantity, or constrain the quantity and quality and minimize the cost, or constrain the quantity
and the cost and maximize the quality. There is no way to avoid the difficult choice of the values
of constraints. The mathematical tricks go not farther than: "Better be healthy and wealthy than
poor and ill". True, still not too exciting.

 The maximization of the monetary profit solves the problem to some extent by applying an
universal criterion. Still, the short-term and long-term profits require very different strategies;
and it is necessary to assign the level of insurance, to account for possible market variations, etc.
These variables must be a priori assigned by the researcher.

 Sometimes, a solution of an optimization problem shows unexpected features: for example,

an optimal trajectory zigzags infinitely often. Such behavior points to an unexpected, but optimal
behavior of the solution. It should not be rejected as a mathematical extravaganza, but thought
through!

 On the other hand, the roughness of the model should not be viewed as a negative factor,
since the simplicity of a model is as important as the accuracy. Recall the joke about the most
accurate geographical map: It is done in the 1:1 scale.
 Unlike the models of a physical phenomena, an optimization models critically depend on
designer's will. Firstly, different aspects of the process are emphasized or neglected depending
on the optimization goal. Secondly, it is not easy to set the goal and the specific constrains for
optimization.
Naturally, one wants to produce more goods, with lowest cost and highest quality. To optimize
the production, one either may constrain by some level the cost and the quality and maximize the
quantity, or constrain the quantity and quality and minimize the cost, or constrain the quantity
and the cost and maximize the quality. There is no way to avoid the difficult choice of the values
of constraints. The mathematical tricks go not farther than: "Better be healthy and wealthy than
poor and ill". True, still not too exciting.
 The maximization of the monetary profit solves the problem to some extent by applying an
universal criterion. Still, the short-term and long-term profits require very different strategies;
and it is necessary to assign the level of insurance, to account for possible market variations, etc.
These variables must be a priori assigned by the researcher.

 Sometimes, a solution of an optimization problem shows unexpected features: for example,

an optimal trajectory zigzags infinitely often. Such behavior points to an unexpected, but optimal
behavior of the solution. It should not be rejected as a mathematical extravaganza, but thought
through!

Special Linear Programming Problems

A short explanation is given what Linear programming is and some basic knowledge you need to
know.

A linear programming problem is mathematically formulated as follows:

A linear function to be maximized or minimized

e.g.
maximize c1 x1 + c2 x2

Problem constraints of the following form

e.g.
a11 x1 + a12 x2 <= b1
a21 x1 + a22 x2 <= b2
a31 x1 + a32 x2 <= b3

Default lower bounds of zero on all variables.

The problem is usually expressed in matrix form, and then becomes:

maximize CT x
subject to A x <= B
x >= 0

So a linear programming model consists of one objective which is a linear equation that must be
maximized or minimized. Then there are a number of linear inequalities or constraints.

cT, A and B are constant matrixes. x are the variables (unknowns). All of them are real, continue
values.
Note the default lower bounds of zero on all variables x. People tend to forget this build-in
default. If no negative (or negative infinite) lower bound is explicitely set on variables, they can
and will take only positive (zero included) values.

The inequalities can be <=, >= or =

Because all numbers are real values, <= is the same as < and >= is the same as >

Also note that both objective function and constraints must be linear equations. This means that
no variables can be multiplied with each other.

This formulation is called the Standard form. It is the usual and most intuitive form of describing
a linear programming problem.

Example:

minimize 3 x1 - x2
subject to -x1 + 6 x2 - x3 + x4 >= -3
7 x2 + 2 x4 = 5
x1 + x2 + x3 = 1
x3 + x4 <= 2

Sometimes, these problems are formulated in the canonical form. All inequalities are converted
to equalities by adding an extra variable where needed:

maximize CT x
subject to A x = B
x >= 0

Above example can then be written as:

minimize 3 x1 - x2
subject to -x1 + 6 x2 - x3 + x4 - s = -3
7 x2 + 2 x4 = 5
x1 + x2 + x3 = 1
x3 + x4 + t = 2

So everywhere an equality was specified, an extra variable is introduced and subtracted (if it was
>) or added (if it was <) to the constraint. These variables also only take positive (or zero) values
only. These extra variables are called slack or surplus variables.

lp_solve add's these variables automatically to its internal structure. The formulator doesn't have
to do it and it is even better not to. There will be fewer variables in the model and thus quicker to
solve.

See Formulation of an lp problem in lpsolve for a practical example.

The right hand side (RHS), the B-vector, must be a constant matrix. Some people see this as a
problem, but it isn't The RHS can always be brought to the left by a simple operation:

A x <= B
Is equal to:

A x - B <= 0
So if B is not constant, just do that.

Basic mathematics also states that if a constraint is multiplied by a negative constant, that the
inequality changes from direction. For example:

5 x1 - 2 x2 >= 3
If multiplied by -1, it becomes:

-5 x1 + 2 x2 <= -3

If the objective is multiplied by -1, then maximization becomes minimization and the other way
around. For example:

minimize 3 x1 - x2
Can also be written as:

maximize -3 x1 + x2

The result will be the same, but changed from sign.

Bounds

Minima and maxima on single variables are special cases of restrictions. They are called bounds.
The optimization algorithm can handle these bounds more effeciently than other restrictions.
They consume less memory and the algorithm is faster with them. As already specified, there is
by default an implicit lower bound of zero on each variable. Only when explicitly another lower
bound is set, the default of 0 is overruled. This other bound can be negative also. There is no
default upper bound on variables. Almost all solvers support bounds on variables. So does
lp_solve.

Ranges

Frequently, it happens that on the same equation a less than and a greater than restriction must be
set. Instead of adding two extra restrictions to the model, it is more performant and less memory
consument to only add one restiction with either the less than or greater than restriction and put
the other inequality on that same constraint by means of a range. Not all solvers support this
feature but lp_solve does.
Integer and binary variables

By default, all variables are real. Sometimes it is required that one or more variables must be
integer. It is not possible to just solve the model as is and then round to the nearest solution. At
best, this result will maybe furfill all constraints, but you cannot be sure of. As you cannot be
sure of the fact that this is the most optimal solution. Problems with integer variables are called
integer or descrete programming problems. If all variables are integer it is called a pure integer
programming problem, else it is a mixed integer programming problem. A special case of integer
variables are binary variables. These are variables that can only take 0 or 1 as value. They are
used quite frequently to program discontinue conditions. lp_solve can handle integer and binary
variables. Binary variables are defined as integer variables with a maximum (upper bound) of 1
on them. See integer variables for a description on them.

Semi-continuous variables

Semi-continuous variables are variables that must take a value between their minimum and
maximum or zero. So these variables are treated the same as regular variables, except that a
value of zero is also accepted, even if there is a minimum bigger than zero is set on the variable.
See semi-continuous variables for a description on them.

Special ordered sets (SOS)

A specially ordered set of degree N is a collection of variables where at most N variables may be
non-zero. The non-zero variables must be contiguous (neighbours) sorted by the ascending value
of their respective unique weights. In lp_solve, specially ordered sets may be of any cardinal
type 1, 2, and higher, and may be overlapping. The number of variables in the set must be equal
to, or exceed the cardinal SOS order. See Special ordered sets (SOS) for a description on them.

lp_solve uses the simplex algorithm to solve these problems. To solve the integer restrictions, the
branch and bound (B&B) method is used.

A transport company has two types of trucks, Type A and Type B. Type
A has a refrigerated capacity of 20 m 3 and a non-refrigerated capacity of 40
m 3 while Type B has the same overall volume wit h equal sections for
refrigerated and non-refrigerated stock. A grocer needs to hire trucks for the
transport of 3,000 m 3 of refrigerated stock and 4,000 m 3 of non-refrigerated
stock. The cost per kilometer of a Type A is $30, and $40 for Type B. How
many trucks of each type should the grocer rent to achieve the minimum total
cost?
 1Choose the unknowns.

x = Type A trucks

y = Type B trucks

2Write the objective function.

f(x,y) = 30x + 40y

3Write the constraints as a system of inequalities.

A B Total

Refrigerated 20 30 3 000

Non-refrigerated 40 30 4 000

20x + 30y ≥ 3 000

40x + 30y ≥ 4 000

x≥0

y≥0

4 Find the set of feasible solutions that graphically represent the

constraints.
 5 Calculate the coordinates of the vertices from the compound of
feasible solutions.

6 Calculate the value of the objective function at each of the

vertices to determine which of them has the maximum or minimum values.

f(0, 400/3) = 30 · 0 + 40 · 400/3 = 5,333.332

f(150, 0) = 30 · 150 + 40 · 0 = 4,500

As x and y must be natural numbers round the value of y.

f(50, 67) = 30 · 50 + 40 ·67 = 4,180 Minimum

The minimum cost is $4,180. To achieve this 50 trucks of Type A and

67 trucks of Type B are needed.

Linear Programming (LP) Problem Example

A linear programming problem is Here is an example of an LP problem:

one in which we are to find the
maximum or minimum value of a Find the maximum value of
linear expression
p = 3x - 2y + 4z
ax + by + cz + . . .
subject to
(called the objective function),
subject to a number of 4x + 3y - z ≥ 3
linear constraints of the form x + 2y + z ≤ 4
x ≥ 0, y ≥ 0, z ≥ 0
Ax + By + Cz + . . .≤ N
The objective function is p = 3x - 2y + 4z. The
or constraints are

Ax + By + Cz + . . .≥ N. 4x + 3y - z ≥ 3
x + 2y + z ≤ 4
The largest or smallest value of the
x ≥ 0, y ≥ 0, z ≥ 0.
objective function is called
the optimal value, and a collection of Q Wait a minute! Why can't I simply choose, say, z to
values of x, y, z, . . . that gives the be really large (z = 1,000,000 say) and thereby make p
optimal value constitutes an optimal as large as I want?
solution. The variables x, y, z, . . . are A You can't because
Select one
called the decision variables.

Sketching the Solution Set of a Example

Linear Inequality
To sketch the linear inequality
To sketch the region represented by a
linear inequality in two variables: 3x - 4y ≤ 12,

A. Sketch the straight line obtained by first sketch the line 3x - 4y = 12.
replacing the inequality with an
equality.

B. Choose a test point not on the line

((0,0) is a good choice if the line does
not pass through the origin, and if the
line does pass through the origin a
point on one of the axes would be a
good choice).

C. If the test point satisfies the

inequality, then the set of solutions is
the entire region on the same side of
the line as the test point. Otherwise it
is the region on the other side of the
line. In either case, shade out the side
that does not contain the solutions,
leaving the solution region showing. Next, choose the origin (0, 0) as the test point (since it is
not on the line). Substituting x=0, y=0 in the inequality
gives

3(0) - 4(0) ≤ 12.

Since this is a true statement, (0, 0) is in the solution set,

so the solution set consists of all points on the same side
as (0, 0). This region is left unshaded, while the (grey)
shaded region is blocked out.

Feasible Region Example

The feasible region determined by a The feasible region for the following collection of
collection of linear inequalities is the inequalities is the unshaded region shown below
collection of points that satisfy all of (including its boundary).
the inequalities.
3x - 4y ≤ 12,
To sketch the feasible region x + 2y ≥ 4
determined by a collection of linear x≥1
inequalities in two variables: Sketch
y ≥ 0.
the regions represented by each
inequality on the same graph,
remembering to shade the parts of the
plane that you do not want. What is
unshaded when you are done is the
feasible region.

Graphical Method Example

The graphical method for solving Minimize C = 3x + 4y subject to the constraints

linear programming problems in two
unknowns is as follows. 3x - 4y ≤ 12,
x + 2y ≥ 4
A. Graph the feasible region. x ≥ 1, y ≥ 0.
B. Compute the coordinates of the
corner points. The feasible region for this set of constraints was shown
C. Substitute the coordinates of the above. Here it is again with the corner points shown.
corner points into the objective
function to see which gives the
optimal value.
D. If the feasible region is not
bounded, this method can be
misleading: optimal solutions always
exist when the feasible region is
bounded, but may or may not exist
when the feasible region is
unbounded. The textbook shows a
straightforward way for determining
whether optimal solutions exist in the The following table shows the value of C at each corner
case of unbounded feasible regions. point:

If you want to see a utility that

automates the whole process, try Point C = 3x + 4y
our Linear Programming Grapher. It
does everything automatically!
(1, 1.5) 3(1)+4(1.5) = 9 minimum

(4, 0) 3(4)+4(0) = 12
 Therefore, the solution is x = 1, y = 1.5, giving the
minimum value C = 9.

Top of Page

Standard Maximization Problem Examples

A standard maximization The following is a standard LP problem:

problem in n unknowns is a linear
programming problem in which we Maximize P = 2x - 3y + z subject to
are required to maximize (not
minimize) the objective function, 4x - 3y + z ≤ 3
subject to constraints of the form x + y + z ≤ 10
x ≥ 0, y ≥ 0, z ≥ 0.
x≥ 0, y≥ 0, z≥ 0, . . . ,
The following is not a standard LP problem:
and further constraints of the form
Maximize P = 2x - 3y + z subject to
Ax + By + Cz + . . . ≤ N,
4x - 3y + z ≥ 3
where A, B, C, . . . and N are numbers x + y + z ≤ 10
with N nonnegative. x ≥ 0, y ≥ 0, z ≥ 0.

Note that the inequality here must be a

"≤," and not "=" or "≥."

Simplex Method for Standard

Maximization Problem

To solve a standard maximization

problem using the simplex method,
we take the following steps:

Step 1. Convert to a system of

equations by introducing slack
variables to turn the constraints into
equations, and rewriting the objective
function in standard form.

Step 2. Write down the

initial tableau.

Step 3. Select the pivot column:

Choose the negative number with the
largest magnitude in the bottom row
(excluding the rightmost entry). Its
column is the pivot column. (If there
are two candidates, choose either one.)
If all the numbers in the bottom row
are zero or positive (excluding the
rightmost entry), then you are done:
the basic solution maximizes the
objective function (see below for the
basic solution).

Step 4. Select the pivot in the pivot

column: The pivot must always be a
positive number. For each positive
entry b in the pivot column, compute
the ratio a/b, where a is the number in
the Answer column in that row. Of
these test ratios, choose the smallest
one. The corresponding number b is
the pivot.

Step 5. Use the pivot to clear the

column in the normal manner (taking
care to follow the exact prescription
for formulating the row operations
described in Chapter 2) and then
relabel the pivot row with the label
from the pivot column. The variable
originally labeling the pivot row is
the departing or exiting variable and
the variable labeling the column is
the entering variable.

Step 6. Go to Step 3.

Basic Solution Example

To get the basic In the following tableau

solution corresponding to any tableau
in the simplex method, set to zero all
x y z s t u p Ans
variables that do not appear as row
labels (these are
the inactive variables).

The value of a variable that does -1 0 0 1 0 0 0 4

appear as a row label (an active
variable) is the number in the
rightmost column in that row divided
1 0 3 0 0 8 0 12
by the number in that row in the
column labeled by the same variable.
4 0 0 0 3 0 0 2

-5 2 0 0 0 6 0 4

6 0 0 0 0 0 5 -25

the basic solution is

x = 0, y = 2, z = 4, s = 4, t = 2/3, u = 0, p = -5,

and the active variables are y, z, s, t, and p.

Top of Page

Nonstandard Constraints

To solve a linear programming problem with constraints of the form Ax + By + . . .≥ N with N

positive, subtract a surplus variable from the left-hand side, rather than adding a slack variable.
The basic solution corresponding to the initial tableau will not be feasible since some of the
active variables will be negative, and so the rules for the initial pivoting are different from those
above.

Star all rows that give a negative value for the associated active variable (except for the objective
variable, which is allowed to be negative). If there are starred rows, you will need to begin
withPhase I:

Phase I: Getting into the Feasible Region (Getting Rid of the Stars) In the first starred row,
find the largest positive number. Use test ratios as in the preceding section to find the pivot in
that column (exclude the bottom row), then pivot on that entry. If the lowest ratio occurs both in
a starred row and an unstarred row, pivot in a starred row rather than the unstarred one. Repeat
until no starred rows remain, then go on to Phase II.

Phase II: Use the Simplex Method for Standard Maximization Problems If there are any
negative entries on the left side of the bottom row after Phase I, use the method described above
for solving standard maximization problems.

For some on-line interactive examples, visit the tutorial for general linear programming
problems.
Simplex Method for Minimization Example
Problem
The minimization LP problem:
To solve a minimization problem
using the simplex method, convert it Minimize C = 3x + 4y - 8z subject to the constraints
into a maximization problem. If you
need to minimize c, instead maximize 3x - 4y ≤ 12,
p = -c. x + 2y + z ≥ 4
4x - 2y + 5z ≤ 20
x ≥ 0, y ≥ 0, z ≥ 0

can be replaced by the

following maximization problem:

Maximize P = -3x - 4y + 8z subject to the constraints

3x - 4y ≤ 12,
x + 2y + z ≥ 4
4x - 2y + 5z ≤ 20
x ≥ 0, y ≥ 0, z ≥ 0.

Solving a Matrix Game Using the Simplex Method

A game may be solved using the simplex method as follows.

Before you start, check for saddle points. If you find one, you have already solved the game; the
optimal strategies are the pure strategies passing through a saddle point. Otherwise, continue
with the following steps.

1. Reduce the payoff matrix by dominance.

2. Add a fixed number k to each of the entries so that they all become non-negative.
3. Set up and solve the associated linear programming problem using the simplex method.
4. Obtain the optimal strategies and the expected value as follows:

Column Strategy

a. Express the solution to the linear programming problem as a column vector.

b. Normalize by dividing each entry of the solution vector by the sum of the entries,
or by the value of the objective variable p.
 c. Insert zeros in positions corresponding to the columns deleted during reduction.

Row Strategy

d. Write down the row vector whose entries are the numbers appearing in the bottom
row of the final tableau underneath the slack variables.
e. Normalize by dividing each entry of the solution vector by the sum of the entries.
f. Insert zeros in positions corresponding to the rows deleted during reduction.

Value of the Game

e = 1/p - k.

2. The Assignment Problem

Assignment Problem

A typical assignment problem, presented in the classic manner, is shown in Fig. 12. Here there
are five machines to be assigned to five jobs. The numbers in the matrix indicate the cost of
doing each job with each machine. Jobs with costs of M are disallowed assignments. The
problem is to find the minimum cost matching of machines to jobs.

Figure 12. Matrix model of the assignment problem.

The network model is in Fig. 13. It is very similar to the transportation model except the external
flows are all +1 or -1. The only relevant parameter for the assignment model is arc cost (not
shown in the figure for clarity) ; all other parameters should be set to default values. The
assignment network also has the bipartite structure.
 Figure 13. Network model of the assignment problem.

The solution to the assignment problem as shown in Fig. 14 has a total flow of 1 in every column
and row, and is the assignment that minimizes total cost.

Figure 14. Solution to the assignment Problem

Description of the assignment problem

In the classical assignment problem, the goal is to find an optimal assignment of agents to tasks
without assigning an agent more than once and ensuring that all tasks are completed. The
objective might be to minimize the total time to complete a set of tasks, or to maximize skill
ratings, or to minimize the cost of the assignments.

The assignment problem is a particular class of transportation linear programming problems with
the supplies and demands equal to integers (often 1). Since all supplies, demands, and bounds on
variables are integral, the assignment problem relies on a nice property of transportation
problems that the optimal solution will be entirely integral. As you will see, the goal of the
objective of the assignment problem (unlike transportation problems) may not have anything to
do at all with moving anything!

In this article we'll deal with one optimization problem, which can be informally defined as:
Assume that we have N workers and N jobs that should be done. For each pair (worker, job) we
know salary that should be paid to worker for him to perform the job. Our goal is to complete all
jobs minimizing total inputs, while assigning each worker to exactly one job and vice versa.

Converting this problem to a formal mathematical definition we can form the following
equations:
- cost matrix, where cij - cost of worker i to perform job j.
- resulting binary matrix, where xij = 1 if and only if ith worker is assigned to jth job.

- one worker to one job assignment.

- one job to one worker assignment.

- total cost function.

We can also rephrase this problem in terms of graph theory. Let's look at the job and workers as
if they were a bipartite graph, where each edge between the ithworker and jth job has weight of cij.
Then our task is to find minimum-weight matching in the graph (the matching will consists
of N edges, because our bipartite graph is complete).

Small example just to make things clearer:

General description of the algorithm

This problem is known as the assignment problem. The assignment problem is a special case of
the transportation problem, which in turn is a special case of the min-cost flow problem, so it can
be solved using algorithms that solve the more general cases. Also, our problem is a special case
of binary integer linear programming problem (which is NP-hard). But, due to the specifics of
the problem, there are more efficient algorithms to solve it. We'll handle the assignment problem
with the Hungarian algorithm (or Kuhn-Munkres algorithm). I'll illustrate two different
implementations of this algorithm, both graph theoretic, one easy and fast to implement
with O(n4) complexity, and the other one with O(n3) complexity, but harder to implement.
There are also implementations of Hungarian algorithm that do not use graph theory. Rather,
they just operate with cost matrix, making different transformation of it (see [1] for clear
explanation). We'll not touch these approaches, because it's less practical for TopCoder needs.

O(n4) algorithm explanation

As mentioned above, we are dealing with a bipartite graph. The main idea of the method is the
following: consider we've found the perfect matching using only edges of weight 0 (hereinafter
called "0-weight edges"). Obviously, these edges will be the solution of the assignment problem.
If we can't find perfect matching on the current step, then the Hungarian algorithm changes
weights of the available edges in such a way that the new 0-weight edges appear and these
changes do not influence the optimal solution.

To clarify, let's look at the step-by-step overview:

Step 0)

A. For each vertex from left part (workers) find the minimal outgoing edge and subtract its
weight from all weights connected with this vertex. This will introduce 0-weight edges (at least
one).

B. Apply the same procedure for the vertices in the right part (jobs).

Actually, this step is not necessary, but it decreases the number of main cycle iterations.

Step 1)

A. Find the maximum matching using only 0-weight edges (for this purpose you can use max-
flow algorithm, augmenting path algorithm, etc.).

B. If it is perfect, then the problem is solved. Otherwise find the minimum vertex cover V (for
the subgraph with 0-weight edges only), the best way to do this is to use Köning's graph theorem.
Step 2) Let and adjust the weights using the following rule:

Step 3) Repeat Step 1 until solved.

But there is a nuance here; finding the maximum matching in step 1 on each iteration will cause
the algorithm to become O(n5). In order to avoid this, on each step we can just modify the
matching from the previous step, which only takes O(n2) operations.

It's easy to see that no more than n2 iterations will occur, because every time at least one edge
becomes 0-weight. Therefore, the overall complexity is O(n4).

O(n3) algorithm explanation

Warning! In this section we will deal with the maximum-weighted matching problem. It's
obviously easy to transform minimum problem to the maximum one, just by setting:
or

Before discussing the algorithm, let's take a look at some of the theoretical ideas. Let's start off
by considering we have a complete bipartite graph G=(V,E)where
and , w(x,y) - weight of edge (x,y).

Vertex and set neighborhood

Let . Then is v's neighborhood, or all vertices that share an

edge with v.

Let . Then is S's neighborhood, or all vertices that share an edge

with a vertex in S.

Vertex labeling

This is simply a function (for each vertex we assign some number called a label).
Let's call this labeling feasible if it satisfies the following
condition: . In other words, the sum of the labels of the
vertices on both sides of a given edge are greater than or equal to the weight of that edge.

Equality subgraph

Let Gl=(V,El) be a spanning subgraph of G (in other words, it includes all vertices from G).
If G only those edges (x,y) which satisfy the following
condition: , then it is an equality subgraph.
In other words, it only includes those edges from the bipartite matching which allow the vertices
to be perfectly feasible.

Now we're ready for the theorem which provides the connection between equality subgraphs and
maximum-weighted matching:

If M* is a perfect matching in the equality subgraph Gl, then M* is a maximum-weighted

matching in G.

The proof is rather straightforward, but if you want you can do it for practice. Let's continue with
a few final definitions:
Alternating path and alternating tree

Consider we have a matching M ( ).

Vertex is called matched if , otherwise it is

called exposed (free, unmatched).

(In the diagram below, W1, W2, W3, J1, J3, J4 are matched, W4, J2 are exposed)

Path P is called alternating if its edges alternate between M and E\M. (For example,
(W4, J4, W3, J3, W2, J2) and (W4, J1, W1) are alternating paths)

If the first and last vertices in alternating path are exposed, it is called augmenting (because we
can increment the size of the matching by inverting edges along this path, therefore matching
unmatched edges and vice versa). ((W4, J4, W3, J3, W2, J2) - augmenting alternating path)

A tree which has a root in some exposed vertex, and a property that every path starting in the
root is alternating, is called an alternating tree. (Example on the picture above, with root in W4)

That's all for the theory, now let's look at the algorithm:

First let's have a look on the scheme of the Hungarian algorithm:

Step 0. Find some initial feasible vertex labeling and some initial matching.

Step 1. If M is perfect, then it's optimal, so problem is solved. Otherwise, some exposed
exists; set . (x - is a root of the alternating tree we're going to build). Go to step
2.

Step 2. If go to step 3, else . Find

 (1)

and replace existing labeling with the next one:

(2)

Now replace with

Step 3. Find some vertex . If y is exposed then an alternating path from x (root
of the tree) to y exists, augment matching along this path and go to step 1. If y is matched
in M with some vertex z add (z,y) to the alternating tree and set , go
to step 2.

And now let's illustrate these steps by considering an example and writing some code.

As an example we'll use the previous one, but first let's transform it to the maximum-weighted
matching problem, using the second method from the two described above. (See Picture 1)

Picture 1

Here are the global variables that will be used in the code:

#define N 55 //max number of vertices in one part

#define INF 100000000 //just infinity

int cost[N][N]; //cost matrix

int n, max_match; //n workers and n jobs
int lx[N], ly[N]; //labels of X and Y parts
int xy[N]; //xy[x] - vertex that is matched with x,
int yx[N]; //yx[y] - vertex that is matched with y
bool S[N], T[N]; //sets S and T in algorithm
int slack[N]; //as in the algorithm description
int slackx[N]; //slackx[y] such a vertex, that
// l(slackx[y]) + l(y) - w(slackx[y],y) = slack[y]
int prev[N]; //array for memorizing alternating paths

Step 0:

It's easy to see that next initial labeling will be feasible:

And as an initial matching we'll use an empty one. So we'll get equality subgraph as on Picture 2.
The code for initializing is quite easy, but I'll paste it for completeness:

void init_labels()
{
memset(lx, 0, sizeof(lx));
memset(ly, 0, sizeof(ly));
for (int x = 0; x < n; x++)
for (int y = 0; y < n; y++)
lx[x] = max(lx[x], cost[x][y]);
}

The next three steps will be implemented in one function, which will correspond to a single
iteration of the algorithm. When the algorithm halts, we will have a perfect matching, that's why
we'll have n iterations of the algorithm and therefore (n+1) calls of the function.

Step 1
According to this step we need to check whether the matching is already perfect, if the answer is
positive we just stop algorithm, otherwise we need to clear S, Tand alternating tree and then find
some exposed vertex from the X part. Also, in this step we are initializing a slack array, I'll
describe it on the next step.

void augment() //main function of the algorithm

{
if (max_match == n) return; //check wether matching is already perfect
int x, y, root; //just counters and root vertex
int q[N], wr = 0, rd = 0; //q - queue for bfs, wr,rd - write and read
//pos in queue
memset(S, false, sizeof(S)); //init set S
memset(T, false, sizeof(T)); //init set T
memset(prev, -1, sizeof(prev)); //init set prev - for the alternating tree
for (x = 0; x < n; x++) //finding root of the tree
if (xy[x] == -1)
{
q[wr++] = root = x;
prev[x] = -2;
S[x] = true;
break;
}

for (y = 0; y < n; y++) //initializing slack array

{
slack[y] = lx[root] + ly[y] - cost[root][y];
slackx[y] = root;
}

Step 2

On this step, the alternating tree is completely built for the current labeling, but the augmenting
path hasn't been found yet, so we need to improve the labeling. It will add new edges to the
equality subgraph, giving an opportunity to expand the alternating tree. This is the main idea of
the method; we are improving the labeling until we find an augmenting path in the equality
graph corresponding to the current labeling. Let's turn back to step 2. There we just change
labels using formulas (1) and (2), but using them in an obvious manner will cause the algorithm
to have O(n4) time. So, in order to avoid this we use a slack array initialized inO(n) time because
we only augment the array created in step 1:

Then we just need O(n) to calculate a delta Δ (see (1)):

Updating slack:
1) On step 3, when vertex x moves from X\S to S, this takes O(n).
2) On step 2, when updating labeling, it's also takes O(n), because:

So we get O(n) instead of O(n2) as in the straightforward approach.

Here's code for the label updating function:

void update_labels()
{
int x, y, delta = INF; //init delta as infinity
for (y = 0; y < n; y++) //calculate delta using slack
if (!T[y])
delta = min(delta, slack[y]);
for (x = 0; x < n; x++) //update X labels
if (S[x]) lx[x] -= delta;
for (y = 0; y < n; y++) //update Y labels
if (T[y]) ly[y] += delta;
for (y = 0; y < n; y++) //update slack array
if (!T[y])
slack[y] -= delta;
}

Step 3

In step 3, first we build an alternating tree starting from some exposed vertex, chosen at the
beginning of each iteration. We will do this using breadth-first search algorithm. If on some step
we meet an exposed vertex from the Y part, then finally we can augment our path, finishing up
with a call to the main function of the algorithm. So the code will be the following:

1) Here's the function that adds new edges to the alternating tree:

void add_to_tree(int x, int prevx)

//x - current vertex,prevx - vertex from X before x in the alternating path,
//so we add edges (prevx, xy[x]), (xy[x], x)
{
S[x] = true; //add x to S
prev[x] = prevx; //we need this when augmenting
for (int y = 0; y < n; y++) //update slacks, because we add new vertex to S
if (lx[x] + ly[y] - cost[x][y] < slack[y])
{
slack[y] = lx[x] + ly[y] - cost[x][y];
slackx[y] = x;
}
}
3) And now, the end of the augment() function:

//second part of augment() function

while (true) //main cycle
{
while (rd < wr) //building tree with bfs cycle
{
x = q[rd++]; //current vertex from X part
for (y = 0; y < n; y++) //iterate through all edges in equality graph
if (cost[x][y] == lx[x] + ly[y] && !T[y])
{
if (yx[y] == -1) break; //an exposed vertex in Y found, so
//augmenting path exists!
T[y] = true; //else just add y to T,
q[wr++] = yx[y]; //add vertex yx[y], which is matched
//with y, to the queue
add_to_tree(yx[y], x); //add edges (x,y) and (y,yx[y]) to the tree
}
if (y < n) break; //augmenting path found!
}
if (y < n) break; //augmenting path found!

update_labels(); //augmenting path not found, so improve

labeling
wr = rd = 0;
for (y = 0; y < n; y++)
//in this cycle we add edges that were added to the equality graph as a
//result of improving the labeling, we add edge (slackx[y], y) to the tree if
//and only if !T[y] && slack[y] == 0, also with this edge we add another one
//(y, yx[y]) or augment the matching, if y was exposed
if (!T[y] && slack[y] == 0)
{
if (yx[y] == -1) //exposed vertex in Y found - augmenting path
exists!
{
x = slackx[y];
break;
}
else
{
T[y] = true; //else just add y to T,
if (!S[yx[y]])
{
q[wr++] = yx[y]; //add vertex yx[y], which is matched with
//y, to the queue
add_to_tree(yx[y], slackx[y]); //and add edges (x,y) and (y,
 //yx[y]) to the tree
}
}
}
if (y < n) break; //augmenting path found!
}

if (y < n) //we found augmenting path!

{
max_match++; //increment matching
//in this cycle we inverse edges along augmenting path
for (int cx = x, cy = y, ty; cx != -2; cx = prev[cx], cy = ty)
{
ty = xy[cx];
yx[cy] = cx;
xy[cx] = cy;
}
augment(); //recall function, go to step 1 of the algorithm
}
}//end of augment() function

The only thing in code that hasn't been explained yet is the procedure that goes after labels are
updated. Say we've updated labels and now we need to complete our alternating tree; to do this
and to keep algorithm in O(n3) time (it's only possible if we use each edge no more than one time
per iteration) we need to know what edges should be added without iterating through all of them,
and the answer for this question is to use BFS to add edges only from those vertices in Y, that are
not in T and for which slack[y] = 0 (it's easy to prove that in such way we'll add all edges and
keep algorithm to be O(n3)). See picture below for explanation:

At last, here's the function that implements Hungarian algorithm:

int hungarian()
{
 int ret = 0; //weight of the optimal matching
max_match = 0; //number of vertices in current matching
memset(xy, -1, sizeof(xy));
memset(yx, -1, sizeof(yx));
init_labels(); //step 0
augment(); //steps 1-3
for (int x = 0; x < n; x++) //forming answer there
ret += cost[x][xy[x]];
return ret;
}

To see all this in practice let's complete the example started on step 0.

Build
Augmenting Build
alternating
path found alternating tr
tree

→ → →

Build
Update labels Update labe
alternating
(Δ=1) (Δ=1)
tree

→ → →

Build
Augmenting Build
alternating
path found alternating tr
tree

→ → →

Update labels Build Update labels

(Δ=2) alternating tree (Δ=1)
 → → →
Build Augmenting
alternating tree path found

Optimal matchi
found
→ →

Finally, let's talk about the complexity of this algorithm. On each iteration we increment
matching so we have n iterations. On each iterations each edge of the graph is used no more than
one time when finding augmenting path, so we've got O(n2) operations. Concerning labeling we
update slack array each time when we insert vertex from X into S, so this happens no more than
n times per iteration, updating slack takes O(n) operations, so again we've got O(n2). Updating
labels happens no more than n time per iterations (because we add at least one vertex
from Y to T per iteration), it takes O(n) operations - again O(n2). So total complexity of this
implementation is O(n3).

Some practice

For practice let's consider the medium problem from SRM 371 (div. 1). It's obvious we need to
find the maximum-weighted matching in graph, where the X part is our players, the Y part is the
opposing club players, and the weight of each edge is:

Though this problem has a much simpler solution, this one is obvious and fast coding can bring
more points.

Try this one for more practice. I hope this article has increased the wealth of your knowledge in
classical algorithms… Good luck and have fun!

Computational Method for the Transportation Problem

This paper explores computational implications of allowing nonlinear objective functions
in the transportation problem. Two types of nonlinearities, including polynomials, are
studied. The choice of these functions resulted from our interest in models of integrated
water management. Zangwill's convex simplex method and the primal method of
transportation problem form the basis of our algorithm. Innovative features of our work
 are compact storage and efficient computation procedures. We study the effects on
computation time of problem size; the density of nonlinear terms; the size of tolerances
for stopping rules; and rules for choice of new variables to enter the solution. We find
that problems up to 95 × 95 in size are capable of reasonably fast solution. A particularly
surprising finding is that one-dimensional search for improving solutions performs
adequately, at least for the kinds of problems posed in this paper. We are encouraged by
our results and believe that models involving nonlinear objective functions may be
tractable even for relatively large problems, thus making possible more accurate
descriptions of real situations.

The transportation problem is a special type of linear programming problem where the objective
is to minimise the cost of distributing a product from a number of sources or origins to a number
of destinations. Because of its special structure the usual simplex method is not suitable for
solving transportation problems. These problems require a special method of solution.
The originof a transportation problem is the location from which shipments are despatched.
Thedestination of a transportation problem is the location to which shipments are transported.
Theunit transportation cost is the cost of transporting one unit of the consignment from an
origin to a destination.

In the most general form, a transportation problem has a number of origins and a number of
destinations. A certain amount of a particular consignment is available in each origin. Likewise,
each destination has a certain requirement. The transportation problem indicates the amount of
consignment to be transported from various origins to different destinations so that the total
transportation cost is minimised without violating the availability constraints and the requirement
constraints. The decision variables xij of a transportation problem indicate the amount to be
transported from the ith origin to the jth destination. Two subscripts are necessary to describe
these decision variables. A transportation problem can be formulated as a linear programming
problem using decision variables with two subscripts.

You are currently

Transportation Problem

A typical transportation problem is shown in Fig. 9. It deals with sources where a supply of some
commodity is available and destinations where the commodity is demanded. The classic
statement of the transportation problem uses a matrix with the rows representing sources and
columns representing destinations. The algorithms for solving the problem are based on this
matrix representation. The costs of shipping from sources to destinations are indicated by the
entries in the matrix. If shipment is impossible between a given source and destination, a large
cost of M is entered. This discourages the solution from using such cells. Supplies and demands
are shown along the margins of the matrix. As in the example, the classic transportation problem
has total supply equal to total demand.

Figure 9. Matrix model of a transportation problem.

The network model of the transportation problem is shown in Fig. 10. Sources are identified as
the nodes on the left and destinations on the right. Allowable shipping links are shown as arcs,
while disallowed links are not included.

Figure 10. Network flow model of the transportation problem.

Only arc costs are shown in the network model, as these are the only relevant parameters. All
other parameters are set to the default values. The network has a special form important in graph
theory; it is called a bipartite network since the nodes can be divided into two parts with all arcs
going from one part to the other.

On each supply node the positive external flow indicates supply flow entering the network. On
each destination node a demand is a negative fixed external flow indicating that this amount
must leave the network. The optimum solution for the example is shown in Fig. 11.
 Figure 11. Optimum solution, z = 46.

Variations of the classical transportation problem are easily handled by modifications of the
network model. If links have finite capacity, the arc upper bounds can be made finite. If supplies
represent raw materials that are transformed into products at the sources and the demands are in
units of product, the gain factors can be used to represent transformation efficiency at each
source. If some minimal flow is required in certain links, arc lower bounds can be set to nonzero
values.

The Transportation and Assignment problems deal with assigning sources and jobs to
destinations and machines. We will discuss the transportation problem first.
Suppose a company has m factories where it manufactures its product and n outlets from where
the product is sold. Transporting the product from a factory to an outlet costs some money which
depends on several factors and varies for each choice of factory and outlet. The total amount of
the product a particular factory makes is fixed and so is the total amount a particular outlet can
store. The problem is to decide how much of the product should be supplied from each factory to
each outlet so that the total cost is minimum.
Let us consider an example.
Suppose an auto company has three plants in cities A, B and C and two major distribution
centers in D and E. The capacities of the three plants during the next quarter are 1000, 1500 and
1200 cars. The quarterly demands of the two distribution centers are 2300 and 1400 cars. The
transportation costs (which depend on the mileage, transport company etc) between the plants
and the distribution centers is as follows:

Cost Table Dist Center D Dist Center E

Plant A 80 215

Plant B 100 108

Plant C 102 68

Which plant should supply how many cars to which outlet so that the total cost is minimum?
The problem can be formulated as a LP model:
Let be the amount of cars to be shipped from source i to destination j. Then our objective is
to minimize the total cost which
is . The constraints are the
ones imposed by the amount of cars to be transported from each plant and the amount each
center can absorb.
The whole model is:
Minimize z =
subject to,
;
;
;
;
;
and integer, i = 1,2,3, j = 1,2.
The problem can now be solved using the simplex method. A convenient procedure is discussed
in the next section.

Transportation Method

A transportation tableau is given below. Each cell represents a shipping route (which is an arc on
the network and a decision variable in the LP formulation), and the unit shipping costs are given
in an upper right hand box in the cell.

To solve the transportation problem by its special purpose algorithm, the sum of the supplies at
the origins must equal the sum of the demands at the destinations (Balanced transportation
problem).
• If the total supply is greater than the total demand, a dummy destination is added with demand
equal to the excess supply, and shipping costs from all origins are zero.

• Similarly, if total supply is less than total demand, a dummy origin is added. The supply at the
dummy origin is equal to the difference of the total supply and the total demand. The costs
associated with the dummy origin are equal to zero.

When solving a transportation problem by its special purpose algorithm, unacceptable shipping
routes are given a cost of +M (a large number).

Develop an Initial Solution

Two methods to get the initial solution:

• Northwest Corner Rule

• Minimum Cell-Cost Method

Northwest Corner Rule

1. Starting from the northwest corner of the transportation tableau, allocate as much quantity as
possible to cell (1,1) from Origin 1 to Destination 1, within the supply constraint of source 1 and
the demand constraint of destination 1.

2. The first allocation will satisfy either the supply capacity of Source 1 or the destination
requirement of Destination 1.

ƒ If the demand requirement for destination 1 is satisfied but the supply capacity for Source 1 is
not exhausted, move on to cell (1,2) for next allocation.

ƒ If the demand requirement for destination 1 is not satisfied but the supply capacity for Source 1
is exhausted, move to cell (2,1)

ƒ If the demand requirement for Destination 1 is satisfied and the supply capacity for Source 1 is
also exhausted, move on to cell (2,2).

3. Continue the allocation in the same manner toward the southeast corner of the transportation
tableau until the supply capacities of all sources are exhausted and the demands of all
destinations are satisfied.
Initial tableau developed using Northwest Corner Method

Total Cost = 12(400)+13(100)+4(700)+9(100)+12(200)+4(500)= 142,000

Minimum Cell-Cost Method

Although the North-west Corner Rule is the easiest, it is not the most attractive because our
objective is not included in the process.

Steps of Minimum Cell-Cost Method

1. Select the cell with the minimum cell cost in the tableau and allocate as much to this cell as
possible, but within the supply and demand constraints.

2. Select the cell with the next minimum cell-cost and allocate as much to this cell as possible
within the demand and supply constraints.

3. Continue the procedure until all of the supply and demand requirements are satisfied. In a case
of tied minimum cell-costs between two or more cells, the tie can be broken by selecting the cell
that can accommodate the greater quantity.

Initial tableau developed using Minimum Cell-Cost Method

Total Cost = 12(300)+4(200)+4(700)+10(100)+9(200)+4(500)= 120,000

MODI Method (for obtaining reduced costs)

Associate a number, ui, with each row and vj with each column.

• Step 1: Set u1 = 0.

• Step 2: Calculate the remaining ui‘s and vj‘s by solving the relationship cij

= ui + vj for occupied cells.

• Step 3: For unoccupied cells (i,j), the reduced cost = cij – ui – vj.

Step 1: For each unoccupied cell, calculate the reduced cost by the MODI method. Select the
unoccupied cell with the most negative reduced cost. (For maximization problems select the
unoccupied cell with the largest reduced cost.) If none, STOP.

Step 2: For this unoccupied cell, generate a stepping stone path by forming a closed loop with
this cell and occupied cells by drawing connecting alternating horizontal and vertical lines
between them. Determine the minimum allocation where a subtraction is to be made along this
path.
Step 3: Add this allocation to all cells where additions are to be made, and subtract this
allocation to all cells where subtractions are to be made along the stepping stone path. (Note: An
occupied cell on the stepping stone path now

becomes 0 (unoccupied). If more than one cell becomes 0, make only one unoccupied; make the
others occupied with 0′s.)

GO TO STEP 1.

Example: Acme Block Co. (ABC)

Acme Block Company has orders for 80 tons of concrete blocks at three suburban locations as
follows: Northwood — 25 tons, Westwood — 45 tons, and Eastwood — 10 tons. Acme has two
plants, each of which can produce 50 tons per week. Delivery cost per ton from each plant to
each suburban location is shown below.

How should end of week shipments be made to fill the above orders?

Since total supply = 100 and total demand = 80, a dummy destination is created with demand of
20 and 0 unit costs.

Iteration 1: Tie for least cost (0), arbitrarily select x14. Allocate 20. Reduce s1 by 20 to 30 and
delete the Dummy column.

Iteration 2: Of the remaining cells the least cost is 24 for x11. Allocate 25. Reduce s1 by 25 to 5
and eliminate the Northwood column.
Iteration 3: Of the remaining cells the least cost is 30 for x12. Allocate 5. Reduce the Westwood
column to 40 and eliminate the Plant 1 row. Iteration 4: Since there is only one row with two
cells left, make the final allocations of 40 and 10 to x22 and x23, respectively.

1. Set u1 = 0

2. Since u1 + vj = c1j for occupied cells in row 1, then v1 = 24, v2 = 30, v4 = 0.

3. Since ui + v2 = ci2 for occupied cells in column 2, then u2 + 30 = 40, hence u2 = 10.

4. Since u2 + vj = c2j for occupied cells in row 2, then 10 + v3 = 42, hence v3 = 32.

Calculate the reduced costs (circled numbers on the previous slide) by cij – ui – vj.

Unoccupied Cell Reduced Cost (1,3) 40 – 0 – 32 = 8 (2,1) 30 – 24 -10 = -4 (2,4) 0 – 10 – 0 = -10

Iteration 1:

The stepping stone path for cell (2,4) is (2,4), (1,4), (1,2), (2,2). The allocations in the subtraction
cells are 20 and 40, respectively. The minimum is 20, and hence reallocate 20 along this path.
Thus for the next tableau:

x24 = 0 + 20 = 20 (0 is its current allocation) x14 = 20 – 20 = 0 (blank for the next tableau) x12 =
5 + 20 = 25

x22 = 40 – 20 = 20
The other occupied cells remain the same.

1. Set u1 = 0.

2. Since u1 + vj = cij for occupied cells in row 1, then v1 = 24, v2 = 30.

3. Since ui + v2 = ci2 for occupied cells in column 2, then u2 + 30 = 40, or u2 =

10.

4. Since u2 + vj = c2j for occupied cells in row 2, then 10 + v3 = 42 or v3 = 32;

and, 10 + v4 = 0 or v4 = -10.

Iteration 2

Calculate the reduced costs (circled numbers on the previous slide) by cij – ui – vj.

Unoccupied Cell Reduced Cost (1,3) 40 – 0 – 32 = 8 (1,4) 0 – 0 – (-10) = 10 (2,1) 30 – 10 – 24 =

-4

The most negative reduced cost is = -4 determined by x21. The stepping stone path for this cell is
(2,1),(1,1),(1,2),(2,2). The allocations in the subtraction cells are 25 and 20 respectively. Thus
the new solution is obtained by reallocating 20 on the stepping stone path. Thus for the next
tableau:

x21 = 0 + 20 = 20 (0 is its current allocation)

x11 = 25 – 20 = 5

x12 = 25 + 20 = 45

x22 = 20 – 20 = 0 (blank for the next tableau)

The other occupied cells remain the same.

1. Set u1 = 0

2. Since u1 + vj = c1j for occupied cells in row 1, then v1 = 24 and v2 = 30.

3. Since ui + v1 = ci1 for occupied cells in column 2, then u2 + 24 = 30 or u2 =6.

4. Since u2 + vj = c2j for occupied cells in row 2, then 6 + v3 = 42 or v3 = 36, and 6 + v4 = 0 or

v4 = -6.

Iteration 3

Calculate the reduced costs (circled numbers on the previous slide) by cij – ui – vj.

Unoccupied Cell Reduced Cost

(1,3) 40 – 0 – 36 = 4
(1,4) 0 – 0 – (-6) = 6
(2,2) 40 – 6 – 30 = 4

Since all the reduced costs are non-negative, this is the optimal tableau.
The Transportation Problem
There is a type of linear programming problem that may be solved using a simplified version of
the simplex technique called transportation method. Because of its major application in solving
problems involving several product sources and several destinations of products, this type of
problem is frequently called the transportation problem. It gets its name from its application to
problems involving transporting products from several sources to several destinations. Although
the formation can be used to represent more general assignment and scheduling problems as well
as transportation and distribution problems. The two common objectives of such problems are
either (1) minimize the cost of shipping munits to n destinations or (2) maximize the profit of
shipping m units to n destinations.

Let us assume there are m sources supplying n destinations. Source capacities, destinations
requirements and costs of material shipping from each source to each destination are given
constantly. The transportation problem can be described using following linear
programmingmathematical model and usually it appears in a transportation tableau.

There are three general steps in solving transportation problems.

We will now discuss each one in the context of a simple example. Suppose one company has
four factories supplying four warehouses and its management wants to determine the minimum-
cost shipping schedule for its weekly output of chests. Factory supply, warehouse demands, and
shipping costs per one chest (unit) are shown in Table 7.1

Table 7.1 ”Data for Transportation Problem”

At first, it is necessary to prepare an initial feasible solution, which may be done in several
different ways; the only requirement is that the destination needs be met within the constraints of
source supply.

7.1 The Transportation Matrix

The transportation matrix for this example appears in Table 7.2, where supply availability at each
factory is shown in the far right column and the warehouse demands are shown in the bottom
row. The unit shipping costs are shown in the small boxes within the cells (see transportation
tableau – at the initiation of solving all cells are empty). It is important at this step to make sure
that the total supply availabilities and total demand requirements are equal. Often there is an
excess supply or demand. In such situations, for the transportation method to work, a dummy
warehouse or factory must be added. Procedurally, this involves inserting an extra row (for an
additional factory) or an extra column (for an ad warehouse). The amount of supply or demand
required by the ‖dummy‖ equals the difference between the row and column totals.

In this case there is:

Total factory supply … 51

Total warehouse requirements … 52

This involves inserting an extra row - an additional factory. The amount of supply by the dummy
equals the difference between the row and column totals. In this case there is 52 – 51 = 1. The
cost figures in each cell of the dummy row would be set at zero so any units sent there would not
incur a transportation cost. Theoretically, this adjustment is equivalent to the simplex procedure
of inserting a slack variable in a constraint inequality to convert it to an equation, and, as in the
simplex, the cost of the dummy would be zero in the objective function.

Table 7.2 "Transportation Matrix for Chests Problem With an Additional Factory
(Dummy)"

7.2 Initial Feasible Solution

Initial allocation entails assigning numbers to cells to satisfy supply and demand constraints.
Next we will discuss several methods for doing this:the Northwest-Corner method, Least-Cost
method, and Vogel's approximation method (VAM).

Table 7.3 shows a northwest-corner assignment. (Cell A-E was assigned first, A-F second, B-F
third, and so forth.) Total cost : 10*10 + 30*4 + 15*10 + 30*1 + 20*12 + 20*2 + 45*12 + 0*1 =
1220($).

Inspection of Table 7.3 indicates some high-cost cells were assigned and some low-cost cells
bypassed by using the northwest-comer method. Indeed, this is to be expected since this method
ignores costs in favor of following an easily programmable allocation algorithm.

Table 7.4 shows a least-cost assignment. (Cell Dummy-E was assigned first, C-E second, B-H
third, A-H fourth, and so on.) Total cost : 30*3 + 25*6 + 15*5 +10*10 + 10*9 + 20*6 + 40*12 +
0*1= 1105 ($).

Table 7.5 shows the VAM assignments. (Cell Dummy-G was assigned first, B-F second, C-E
third, A-H fourth, and so on.) Note that this starting solution is very close to the optimal solution
obtained after making all possible improvements (see next chapter) to the starting solution
obtained using the northwest-comer method. (See Table 7.3.) Total cost: 15*14 + 15*10 + 10*10
+ 20*4 + 20*1 + 40*5 + 35*7 + 0*1 = 1005 ($).

Table 7.3 ”Northwest – Corner Assignment”

 Table 7.4"Least - Cost Assignment"

Table 7.5 "VAM Assignment"

7.3 Develop Optimal Solution

To develop an optimal solution in a transportation problem involves evaluating each unused cell
to determine whether a shift into it is advantageous from a total-cost stand point. If it is, the shift
is made, and the process is repeated. When all cells have been evaluated and appropriate shifts
made, the problem is solved. One approach to making this evaluation is the Stepping stone
method.
The term stepping stone appeared in early descriptions of the method, in which unused cells were
referred to as "water" and used cells as "stones"— from the analogy of walking on a path of
stones half-submerged in water. The stepping stone method was applied to the VAM initial
solution, as shown in Table 7.5

Table 7.6 shows the optimal solutions reached by the Stepping stone method. Such solution is
very close to the solution found using VAM method.

Table 7.6 "Optimal Matrix, With Minimum Transportation Cost of $1,000."

7.4 Alternate Optimal Solutions

When the evaluation of any empty cell yields the same cost as the existing allocation, an
alternate optimal solution exists (see Stepping Stone Method – alternate solutions). Assume that
all other cells are optimally assigned. In such cases, management has additional flexibility and
can invoke nontransportation cost factors in deciding on a final shipping schedule.
 Table 7.7 "Alternate Optimal Matrix for the Chest Transportation Problem, With
Minimum Transportation Cost of $1,000.

7.5 Degeneracy

Degeneracy exists in a transportation problem when the number of filled cells is less than the
number of rows plus the number of columns minus one (m + n - 1). Degeneracy may be
observed either during the initial allocation when the first entry in a row or column satisfies both
the row and column requirements or during the Stepping stone method application, when the
added and subtracted values are equal. Degeneracy requires some adjustment in the matrix to
evaluate the solution achieved. The form of this adjustment involves inserting some value in an
empty cell so a closed path can be developed to evaluate other empty cells. This value may be
thought of as an infinitely small amount, having no direct bearing on the cost of the solution.

Procedurally, the value (often denoted by the Greek letter epsilon, - ) is used in exactly the
same manner as a real number except that it may initially be placed in any empty cell, even
though row and column requirements have been met by real numbers. A degenerate
transportation problem showing a Northwest Corner initial allocation is presented in Table 7.8,
where we can see that if were not assigned to the matrix, it would be impossible to evaluate
several cells.

Once a has been inserted into the solution, it remains there until it is removed by subtraction
or until a final solution is reached.

While the choice of where to put an is arbitrary, it saves time if it is placed where it may be
used to evaluate as many cells as possible without being shifted.
7.6 Transportation Problem with a Maximization as a Criterion

A fictive corporation A has a contract to supply motors for all tractors produced by a fictive
corporation B. Corporation B manufactures the tractors at four locations around Central Europe:
Prague, Warsaw, Budapest and Vienna. Plans call for the following numbers of tractors to be
produced at each location:

Prague 9 000

Warsaw 12 000

Budapest 9 000

Corporation A has three plants that can produce the motors. The plants and production capacities
are

Hamburg 8 000

Munich 7 000

Leipzig 10 000

Dresden 5 000

Due to varying production and transportation costs, the profit earns on each motor depends on
where they were produced and where they were shipped. The following transportation table
(Table 7.9) gives the accounting department estimates of the euro profit per unit (motor).

Table 7.9 "The Euro Profit Per One Shipped Motor"

Table 7.10 shows a highest - profit assignment (Least Cost method modification). In contrast to
the Least – Cost method it allocates as much as possible to the highest-cost cell. (Cell Hamburg -
Budapest was assigned first, Munich - Warsaw second, Leipzig - Warsaw third, Leipzig –
Budapest fourth, Dresden – Prague fifth and Leipzig – Prague sixth.) Total profit : 3 335 000
euro.

Table 7.10 "Highest - Profit Assignment"

Applying the Stepping Stone method (modified for maximization purposes) to the initial solution
we can see that no other transportation schedule can increase the profit and so the Highest –
Profit initial allocation is also an optimal solution of this transportation problem.

7.7 The Transshipment Problem

The transshipment problem is similar to the transportation problem except that in the
transshipment problem it is possible to both ship into and out of the same node (point). For the
transportation problem, you can ship only from supply points to demand points. For the
transshipment problem, you can ship from one supply point to another or from one demand point
to another. Actually, designating nodes as supply points or demand points becomes confusing
when you can ship both into and out of a node. You can make the designations clearer if you
classify nodes by their net stock position-excess (+), shortage (-), or 0.

One reason to consider transshipping is that units can sometimes be shipped into one city at a
very low cost and then transshipped to other cities. In some situations, this can be less expensive
than direct shipment.
Let's consider the balanced transportation problem as an example.

Picture 7.1 shows the net stock positions for the three warehouses and four customers. Say that it
is possible to transship through Pilsen to both Innsbruck and Linz. The transportation cost from
Pilsen to Innsbruck is 300 euro per unit, so it costs less to ship from Warsaw to Innsbruck by
going through Pilsen. The direct cost is 950 euro, and the transshipping cost is 600 + 300 = 900
euro. Because the transportation cost is 300 euro from Pilsen to Innsbruck, the cost of
transshipping from Prague through Pilsen to Innsbruck is 400 euro per unit. It is cheaper to
transship from Prague through Pilsen than to ship directly from Prague to Innsbruck.
 Picture 7.1 "Transshipment Example in the Form of a Network Model"

There are two possible conversions to a transportation model. In the first conversion, make each
excess node a supply point and each shortage node a demand point. Then, find the cheapest
method of shipping from surplus nodes to shortage nodes considering all transshipment
possibilities. Let's perform the first conversion for the Picture 7.1 example. Because a
transportation table Prague, Warsaw, and Vienna have excesses, they are the supply points.
Because Krakow, Pilsen, Innsbruck, and Linz have shortages, they are the demand points. The
cheapest cost from Warsaw to Innsbruck is 900 euro, transshipping through Pilsen. The cheapest
cost from Prague to Innsbruck is 400 euro, transshipping through Pilsen too. The cheapest cost
from all other supply points to demand points is obtained through direct shipment. Table 7.11
shows the balanced transportation table for this transshipment problem. For a simple
transportation network, finding all of the cheapest routes from excess nodes to shortage nodes is
easy. You can list all of the possible routes and select the cheapest. However, for a network with
many nodes and arcs, listing all of the possible routes is difficult.

Table 7.11 "The Transshipment Problem After Conversion to a Transportation

Model"
The second conversion of a transshipment problem to a transportation model doesn't require
finding all of the cheapest routes from excess nodes table to shortage nodes. The second
conversion requires more supply and demand nodes than the first conversion, because the points
where you can ship into and out of, occur in the converted transportation problem twice – first as
a supply point and second as a demand point.

7.8 The Assignment Problem

Another transportation problem is the assignment problem. You can use this problem to assign
tasks to people or jobs to machines. You can also use it to award contracts to bidders. Let's
describe the assignment problem as assigning n tasks to n people. Each person must do one and
only one task, and each task must be done by only one person. You represent each person and
each task by a node. Number the people 1to n, and number the tasks 1 to n. The assignment
problem can be described simply using a verbal model or using a linear
programmingmathematical model .

For example, say that five groups of computer users must be trained for five new types of
software. Because the users have different computer skill levels, the total cost of trainings
depends on the assignments.
 Table 7.12 ”Cost of Trainings According to the Groups of Users”

Picture 7.2 "Network Model for Assignment Problem"

Table 7.12 shows the cost of training for each assignment of a user group (A through E) to a
software type (S1 through S5). Picture 7.2 is a network model of this problem.

A balanced assignment problem has the same number of people and tasks. For a balanced
assignment problem, the relationships are all equal. Each person must do a task. For an
unbalanced assignment problem with more people than tasks, some people don't have to do a
task and the first class of constraints is of the type. In general, the simplex method does not
guarantee that the optimal values of the decision variables are integers. Fortunately, for the
assignment model, all of the corner point solutions have integer values for all of the variables.
Therefore, when the simplex method determines the optimal corner point, all of the variable
values are integers and the constraints require that the integers be either 1 or 0 (Boolean).

7.8.1 Conversion to a Balanced Transportation Table

It's not surprising that the variable values for corner point solutions to the assignment model are
integers. The assignment model is a special case of the transportation problem, and the
transportation problem has integer variable values for every corner point.

For the assignment model, the number of supply and demand points are both n. The supply
points correspond to each person, and the demand points correspond to each task.
Furthermore, every supply amount is 1 and every demand amount is 1. There is one of each
person and one of each task.
 Table 7.13 ”The Computer Users Assignment Problem in the Transportation Table
Format”

Table 7.13 represents the computer users assignment problem in the balanced transportation
table format. For the computer users assignment problem, you minimize the total cost of training.
Because number of users = 5 and software types = 5, the transportation problem is balanced. You
can use standard method for initial solution finding (Northwest-Corner method, Least-Cost
method, or Vogel's approximation method (VAM)and a Stepping–Stone method for developing
an optimal solution.

Thinking of the transportation problem as shipping items between cities is helpful, but it's the
mathematical structure that is important. The assignment problem has the same mathematical
structure as the transportation problem, even though it has nothing to do with shipping units.

Note, that each feasible solution of assignment problem will always be strongly
degenerated (number of nonzero variables will always be n).

7.9 Conclusion

The transportation problem is only a special topic of the linear programming problems. It would
be a rare instance when a linear programming problem would actually be solved by hand. There
are too many computers around and too many LP software programs to justify spending time for
manual solution.( There are also programs that assist in the construction of the LP or TP model
itself. Probably the best known is GAMS—General Algebraic Modeling System (GAMS-
General, San Francisco, CA). This provides a high-level language for easy representation of
complex problems.)
 3.An Initial Basic Feasible Solution of the Transportation Problem

Transportation Problem

The problem details trying to find the cheapest way in moving a product from a number of
suppliers to a number of warehouses. The way these problems are normally represented are in a
table:

In this example, bakery two has 8 loaves of bread that it would like to sell. And, for example, it
would cost £4 to send a loaf of bread from bakery 2 to warehouse 1.
Formation as a Linear Programming Problem

When you're forming problems in the form of a Linear Programming problem, you have to
include two things, the objective function, that is, what you want to maximise or minimise; as
well as theconstraints. You also need to set your decision variables. This is done in the
following way:

= The no. of loaves sent from bakery to warehouse

From here, you can set your constraints. Each of these formulae basically state that you can't
send more bread out than a supermarket has, or that you can't send more bread to a warehouse
that it wants.

The objective function is equal to the total cost of all of it; namely, the sum of each of the loaves
of bread on each route, times the cost for that route.

This could be solved using the simplex method. Fortunately, you will never be asked to do it.
Transport problem
transportation problem

One of the most important special cases of the general problem of linear programming. The
formulation of the transportation problem is as follows.

Suppose that at points some homogeneous commodity is produced, where the

volume of output of the commodity at consists of units, . The goods produced
at these production points have to be delivered to consumer points , where the volume
of consumption at consists of units of goods. It is assumed that transportation of the
finished product is possible from any production point to any consumer point and that the
transportation cost necessary for the transfer of a unit of goods from to is monetary
units. Then the problem consists of organizing a plan of shipment such that the total
transportation cost is a minimum.

Formally, the problem is posed in the following way. Let be the quantity of goods shipped
from to . It is required to determine a set of quantities satisfying the conditions

and minimizing the linear form

The set of constraints (1) refers to the fact that the volume shipped from each production point is
exactly equal to the volume produced at this point, while the volume received at a consumer
point corresponds exactly to its requirements. Under these constraints, a necessary and sufficient
condition for the solvability of the transportation problem is that the following balance condition
should hold:
The following two circumstances are specific to the transportation problem: a) each of the
variables enters into two equations of the system (1); b) all the coefficients of the
variables in the constraints take the values 0 or 1 only. Conditions a) and b) enable one to
devise algorithms for the solution of the transportation problem that are essentially simpler than
the simplex method, which is one of the basic methods for solving linear programming problems.

The best known of these algorithms are the method of potentials and the so-called Hungarian
method. The method of potentials is a method of successive improvement of the plan (of
shipments) using the second duality theorem for verifying optimality [1]. The Hungarian method
is a method of successively constructing an admissible plan, which automatically turns out to be
optimal. At the basis of the Hungarian algorithm is the method of alternating chains [2].

Two generalizations of the classical transportation problem are known; they reflect situations
encountered in practice. The open model of the transportation problem: a transportation problem
in which the balance condition (2) does not hold; this means that either the total volume
produced exceeds the total volume required, or vice versa. This problem reduces to the classical
transportation problem by introducing a fictitious production (or consumer) point with
production (or consumption) capacity equal to the difference between the volumes produced and
required.

Multi-index transportation problems retain the general problem of minimizing transportation

costs, but take into account inhomogeneity of the goods (output devices) and inhomogeneity of
the means of transportation.
In non-Russian literature, the transportation problem is sometimes called the Hitchcock problem.

4.The Transportation Problem

The Transportation and Assignment problems deal with assigning sources and jobs to
destinations and machines. We will discuss the transportation problem first.
Suppose a company has m factories where it manufactures its product and n outlets from where
the product is sold. Transporting the product from a factory to an outlet costs some money which
depends on several factors and varies for each choice of factory and outlet. The total amount of
the product a particular factory makes is fixed and so is the total amount a particular outlet can
store. The problem is to decide how much of the product should be supplied from each factory to
each outlet so that the total cost is minimum.
Let us consider an example.
Suppose an auto company has three plants in cities A, B and C and two major distribution
centers in D and E. The capacities of the three plants during the next quarter are 1000, 1500 and
1200 cars. The quarterly demands of the two distribution centers are 2300 and 1400 cars. The
transportation costs (which depend on the mileage, transport company etc) between the plants
and the distribution centers is as follows:

Cost Table Dist Center D Dist Center E

Plant A 80 215

Plant B 100 108

Plant C 102 68

Which plant should supply how many cars to which outlet so that the total cost is minimum?
The problem can be formulated as a LP model:
Let be the amount of cars to be shipped from source i to destination j. Then our objective is
to minimize the total cost which
is . The constraints are the
ones imposed by the amount of cars to be transported from each plant and the amount each
center can absorb.
The whole model is:
Minimize z =
subject to,
;
;
;
;
;
and integer, i = 1,2,3, j = 1,2.
In mathematics and economics, transportation theory is a name given to the study of optimal
transportation and allocation of resources.
The problem was formalized by the French mathematician Gaspard Monge in 1781.[1]
In the 1920s A.N. Tolstoi was one of the first to study the transportation problem
mathematically. In 1930, in the collectionTransportation Planning Volume I for the National
Commissariat of Transportation of the Soviet Union, he published a paper "Methods of Finding
the Minimal Kilometrage in Cargo-transportation in space".[2][3]
Major advances were made in the field during World War II by
the Soviet/Russian mathematician and economist Leonid Kantorovich.[4]Consequently, the
problem as it is stated is sometimes known as the Monge–Kantorovich transportation
problem.
Mines and factories[edit]
Suppose that we have a collection of n mines mining iron ore, and a collection of n factories
which consume the iron ore that the mines produce. Suppose for the sake of argument that these
mines and factories form two disjoint subsets M and F of the Euclidean plane R2. Suppose also
that we have a cost function c : R2 × R2 → [0, ∞), so that c(x, y) is the cost of transporting one
shipment of iron from x toy. For simplicity, we ignore the time taken to do the transporting. We
are also assume that each mine can supply only one factory (no splitting of shipments) and that
each factory requires precisely one shipment to be in operation (factories cannot work at half- or
double-capacity). Having made the above assumptions, a transport plan is
a bijection T : M → F– i.e. an arrangement whereby each mine m ∈M supplies precisely one
factory T(m) ∈ F. We wish to find the optimal transport plan, the plan T whose total cost

is the least of all possible transport plans from M to F. This motivating special case of the
transportation problem is in fact an instance of the assignment problem.
Moving books: the importance of the cost function
following simple example illustrates the importance of the cost function in determining the
optimal transport plan. Suppose that we have n books of equal width on a shelf (the real
line), arranged in a single contiguous block. We wish to rearrange them into another
contiguous block, but shifted one book-width to the right. Two obvious candidates for the
optimal transport plan present themselves:

1. move all n books one book-width to the right; ("many small moves")
2. move the left-most book n book-widths to the right and leave all other books fixed.
("one big move")
If the cost function is proportional to Euclidean distance (c(x, y) = α|x − y|) then these two
candidates are both optimal. If, on the other hand, we choose the strictly convex cost
function proportional to the square of Euclidean distance (c(x, y) = α|x − y|2), then the "many
small moves" option becomes the unique minimizer.
Interestingly, while mathematicians prefer to work with convex cost functions[citation needed],
economists prefer concave ones. The intuitive justification for this is that once goods have
been loaded on to, say, a goods train, transporting the goods 200 kilometres costs much less
than twice what it would cost to transport them 100 kilometres. Concave cost functions
represent this economy of scale.
Abstract formulation of the problem

Monge and Kantorovich formulations

The transportation problem as it is stated in modern or more technical literature looks
somewhat different because of the development ofRiemannian geometry and measure
theory. The mines-factories example, simple as it is, is a useful reference point when
thinking of the abstract case. In this setting, we allow the possibility that we may not wish to
keep all mines and factories open for business, and allow mines to supply more than one
factory, and factories to accept iron from more than one mine.
Let X and Y be two separable metric spaces such that any probability measure on X (or Y) is
a Radon measure (i.e. they are Radon spaces). Let c : X × Y → [0, ∞] be a Borel-measurable
function. Given probability measures μ on X and ν on Y, Monge's formulation of the optimal
transportation problem is to find a transport map T : X → Y that realizes the infimum

where T∗(μ) denotes the push forward of μ by T. A map T that attains this infimum
(i.e. makes it a minimum instead of an infimum) is called an "optimal transport map".
Monge's formulation of the optimal transportation problem can be ill-posed, because
sometimes there is no T satisfying T∗(μ) = ν: this happens, for example, when μ is
a Dirac measure but ν is not).
We can improve on this by adopting Kantorovich's formulation of the optimal
transportation problem, which is to find a probability measure γ on X × Y that attains the
infimum

where Γ(μ, ν) denotes the collection of all probability measures

on X × Y with marginals μ on X and ν on Y. It can be shown[5] that a minimizer for
this problem always exists when the cost function X is lower semi-continuous and
Γ(μ, ν) is a tight collection of measures (which is guaranteed for Radon
spaces X and Y). (Compare this formulation with the definition of the Wasserstein
metric W1 on the space of probability measures.) A gradient descent formulation for
the solution of the Monge-Kantorovich problem was given by Sigurd Angenent,
Steven Haker, and Allen Tannenbaum.[6]
Duality formula[edit]
The minimum of the Kantorovich problem is equal to

where the supremum runs over all pairs of bounded and continuous
functions φ : X → R and ψ : Y → R such that

Solution of the problem[edit]

Optimal transportation on the real line[edit]

For 1 ≤ p < ∞, let denote the collection of probability
measures on R that have finite pth moment. Let and
let c(x, y) = h(x−y), where h : R → [0, ∞) is a convex function.

1. If μ has no atom, i.e., if the cumulative distribution

function Fμ : R → [0, 1] of μ is a continuous function,
then is an optimal transport map. It is the
unique optimal transport map if h is strictly convex.
2. We have

Separable Hilbert spaces[edit]

Let X be a separable Hilbert space. Let denote the collection of
probability measures on X such that have finite pth moment; let
denote those elements that are Gaussian regular: if g is
any strictly positive Gaussian measure on Xand g(N) = 0, then μ(N) = 0
also.

Let , , for p ∈ (1,

∞), p−1 + q−1 = 1. Then the Kantorovich problem has a unique solution κ,
and this solution is induced by an optimal transport map: i.e., there exists
a Borel map r ∈ Lp(X, μ; X) such that

Moreover, if ν has bounded support, then

 for μ-almost all x ∈ X for some locally Lipschitz, c-concave and
maximal Kantorovich potential φ. (Here ∇ φ denotes the Gâteaux
derivative of φ.)

4.Basic Mathematics And Optimization

In mathematics, computer science, or management science, mathematical
optimization(alternatively, optimization or mathematical programming) is the selection of a
best element (with regard to some criteria) from some set of available alternatives.[1]
In the simplest case, an optimization problem consists of maximizing or minimizing a real
function by systematically choosing input values from within an allowed set and computing
the value of the function. The generalization of optimization theory and techniques to other
formulations comprises a large area of applied mathematics. More generally, optimization
includes finding "best available" values of some objective function given a defined domain,
including a variety of different types of objective functions and different types of domains.
An optimization problem can be represented in the following way

Given: a function f : A R from some set A to the real numbers

Sought: an element x0 in A such that f(x0) ≤ f(x) for all x in A ("minimization") or such
that f(x0) ≥ f(x) for all x in A ("maximization").
Such a formulation is called an optimization problem or a mathematical
programming problem (a term not directly related tocomputer programming, but still in
use for example in linear programming – see History below). Many real-world and
theoretical problems may be modeled in this general framework. Problems formulated
using this technique in the fields of physics and computer vision may refer to the
technique as energy minimization, speaking of the value of the function f as
representing the energy of thesystem being modeled.
Typically, A is some subset of the Euclidean space Rn, often specified by a set
of constraints, equalities or inequalities that the members of A have to satisfy.
The domain A of f is called the search space or the choice set, while the elements
of A are calledcandidate solutions or feasible solutions.
The function f is called, variously, an objective function, a loss function or cost
function (minimization),[2] indirect utility function(minimization),[3] a utility
function (maximization), or, in certain fields, an energy function, or energy functional. A
feasible solution that minimizes (or maximizes, if that is the goal) the objective function is called
an optimal solution.
By convention, the standard form of an optimization problem is stated in terms of minimization.
Generally, unless both the objective function and the feasible region are convex in a
minimization problem, there may be several local minima, where a local minimum x* is defined
as a point for which there exists some δ > 0 so that for all x such that
holds; that is to say, on some region around x* all of the function values are greater than or equal
to the value at that point. Local maxima are defined similarly.
A large number of algorithms proposed for solving non-conve problems – including the majority
of commercially available solvers – are not capable of making a distinction between local
optimal solutions and rigorous optimal solutions, and will treat the former as actual solutions to
the original problem. The branch of applied mathematics and numerical analysis that is
concerned with the development of deterministic algorithms that are capable of guaranteeing
convergence in finite time to the actual optimal solution of a non-convex problem is
called global optimization.
Notation[edit]

Optimization problems are often expressed with special notation. Here are some examples.
Minimum and maximum value of a function[edit]
Consider the following notation:

This denotes the minimum value of the objective function , when

choosing x from the set of real numbers . The minimum value in this case
is , occurring at .
Similarly, the notation

asks for the maximum value of the objective function 2x, where x may be any real number. In
this case, there is no such maximum as the objective function is unbounded, so the answer is
"infinity" or "undefined".
Optimal input arguments[edit]
Consider the following notation:

or equivalently

This represents the value (or values) of the argument x in the interval that
2
minimizes (or minimize) the objective functionx + 1 (the actual minimum value of that function
is not what the problem asks for). In this case, the answer is x = -1, since x = 0 is infeasible, i.e.
does not belong to the feasible set.
Similarly,
 or equivalently

represents the pair (or pairs) that maximizes (or maximize) the value of the objective
function , with the added constraint that x lie in the interval (again, the actual
maximum value of the expression does not matter). In this case, the solutions are the pairs of the
form (5, 2kπ) and (−5,(2k+1)π), where k ranges over all integers.
Arg min and arg max are sometimes also written argmin and argmax, and stand for argument
of the minimum and argument of the maximum.

Games Theory
Game theory is the science of strategy. It attempts to determine mathematically and logically the
actions that ―players‖ should take to secure the best outcomes for themselves in a wide array of
―games.‖ The games it studies range from chess to child rearing and from tennis to takeovers.
But the games all share the common feature of interdependence. That is, the outcome for each
participant depends on the choices (strategies) of all. In so-called zero-sum games the interests of
the players conflict totally, so that one person‘s gain always is another‘s loss. More typical are
games with the potential for either mutual gain (positive sum) or mutual harm (negative sum), as
well as some conflict.

Game theory was pioneered by Princeton mathematician JOHN VON NEUMANN. In the early years
the emphasis was on games of pure conflict (zero-sum games). Other games were considered in a
cooperative form. That is, the participants were supposed to choose and implement their actions
jointly. Recent research has focused on games that are neither zero sum nor purely cooperative.
In these games the players choose their actions separately, but their links to others involve
elements of both COMPETITION and cooperation.

Games are fundamentally different from decisions made in a neutral environment. To illustrate
the point, think of the difference between the decisions of a lumberjack and those of a general.
When the lumberjack decides how to chop wood, he does not expect the wood to fight back; his
environment is neutral. But when the general tries to cut down the enemy‘s army, he must
anticipate and overcome resistance to his plans. Like the general, a game player must recognize
his interaction with other intelligent and purposive people. His own choice must allow both for
conflict and for possibilities for cooperation.

The essence of a game is the interdependence of player strategies. There are two distinct types of
strategic interdependence: sequential and simultaneous. In the former the players move in
sequence, each aware of the others‘ previous actions. In the latter the players act at the same
time, each ignorant of the others‘ actions.

A general principle for a player in a sequential-move game is to look ahead and reason back.
Each player should figure out how the other players will respond to his current move, how he
will respond in turn, and so on. The player anticipates where his initial decisions will ultimately
lead and uses thisINFORMATION to calculate his current best choice. When thinking about how
others will respond, he must put himself in their shoes and think as they would; he should not
impose his own reasoning on them.

In principle, any sequential game that ends after a finite sequence of moves can be ―
solved‖
completely. We determine each player‘s best strategy by looking ahead to every possible
outcome. Simple games, such as tic-tac-toe, can be solved in this way and are therefore not
challenging. For many other games, such as chess, the calculations are too complex to perform in
practice—even with computers. Therefore, the players look a few moves ahead and try to
evaluate the resulting positions on the basis of experience.

In contrast to the linear chain of reasoning for sequential games, a game with simultaneous
moves involves a logical circle. Although the players act at the same time, in ignorance of the
others‘ current actions, each must be aware that there are other players who are similarly aware,
and so on. The thinking goes: ― I think that he thinks that I think . . .‖ Therefore, each must
figuratively put himself in the shoes of all and try to calculate the outcome. His own best action
is an integral part of this overall calculation.

This logical circle is squared (the circular reasoning is brought to a conclusion) using a concept
of equilibrium developed by the Princeton mathematician JOHN NASH. We look for a set of
choices, one for each player, such that each person‘s strategy is best for him when all others are
playing their stipulated best strategies. In other words, each picks his best response to what the
others do.
Sometimes one person‘s best choice is the same no matter what the others do. This is called a
―dominant strategy‖ for that player. At other times, one player has a uniformly bad choice—a
―dominated strategy‖—in the sense that some other choice is better for him no matter what the
others do. The search for an equilibrium should begin by looking for dominant strategies and
eliminating dominated ones.

When we say that an outcome is an equilibrium, there is no presumption that each person‘s
privately best choice will lead to a collectively optimal result. Indeed, there are notorious
examples, such as the PRISONERS’ DILEMMA (see below), where the players are drawn into a bad
outcome by each following his best private interests.

Nash‘s notion of equilibrium remains an incomplete solution to the problem of circular reasoning
in simultaneous-move games. Some games have many such equilibria while others have none.
And the dynamic process that can lead to an equilibrium is left unspecified. But in spite of these
flaws, the concept has proved extremely useful in analyzing many strategic interactions.

It is often thought that the application of game theory requires all players to be hyperrational.
The theory makes no such claims. Players may be spiteful or envious as well as charitable and
empathetic. Recall George Bernard Shaw‘s amendment to the Golden Rule: ―D o not do unto
others as you would have them do unto you. Their tastes may be different.‖ In addition to
different motivations, other players may have different information. When calculating an
equilibrium or anticipating the response to your move, you always have to take the other players
as they are, not as you are.

The following examples of strategic interaction illustrate some of the fundamentals of game
theory.

The prisoners’ dilemma. Two suspects are questioned separately, and each can confess or keep
silent. If suspect A keeps silent, then suspect B can get a better deal by confessing. If A
confesses, B had better confess to avoid especially harsh treatment. Confession is B‘s dominant
strategy. The same is true for A. Therefore, in equilibrium both confess. Both would fare better if
they both stayed silent. Such cooperative behavior can be achieved in repeated plays of the game
because the temporary gain from cheating (confession) can be outweighed by the long-run loss
due to the breakdown of cooperation. Strategies such as tit-for-tat are suggested in this context.
Mixing moves. In some situations of conflict, any systematic action will be discovered and
exploited by the rival. Therefore, it is important to keep the rival guessing by mixing your
moves. Typical examples arise in SPORTS—whether to run or to pass in a particular situation in
football, or whether to hit a passing shot crosscourt or down the line in tennis. Game theory
quantifies this insight and details the right proportions of such mixtures.

Strategic moves. A player can use threats and promises to alter other players‘ expectations of his
future actions, and thereby induce them to take actions favorable to him or deter them from
making moves that harm him. To succeed, the threats and promises must be credible. This is
problematic because when the time comes, it is generally costly to carry out a threat or make
good on a promise. Game theory studies several ways to enhance credibility. The general
principle is that it can be in a player‘s interest to reduce his own freedom of future action. By so
doing, he removes his own temptation to renege on a promise or to forgive others‘
transgressions.

For example, Cortés scuttled all but one of his own ships on his arrival in Mexico, purposefully
eliminating retreat as an option. Without ships to sail home, Cortés would either succeed in his
conquest or perish. Although his soldiers were vastly outnumbered, this threat to fight to the
death demoralized the opposition, who chose to retreat rather than fight such a determined
opponent. Polaroid Corporation used a similar strategy when it purposefully refused to diversify
out of the instant photography market. It was committed to a life-or-death battle against any
intruder in the market. When Kodak entered the instant photography market, Polaroid put all its
resources into the fight; fourteen years later, Polaroid won a nearly billion-dollar lawsuit against
Kodak and regained its MONOPOLY market. (Polaroid‘s focus on instant film products later
proved costly when the company failed to diversify into digital photography.)

Another way to make threats credible is to employ the adventuresome strategy of

brinkmanship—deliberately creating a risk that if other players fail to act as you would like them
to, the outcome will be bad for everyone. Introduced by Thomas Schelling in The Strategy of
Conflict,brinkmanship ― is the tactic of deliberately letting the situation get somewhat out of
hand, just because its being out of hand may be intolerable to the other party and force his
accommodation.‖ When mass demonstrators confronted totalitarian governments in Eastern
Europe and China, both sides were engaging in just such a strategy. Sometimes one side backs
down and concedes defeat; sometimes tragedy results when they fall over the brink together.
Bargaining. Two players decide how to split a pie. Each wants a larger share, and both prefer to
achieve agreement sooner rather than later. When the two take turns making offers, the principle
of looking ahead and reasoning back determines the equilibrium shares. Agreement is reached at
once, but the cost of delay governs the shares. The player more impatient to reach agreement gets
a smaller share.

Concealing and revealing information. When one player knows something that others do not,
sometimes he is anxious to conceal this information (his hand in poker) and at other times he
wants to reveal it credibly (a company‘s commitment to quality). In both cases the general
principle is that actions speak louder than words. To conceal information, mix your moves.
Bluffing in poker, for example, must not be systematic. Recall Winston Churchill‘s dictum of
hiding the truth in a ―
bodyguard of lies.‖ To convey information, use an action that is a credible
―signal,‖ something that would not be desirable if the circumstances were otherwise. For
example, an extended warranty is a credible signal to the consumer that the firm believes it is
producing a high-quality product.

Recent advances in game theory have succeeded in describing and prescribing appropriate
strategies in several situations of conflict and cooperation. But the theory is far from complete,
and in many ways the design of successful strategy remains an art.

What economists call game theory psychologists call the theory of social situations, which is an
accurate description of what game theory is about. Although game theory is relevant to parlor
games such as poker or bridge, most research in game theory focuses on how groups of people
interact. There are two main branches of game theory: cooperative and noncooperative game
theory. Noncooperative game theory deals largely with how intelligent individuals interact with
one another in an effort to achieve their own goals. That is the branch of game theory I will
discuss here.
In addition to game theory, economic theory has three other main branches:decision
theory, general equilibrium theory and mechanism design theory. All are closely connected to
game theory.
Decision theory can be viewed as a theory of one person games, or a game of a single player
against nature. The focus is on preferences and the formation of beliefs. The most widely used
form of decision theory argues that preferences among risky alternatives can be described by the
maximization of the expected value of a numerical utility function, where utility may depend on
a number of things, but in situations of interest to economists often depends on money income.
Probability theory is heavily used in order to represent the uncertainty of outcomes, and Bayes
Law is frequently used to model the way in which new information is used to revise beliefs.
Decision theory is often used in the form of decision analysis, which shows how best to acquire
information before making a decision.
General equilibrium theory can be viewed as a specialized branch of game theory that deals with
trade and production, and typically with a relatively large number of individual consumers and
producers. It is widely used in the macroeconomic analysis of broad based economic policies
such as monetary or tax policy, in finance to analyze stock markets, to study interest and
exchange rates and other prices. In recent years, political economy has emerged as a combination
of general equilibrium theory and game theory in which the private sector of the economy is
modeled by general equilibrium theory, while voting behavior and the incentive of governments
is analyzed using game theory. Issues studied include tax policy, trade policy, and the role of
international trade agreements such as the European Union.
Mechanism design theory differs from game theory in that game theory takes the rules of the
game as given, while mechanism design theory asks about the consequences of different types of
rules. Naturally this relies heavily on game theory. Questions addressed by mechanism design
theory include the design of compensation and wage agreements that effectively spread risk
while maintaining incentives, and the design of auctions to maximize revenue, or achieve other
goals.

An Instructive Example

One way to describe a game is by listing the players (or individuals) participating in the game,
and for each player, listing the alternative choices (called actions or strategies) available to that
player. In the case of a two-player game, the actions of the first player form the rows, and the
actions of the second player the columns, of a matrix. The entries in the matrix are two numbers
representing the utility or payoff to the first and second player respectively. A very famous game
is the Prisoner's Dilemma game. In this game the two players are partners in a crime who have
been captured by the police. Each suspect is placed in a separate cell, and offered the opportunity
to confess to the crime. The game can be represented by the following matrix of payoffs

not confess confess

not confess 5,5 -4,10

confess 10,-4 1,1

Note that higher numbers are better (more utility). If neither suspect confesses, they go free, and
split the proceeds of their crime which we represent by 5 units of utility for each suspect.
However, if one prisoner confesses and the other does not, the prisoner who confesses testifies
against the other in exchange for going free and gets the entire 10 units of utility, while the
prisoner who did not confess goes to prison and which results in the low utility of -4. If both
prisoners confess, then both are given a reduced term, but both are convicted, which we represent
by giving each 1 unit of utility: better than having the other prisoner confess, but not so good as
going free.
This game has fascinated game theorists for a variety of reasons. First, it is a simple
representation of a variety of important situations. For example, instead of confess/not confess
we could label the strategies "contribute to the common good" or "behave selfishly." This
captures a variety of situations economists describe as public goods problems. An example is the
construction of a bridge. It is best for everyone if the bridge is built, but best for each individual
if someone else builds the bridge. This is sometimes refered to in economics as an externality.
Similarly this game could describe the alternative of two firms competing in the same market,
and instead of confess/not confess we could label the strategies "set a high price" and "set a low
price." Naturally it is best for both firms if they both set high prices, but best for each individual
firm to set a low price while the opposition sets a high price.
A second feature of this game, is that it is self-evident how an intelligent individual should
behave. No matter what a suspect believes his partner is going to do, it is always best to confess.
If the partner in the other cell is not confessing, it is possible to get 10 instead of 5. If the partner
in the other cell is confessing, it is possible to get 1 instead of -4. Yet the pursuit of individually
sensible behavior results in each player getting only 1 unit of utility, much less than the 5 units
each that they would get if neither confessed. This conflict between the pursuit of individual
goals and the common good is at the heart of many game theoretic problems.
A third feature of this game is that it changes in a very significant way if the game is repeated, or
if the players will interact with each other again in the future. Suppose for example that after this
game is over, and the suspects either are freed or are released from jail they will commit another
crime and the game will be played again. In this case in the first period the suspects may reason
that they should not confess because if they do not their partner will not confess in the second
game. Strictly speaking, this conclusion is not valid, since in the second game both suspects will
confess no matter what happened in the first game. However, repetition opens up the possibility
of being rewarded or punished in the future for current behavior, and game theorists have
provided a number of theories to explain the obvious intuition that if the game is repeated often
enough, the suspects ought to cooperate.

1. Games and Linear Programming

An Application of Linear Programming in Game Theory

I took the Combinatorial Optimization class at AIT Budapest (Aquincum Institute of

Technology) with David Szeszler, a Professor at the Budapest University of Technology and
Economics. We touched on some Graph Theory, Linear Programming, Integer Programming,
the Assignment Problem, and the Hungarian method. My favorite class in the course was focused
on applying Linear Programming in Game Theory. I’ll summarize the most important aspects of
that class in this blog post. I hope this piques your interest in Game Theory (and in
attending AIT).
Basics of Linear Programming

First, I want to touch on some topics in Linear Programming for those who don‘t know much
about setting up a linear program (which is basically a system of linear inequalities with a
maximization function or a minimization function). You can skip this section if you are
confident about the subject.

Linear Programming is basically a field of mathematics that has to do with determining the
optimum value in a feasible region. In determining the optimum value, one of two questions can
be asked: find the minimum point/region or find the maximum point/region. The feasible region
in a linear program is determined by a set of linear inequalities. For a feasible region to even
exist, the set of linear inequalities must be solvable.
A typical linear program is given in this form: . is a row vector of
dimension . is an matrix called the incidence matrix. is a column vector of
dimension . This is called the primal program. The primal program is used to
solve maximization problems. The dual of this primal program is of the
form . are the same as previously defined. is a row vector of
dimension . This is called the dual program. The dual is just a derivation of the primal
program that is used to solve minimization problems.
Having introduced primal and dual programs, the next important theory in line is the duality
theorem. The duality theorem states that = . In
other words, the maximum of the primal program is equal to the minimum of the dual program
(provided that the primal program is solvable and bounded from above). Using this ―tool‖, every
minimization problem can be converted to a maximization problem and vice versa (as long as the
initial problem involves a system of linear inequalities that can be set up as a linear program with
a finite amount of linear constraints and one objective function).
There are linear program solvers out there (both open-source and commercial). Most linear
program solvers are based on the simplex method. I acknowledge that the summary of Linear
Programming given here is devoid of some details. Linear programming is a large field that
cannot be wholly summarized in a few sentences. For more information on linear programming,
check out this wikipedia page.
Sample Game Theory Problem

Suppose that I and my roommate Nick are playing a game called Yo!. The game rules are as
follows: if we both say Yo!, I get $2. If I say Yo! but Nick says YoYo!, I lose $3. On the other
hand, if we both say YoYo!, I get $4. If I say YoYo! but Nick says Yo!, I lose $3. The rules are
summarized in the table below:
Daniel

Nick Yo! YoYo!

 Yo! $2 $-3

YoYo! $-3 $4

The values make up the payoff matrix. When Daniel gains, Nick loses. When Nick gains, Daniel
loses. A negative value (e.g. $-3) indicates that Daniel loses but Nick gains. Now the question
surfaces: is there a smart way of playing this game so that I always win? Of course, if I could
predict Nick‘s next move all the time, then I‘ll certainly play to win. But I can‘t. I must come up
with a strategy that reduces the risk of me losing to a minimum and increases my chance of
winning. In other words, I want to maximize my minimum expected value. So I wish to know
how often I should say Yo! and how often I should say YoYo!. This problem is equivalent to
trying to find a probability column vector of dimension 2 (for the two possible responses Yo!,
YoYo!). Such a probability vector is called a mixed strategy. For example, a mixed strategy for
Daniel could be the column vector: . This translates to saying YoYo! three-quarters
of the time and saying Yo! a quarter of the time. My expected value is
then . This mixed strategy doesn‘t seem optimal! In fact, it‘s not
as we‘ll see later!
This kind of Game Theory problem where we wish to obtain an optimal mixed strategy for the
Column player (in this case, Daniel) and an optimal mixed strategy for the Row player (in this
case, Nick) is called a Two-player, zero sum game. A mixed strategy for the Column player is
an -dimensional probability vector ; that is, a column vector with nonnegative entries that add
up to 1. The entry of the mixed strategy measures the probability that the Column player will
choose the column. In any Two-player, zero sum game, the problem is to maximize the
worst-case gain of the Column player which is equivalent to finding
is a probability vector where represents the payoff matrix

Analogously, the problem of minimizing the worst-case loss of the Row player is equivalent to
finding

is a probability vector where is the payoff matrix

There‘s a theorem that states that is a probability

vector = is a probability vector = . We call the common value of the
game. This theorem is called thez
Minimax Theorem

The Minimax Theorem was proved by John von Neumann (one of the greatest polymaths of all
time, I think). It states that ―F
or every two-player, zero sum game the maximum of the minimum
expected gain of the Column player is equal to the minimum of the maximum expected losses of
the Row player‖. In other words, there exists the optimum mixed strategies and for the
Column player and the Row player respectively and a common value such that
1. No matter how the Row player plays, guarantees an expected gain of at least to the
Column player and
2. No matter how the Column player plays, guarantees an expected loss of at most to the
Row player
Solving the Two-Player, Zero Sum Game

Now let‘s try to solve the Yo! game. First, we aim to obtain the mixed strategy for the Column
player. Let be the mixed strategy where for which and
. We wish to find the maximum of where is the payoff matrix. To make this into a
linear program, we can say . So is worst-case gain of Daniel. We wish to
maximize . Since is the minimum possible value of , we obtain the following linear
constraints




Solving the linear program gives us and . So the optimal
mixed strategy for the Column player is . This translates to saying that if
Daniel says Yo! of the time and YoYo! of the time, his worst-case gain will
be . In other words, Daniel will lose at most the value of the game no matter how
Nick plays. According to the minimax theorem, this is optimal.

Note that this doesn‘t mean that Daniel will always lose the game but that he can lose by at
most the value of the game. If Nick doesn‘t play optimally (Nick doesn‘t use his optimal
mixed strategy), Daniel will most likely win!

Nick could obtain his optimal strategy by solving the dual of the primal program to obtain the
vector which will be his optimal mixed strategy.

The minimax theorem is an interesting and very useful application of Linear Programming in
Game Theory. Two-player, zero sum games can also be solved using Nash Equilibrium which is
very closely related to the minimax theorem but applies to two or more players. Nash
Equilibrium was first proposed by John Nash. There are many Two-player games
including Poker, Card games, Betting games, and so on. As a result, Linear Programming is
used in the Casino!

Matrix game theory is concerned with how two players make decisions when they are faced with
known exact payoffs. The aim of this paper is to develop a simple and an effective linear
programming method for solving matrix games in which the payoffs are expressed with
intervals. Because the payoffs of the matrix game are intervals, the value of the matrix game is
an interval as well. Based on the definition of the value for matrix games, the value of the matrix
game may be regarded as a function of values in the payoff intervals, which is proven to be non-
decreasing. A pair of auxiliary linear programming models is formulated to obtain the upper
bound and the lower bound of the value of the interval-valued matrix game by using the upper
bounds and the lower bounds of the payoff intervals, respectively. By the duality theorem of
linear programming, it is proven that two players have the identical interval-type value of the
interval-valued matrix game. Also it is proven that the linear programming models and method
proposed in this paper extend those of the classical matrix games. The linear programming
method proposed in this paper is demonstrated with a real investment decision example and
compared with other similar methods to show the validity, applicability and superiority.

Matrix game theory is concerned with how two players make decisions when they are faced with
known exact payoffs. The aim of this paper is to develop a simple and an effective linear
programming method for solving matrix games in which the payoffs are expressed with
intervals. Because the payoffs of the matrix game are intervals, the value of the matrix game is
an interval as well. Based on the definition of the value for matrix games, the value of the matrix
game may be regarded as a function of values in the payoff intervals, which is proven to be non-
decreasing. A pair of auxiliary linear programming models is formulated to obtain the upper
bound and the lower bound of the value of the interval-valued matrix game by using the upper
bounds and the lower bounds of the payoff intervals, respectively. By the duality theorem of
linear programming, it is proven that two players have the identical interval-type value of the
interval-valued matrix game. Also it is proven that the linear programming models and method
proposed in this paper extend those of the classical matrix games. The linear programming
method proposed in this paper is demonstrated with a real investment decision example and
compared with other similar methods to show the validity, applicability and superiority.

2.Graphical Method and Dominance

Graphical Method for Games

a) Let us consider a 2 x n game i.e., the payoff matrix will consist of 2 rows and n columns. So
player A (or, row-player) will have two strategies. Also assume that there is no saddle point.
Then the problem can be solved by using the following procedure :
(i) Reduce the size of the payoff matrix using the rules of dominance, if it is applicable.

(ii) Let p be the probability of selection of strategy I and 1 - p be the probability of

selection of strategy II by player A.

Write down the expected gain function of player A with respect to each of the strategies of
player B .

(iii) Plot the gain functions on a graph. Keep the gain function on y-axis and p on x-axis.
Here p will take the value 0 and 1 .

(iv) Find the highest intersection point in the lower boundary (i.e., lower envelope) of the
graph. Since player A is a maximum player, then this point will be a maximum point.

(v) If the number of lines passing through the maximum point is only two, then obtain a
2 x 2 payoff matrix by retaining the columns corresponding to these two lines. Go to step (vii)
else go to step (vi).

(vi) If more than two lines passing through the maximum point then identify two lines
with opposite slopes and form the 2 x 2 payoff matrix as described in step (v).

(vii) Solve the 2 x 2 game.

Example 6. Consider the following game and solve it using graphical method.

Solution. It is observed that there is no saddle point. Column V is dominated by column I and
column II is dominated by column IV. Therefore delete column V and column II and the payoff
matrix is reduced as follows :
Let p be the probability of selection of strategy I and (1-p) be the probability of selection of
strategy II by player A. Therefore, the expected gain (or payoff) functions to player A with
respect to different strategies of player B is given below :

B 's strategy A's expected A's expected gain

gain function p=0 p=1
I 3p - 2(1 - p) = 5p – 2 -2 3
III 6p - ( 1 - p) = 7p – 1 -1 6
IV -p + 2(1 - p) = -3p + 2 2 –1

Now the A's expected gain function is plotted in Fig. 9 . 1 . It is observed that line I and IV
passes through the highest point of the lower boundary. Hence we can form 2 x 2 payoff matrix
by taking the columns due to I and IV for player A and it is displayed below :
sss

Player B
I IV
I 3 -1
Player A II -2 2

Let the mixed strategies for A be SA = ( I II p1 p2 )

With p2 =1- p1

the mixed strategies for B be SB = ( I II III IV V q1 0 0 q2 0 )

q2 = 1- ql
Therefore,

P1 = 2 - (-2) / (3 + 2) - (-1 - 2) =1/ 2 , p2 = 1 – p1 = 1/2

P2 = 2 - (-1)/ (3 + 2) - (- 1 - 2) = 3/8, q2 = 1 – q1 = 5/8

v = 6-2 / (3 + 2) -(- 1 - 2) = 1/ 2

The optimal mixed strategies for A is

SA = ( I II ½ ½ )

the optimal mixed strategies for B is

Sb = ( I II III IV V 3/8 0 0 5/8 0 )

value of game = 1/2

(b) Let us consider a m x 2 game i.e., the payoff matrix will consist of m rows and 2 columns.
Also assume that there is no saddle point. Then the problem can be solved by using the following
procedure :

(i) Reduce the size of the payoff matrix using the rules of dominance, if it is applicable.

(ii) Let q ·be the probability of selection of strategy I and 1-q be the probability of
selection of strategy II by the player B .

Write down the expected gain function of player B with respect to each of the strategies of player
A.

(iii) Plot the gain functions on a graph. Keep the gain function on y-axis and q on x-
axis. Here q will take the value 0 and 1 .
(iv) Find the lowest intersection point in the upper boundary (i.e., upper envelope) of the graph.
Since player B is a minimax player, then this point will be a minimax point.
(v) If the number of lines passing through the minimax point is only two, then obtain a 2 x 2
payoff matrix by retaining the rows corresponding to these two lines. Go to step (vii) else goto
step (vi).

(vi) If more than two lines passing through the minimax point then identify two lines with
opposite slopes and form a 2 x 2 payoff matrix as described in step (v).

(vii) Solve the 2 x 2 game.

Example 7. Consider the following game and solve it using graphical method.

Player B
I II
I 2 1
II 1 3
Player A III 4 -1
IV 5 -2

Solution. The given game dose not have saddle p0int. Also it is observed that none of the rows
can be deleted using the rules of dominance.

Let q be the probability of selection of strategy I and 1 - q be the probability of selection of

strategy II by player B. Therefore, the expected gain (or pay respect to different strategies of
player A is given below

A's strategy B 's expected B's expected gain

gain function q=0 q=1
I 2q + (1 - q) = q + 1 1 2
II q + 3(1 - q) = -2q + 3 3 1
III 4q - ( 1 - q) = 5q – 1 -1 4
IV 5q - 2(1- q) = 7q - 2 -2 5

Now the B 's expected gain function is plotted in Fig. 9 .2.

It is observed that the line II and IV passes through the lowest point of the upper boundary.
Hence we can form 2 x 2 payoff matrix by taking the rows due to II and IV for player B and it is
displayed below :
Player B
I II
II 1 3
Player A
IV 5 -2

Let the mixed strategies for A be SA = I II III IV

( 0 P1 0 P2)

With P2 = 1 - P 1

and the mixed strategies for B be SB = I II

q1 q2
with q2 =1-q1

Therefore,

P1 =-2 -5 /(1-2 –(5+3) =7/9 ,p2 =1 –p1 =2/9

Q1 = -2 -3 /(1-2) –(5+3) =5/9 ,q2 = 1-q1 =4/9

V =-2 -15 /(1-2) –(5+3) =17/9

. . The optimal mixed strategies for A is

SA = I II III IV
0 7/9 0 2/9

the optimal mixed strategies for B is

SB = I II
5/9 4/9

value of game = 1/2

Two-Person Zero Sum Game

In a two-person zero sum game, each of the two players is given a choice between several
prescribed moves at each turn, and each player's loss is equal to the other player's gain.

The payoff matrix of a two-person zero sum game has rows labeled by the "row player's" moves
and columns labeled by the opposing "column player's" moves. The ijentry of the matrix is
the payoff that accrues to the row player in the event that the row player uses move i and the
column player uses move j.

Top of Page

Example

Paper, Scissors, Rock

Rock beats (crushes) scissors; scissors beat (cut) paper, and paper beats (wraps) rock.
Each +1 entry indicates a win for the row player, -1 indicates a loss, and 0 indicates a tie.

Do you want to play? Click on a row strategy...

Column Player

0 -1 1

Row Player 1 0 -1

-1 1 0

Click on a row move.

Mixed Strategy, Expected Value

A player uses a pure strategy if he or she uses the same move at each round of the game.
A mixed strategy is a method of playing a game where, at each round of the game, the player
chooses a move at random so that each is used a predetermined fraction of the time.

We represent a mixed (or pure) strategy for the row player by a row matrix (probability vector)

R = [a b c . . . ]

with the same number of entries as there are rows, where each entry represents the fraction of
times the corresponding row is played (or the probability of using that strategy) and
where a + b + . . . = 1.

A mixed strategy for the column player is represented by a similar column matrix C. For both
row and column players, pure strategies is represented by vectors in with a single 1 and the rest
zeros.

The expected value of the game with payoff matrix P corresponding to the mixed
strategies R and C is given by

e = RPC

The expected value of the game is the average payoff per round if each player uses their mixed
strategy for a large number of rounds.

Example

Here is a variant of "paper, scissors, rock in which "paper/paper" and "rock/rock" is no longer a
draw.

2 -1 1

1 0 -1

-1 1 -2

Suppose the row player uses the mixed strategy

R = [0.75 0 0.25]

(play paper 75% of the time, scissors 0% of the time and rock 25% of the time) and the column
player plays scissors and rock each 50% of the time;

C = 0.5

0.5 .

Then the expected value of the game is

e = RPC

2 -1 1 0

= [0.75 0 0.25] 1 0 -1 0.5

-1 1 -2 0.5

= -0.125

Criterion, Fundamental Principles of Game Theory

Minimax Criterion
A player using zzzzzzzthe minimax criterion chooses a strategy that, among all possible
strategies, minimizes the effect of the other player's best counter-strategy. That is, an optimal
(best) strategy according to the minimax criterion is one that minimizes the maximum damage
the opponent can cause.

Finding the minimax strategy is called solving the game. In the third part of the tutorial for this
topic we show a graphical method for solving 2×2 games. For general games, one uses the
simplex method (see the next topic summary). However, one can frequently simplify a game and
sometimes solve it by "reducing by dominance" and/or checking whether it is "strictly
determined" (see below).

Fundamental Principles of Game Theory When analyzing any game, we make the following
assumptions about both players:

1. Each player makes the best possible move.

2. Each player knows that his or her opponent is also making the best possible move

Example
Consider the following game.

Column Strategy

A B C

1 0 -1 1
Row
2 0 0 2
Strategy
3 -1 -2 3

If the row player follows Principle 1, (s)he should never play Move 1 because Move 2 gives
better payoffs no matter what move the column player chooses. (The payoffs in Row 2 are all at
least as high as the corresponding ones in Row 1.)

Further, following Principle 2, the row player expects that the column player will never play
Move A, since Move B gives better payoffs as far as the column player is concerned. (The
payoffs in Column B are all at least as low as the corresponding ones in Column A.)

Reducing by Dominance

One move dominates another if all its payoffs are at least as advantageous to the player than the
corresponding ones in the other move. In terms of the payoff matrix, we can say it this way:

A. Row r in the payoff matrix dominates row s if each payoff in row r is ≥ the
corresponding payoff in row s.
B. Column r in the payoff matrix dominates column s if each payoff in row r is ≤ the
corresponding payoff in column s.

Note that if two rows or columns are equal, then each dominates the other. A row or
column strictly dominates another if the one dominates the other and they are not equal.

Following the first principle of game theory, the move corresponding to a strictly dominated row
or column will never be played, and both players are aware of this by the second principle. Thus
each player following the principles of game theory will repeatedly eliminate dominated rows
and columns as the case may be. (If two rows or columns are equal, then there is no reason to
choose one over the other, so either may be eliminated.) This process is called reduction by
dominance.

Consider the above game once again.

 Column Strategy

A B C

1 0 -1 1
Row
2 0 0 2
Strategy
3 -1 -2 3

Since the entries in Row 2 are ≥ the corresponding ones in Row 1, Row 2 dominates Row 1.

Since the entries in Column B are ≤ the corresponding ones in Column A, Column B dominates
Column A.

Reducing the Above Game by Dominance

Since Row 2 dominates Row 1 we eliminate Row 1 to obtain

ABC

2 0 0 2

3 -1 -2 3

Since Column B now dominates both Columns A and C we eliminate both Columns A and C to
obtain

2 0

3 -2

Since the top row now dominates the bottom row, we eliminate the bottom row, and we are
reduced to the following 1×1 matrix

2( 0 )
In this case, we have solved the game by reduction by dominance: The row player should always
play 2 and the column player should always play B. Since the corresponding payoff is 0, we say
that the game is fair (neither player has an advantage over the other).

Note that we were lucky here: Not all games can be reduced by dominance to a 1×1 game.

Top of Page

Saddle Point, Strictly Determined Game

A saddle point is a payoff that is simultaneously a row minimum and a column maximum. To
locate saddle points, circle the row minima and box the column maxima. The saddle points are
those entries that are both circled and boxed.

A game is strictly determined if it has at least one saddle point. The following statements are
true about strictly determined games.

A. All saddle points in a game have the same payoff value.

B. Choosing the row and column through any saddle point gives minimax strategies for both
players. In other words, the game is solved via the use of these (pure) strategies.

The value of a strictly determined game is the value of the saddle point entry. A fairgame has
value of zero, otherwise it is unfair or biased.

Example

In the above game, there are two saddle points, shown in color.

ABC

1 0 -1 1

2 0 0 2

3 -1 -2 3

Since the saddle point entries are zero, this is a fair game.

The on-line game theory utility can be used to check any game (up to 5×5) for saddle points.
Try it out.

1. Games with Mixed Strategies

1. Mixed Strategies
We have seen that Nash equilibria are likely outcomes of games. But what happens in games
without Nash equilibrium in pure strategies? Even very simple games like ROCK-SCISSORS-
PAPER do not have any such Nash equilibrium. Whereas in games with pure Nash equilibria
players have even an interest of communicating their strategy to the other player before the
game, in ROCK-SCISSORS-PAPER it is crucial to leave your opponent in the dark about what
you are planning. You want to surprise your opponent, and such a surprise is best achieved by
surprising yourself. This could be done by delegating the decision about your strategy to a
random device. This is essentially the idea of a mixed strategy.

An example for a mixed strategy in ROCK-SCISSORS-PAPER is to play "rock", "scissors", or

"paper" with probabilities 50%, 25%, or 25%, respectively. Before the game is played, the player
decides randomly, based on these probabilities which pure strategy to use. Pure strategies can
also be viewed as mixed strategies where one strategy is chosen with probability 100% and the
others with probabilities 0%.

If we add the above mentioned mixed strategy (50% rock, 25% scissors, 25% paper) as an option
for the first player in ROCK-SCISSORS-PAPER, then the expected payoffs for player 1 against
player 2's pure strategies Rock, Scissors, Paper, are 0, 0.25, -0.25 respectively.

Rock Scissors Paper

Rock 0 1 -1

Scissors -1 0 1

Paper 1 -1 0

50-25-25 mix 0 0.25 -0.25

If the first player, Ann, plays the mixed strategy and Beth plays Rock, then with 50% probability
there will be a tie Rock versus Rock, with 25% probability Beth will win (Beth's Rock against
Ann's Scissors), and with 25% probability Ann will show Paper and win against Beth's Rock.
Thus the expected payoff for Ann when playing this mixed strategy against Beth's Rock equals
50%·0 + 25%·(-1) + 25%·1 = 0. Thus the values in the fourth row are expected values of the
corresponding values in the other rows and same column, using the probabilities of the mix. For
instance, the second value in the fourth row is 50%·1 + 25%·0 + 25%·(-1) = 0.25, and the third
one 50%·(-1) + 25%·1 + 25%·0 = -0.25. Even though this new mixed strategy doesn't dominate
any of the pure strategies, the newly added mixed strategy may be attractive to a player aiming at
the maximin strategy since it guarantees a payoff of -0.25 compared to -1 in the other three
cases.
Of course, the other player Beth is also entitled to mixed strategies. We assume that Beth
chooses a mix of 25% rock, 50% scissors, 25% paper. Inserting this 25-50-25 mix as another one
of Beth's options, we obtain the following bimatrix:

Rock Scissors Paper 25-50-25 Mix

Rock 0 1 -1 0.25

Scissors -1 0 1 0

Paper 1 -1 0 -0.25

50-25-25 mix 0 0.25 -0.25 0.0625

The new values are computed as before, as expected values, using the payoffs of the same row,
weighted by the probability of the mix. For instance, the last entry in the first row is computed as
25%·0 + 50%·1 + 25%·(-1) = 0.25. Even the payoff for 50-25-25 mix against 25-50-25 mix is
computed this way, using the fourth row values, as 25%·0 + 50%·0.25 + 25%·(-0.25) = 0.0625.
Maybe not too surprisingly, Ann's mix, with its emphasis on rock, beats the Beth's mix, that is
heavy on scissors.

1.1 Best Response

Class Activity: Find a good response to Jane playing the 50% rock, 25% scissors, 25% paper strategy
in this applet. Play again until you have won 3 times more than Jane. I know you can do it.

As can be seen in the matrix above, "Paper" is better against Ann's 50-25-25 mix than "Rock" or
"Scissors", since it gives an expected payoff of 0.25 for Beth. The opponent emphasizes "Rock",
so you must emphasize the move beating "Rock", which is "Paper". Of course we could also
look for best responses among the mixed strategies, but why would we mix strategies of different
value---we would just dilute the best response.

This is also true in general. Since the payoffs of a mixed strategies are weighted averages of the
payoffs for the pure strategies, the maximum payoff is always achieved at a pure strategy.

Fact: Among the best responses to mixed strategies of the other players is always also a pure strategy.

The mixed strategies that are best responses are just all combinations of the best response pure
strategies. To formulate this, the notion of support may be useful---the support of a mixed
strategy are all pure strategies that occur with nonzero probability.
Indifference Theorem: A mixed strategy of Ann is a best response to mixed strategies of the other players if
each pure strategy in its support is also a best response to to the given mixed strategies of the other players.

So do we even need these fancy mixed strategies, if we can always react with old-fashioned pure
strategies? This is discussed next.

1.2 Brown's fictitious play

Assume a game is repeated over and over. How would we find out what mixed strategy the other
player is using? He or she may not tell us. Well, you may want to observe very closely what the
other player is doing, and react accordingly. You may want to always play the best
response pure strategy to the observed mixed strategy played by the other player.

However, the other player may observe you as well, observe your strategy as a mixed strategy,
and react accordingly. That means, you will adapt your play according to the play of the other
player, and the other player adapts his or her play according to your observed play. Even though
both players think they are always playing a pure strategy, it looks as if they would play mixed
strategies, since their pure strategies are changing over time. In their histories, they play the
different pure strategies with some relative frequencies, which are perceived as probabilities by
the other player. These relative frequencies do change over time, but maybe eventually less and
less---they might converge. This idea has been formulated by G.W. Brown [Brown 51]. He
proposed that for two-person zero-sum games such a process will eventually converge and that
these resulting mixed strategies are both best responses to each other.

This process can be simulated in the following Excel sheet. In ROCK-SCISSORS-PAPER, the
resulting mixes of the histories are 1/3 Rock, 1/3 Scissors, and 1/3 Paper.

The following example illustrates that the pure strategies do not always occur with equal
frequency:

CRASH ROCK-SCISSORS-PAPER is a simultaneous zero-sum game played by two players. Each one
has three moves, "Rock", "Scissors", and "Paper". "Scissors" wins against "Paper", "Paper" wins against
"Rock", but "Rock" wins big against "Scissors", crashing it with a loud sound, and giving a double payoff
to the winner. It is ROCK-SCISSORS-PAPER with this modified payoff matrix:

Rock Scissors Paper

Rock 0 2 -1

Scissors -2 0 1
 Paper 1 -1 0

Try the matrix in the same Excel sheet. Both players will eventually play the 25%-25%-50% mix.
Although "Scissors" seems to be the weakest move in this modified game, it is played more often than the
other two moves.

To give another example, this time for a non-simultaneous game, let us look at the normal form of VNM
POKER(2,4,2,3) discussed in the previous two chapters. After eliminating weakly dominated strategies,
we get the following matrix:

FC CC

CR 0 2/7

RR 1/7 0

Running it through our Excel sheet, we get a mix of 1/3 "CR" and 2/3 "RR" for Ann, and 2/3 "FC" and
1/3 "CC" for Beth. Note that these mixed strategies translate into Ann raising with probability 2/3 when
holding a low-value card, and always raising when holding a high-value card, whereas Beth would call
with probability 1/3 when holding a low-value card, and always when holding a high-value card. The
expected payoff for Ann when both play their mix equals 2/21.

Brown's fictitious play is important even if you play the same opponent only once. In a sense,
both players would simulate thousands of rounds in their heads, and arrive at the mixed
strategies. That's where the word "fictitious" comes from. Even though, after this one game,
nobody can observe your probabilities by which you have chosen the pure strategies, it is
important to have them and to select your move accordingly.

1.3 Mixed Maximin Strategy, Mixed Security Level, and Linear Programs

Up to now, the maximin strategy was supposed to be a pure strategy. From now on we also allow
mixed strategies.

First note that for every mixed strategy of Ann, Ann's worst outcome is always achieved at a
pure strategy of Beth. The reasoning is the same as for the fact mentioned above that among the
best responses, there is always a pure strategy one. Ann's payoff for any mixed strategy of Beth
is just a weighted average of Ann's payoffs for the different pure strategies of Beth, and such a
weighted average can not be less than the smallest value.

For every one of the infinitely many mixed strategies of Ann, we next create the row of the
payoffs versus the finitely many pure strategies of Beth. The mixed maximin strategy is the
mixed strategy with the highest lowest entry in the row---it is the mixed strategy that guarantees
the highest expected payoff, which we may call the mixed security level for Ann.

Take ROCK-SCISSORS-PAPER2 as example, where whenever Ann plays scissors the bid is
doubled. The Payoff matrix of this zero-sum game is as follows:

BRock BScissors BPaper

ARock 0 1 -1

AScissors -2 0 2

APaper 1 -1 0

We give a few rows. The formula for calculating the entries in a row is given in the last row.

BRock BScissors BPaper

ARock 0 1 -1

AScissors -2 0 2

APaper 1 -1 0

1/3-1/3-1/3-mix -1/3 0 1/3

1/4-1/4-1/2-mix 0 -1/4 1/4

2/5-1/5-2/5-mix 0 0 0
 p1/p2/p3-mix p1·0 + p2·(-2) + p3·1 p1·1 + p2·0 + p3·(-1) p1·(-1) + p2·2 + p3·0

The lowest entries in the first six rows are -1, -2, -1, -1/3, -1/4, and 0. So, among these six pure
and mixed strategies, the one with the highest guarantee for Ann is the 2/5-1/5-2/5-mix. But keep
in mind that there are infinitely many rows, there might be rows with better guarantees coming.
Or not?

Let's now move to a general 3 × 3 bimatrix.

Left Middle Right

Up A1,1, B1,1 A1,2, B1,2 A1,3, B1,3

Middle A2,1, B2,1 A2,2, B2,2 A2,3, B2,3

Down A3,1, B3,1 A3,2, B3,2 A3,3, B3,3

Ann's task is to maximize the minimum of the three values p1·A1,1 + p2·A2,1 + p3·A3,1, p1·A1,2 +
p2·A2,2 + p3·A3,2, and p1·A1,3 + p2·A2,3 + p3·A3,3, by choosing her three probabilities p1, p2, and
p3 (non-negative, with the sum of the three probabilities equal to 1). We could reformulate this
by saying that

Ann has to choose four values, v and p1, p2, and p3,
obeying
o p1 ≥ 0,
o p2 ≥ 0,
o p3 ≥ 0,
o p1+p2+p3=1,
o v ≥ p1·A1,1 + p2·A2,1 + p3·A3,1,
o v ≥ p1·A1,2 + p2·A2,2 + p3·A3,2, and
o v ≥ p1·A1,3 + p2·A2,3 + p3·A3,3,
such that v is maximized under these restrictions.

This is a so-called linear program.

Using methods from this rather sophisticated topic, (which will not be covered in this book) one
finds that in the example ROCK-SCISSORS-PAPER2 above, p1=2/5, p2=1/5, p3=2/5 is indeed
the only mixed maximin strategy, with mixed security level of 0. For Beth, the mixed
maximin strategy is q1=1/3, q2=1/3, q3=1/3.

2. Mixed Nash Equilibria

A Nash equilibrium of mixed strategies is called a mixed Nash equilibrium.

2.1 Two-player Zero-Sum Games

These special games have two nice features:

von Neumann's Theorem [1928]: Every finite two-person zero-sum game has at least
one Nash equilibrium in mixed strategies. They are the maximin mixed strategies.

Actually von Neumann's proved a more general result, his famous "Minimax Theorem",
from which the theorem above follows easily. Such a Nash equilibrium can also be
obtained by Brown's fictitious play process described in the subsection above:

Theorem (Julia Robinson) [Robinson 1951]: If two players play a zero-sum game in
normal form repeatedly, and if in each round each player chooses the best response
pure strategy against the observed mixed strategy of the total history of the other
player, then the mixed strategies of the whole history converge to a pair of mixed
strategies forming a Nash equilibrium.

2.2 Non Zero-Sum Games or More Players

Things become more complicated in these cases. Let me illustrate this with the following two
examples, one game is zero-sum, the other isn't:

Game 1: Left Right Game 2: Left Right

Up 0, 0 10, -10 Up 0, 0 10, 5

Down 5, -5 0, 0 Down 5, 10 0, 0

Note however that the payoffs for Ann (but not for Beth) are identical in the corresponding
situations.

Let's start with the zero-sum Game 1: Ann's optimal mixed strategy is to choose "Up" with
probability 1/3 (and therefore "Down" with probability 2/3), and Beth best chooses "Left" with
probability 2/3. The expected payoff for Ann in this mixed Nash equilibrium is 10/3 for Ann, and
therefore -10/3 for Beth.

Therefore this strategy is the maximin strategy for Ann if mixed strategies are allowed. But since
the maximin strategy and value only depend on Ann's payoffs, and these don't change between
Game 1 and Game 2, the strategy with 1/3 "Up" and 2/3 "Down" is also the maximin strategy for
Ann in Game 2 (with mixed strategies allowed). Since Game 2 is symmetric, Beth has a similar
maximin strategy of choosing 1/3 "Left" and 2/3 "Right". The expected payoff for both when
both play this strategy is 10/3. But this is not a Nash equilibrium in Game 2---every deviation of
Ann in the direction of more "Up" is rewarded, as is every deviation of Beth in the direction of
more "Left". Actually we will show below that Game 2, if mixed strategies are allowed, has three
mixed Nash equilibria:

In the first, Ann chooses "Up" with probability 2/3 and Beth chooses "Left" with
probability 2/3, having the same expected payoff of 10/3 for both, but no reason to
deviate.
In the second one Ann chooses "Up" and Beth chooses "Right". Payoffs are 10
respectively 5.
The last Nash equilibrium is where Ann chooses "Down" and Beth "Left". Payoffs are 5
and 10.

Although the maximin strategies do not necessarily form a mixed Nash equilibrium for general
games, mixed Nash equilibriua always exist, even for non-zero-sum games with an arbitrary
number of players:

Nash's Theorem (1950): Every finite game has at least one Nash equilibrium in pure or mixed
strategies.

The mathematician John Forbes Nash Jr. (*1928) obtained this result in his Ph.D. dissertation He
was awarded the 1994 Nobel Prize in Economics (shared with Reinhard Selten and John
Harsanyi) for this result and some other papers written around 1950-1954. Mathematically more
challenging were his results about questions in differential geometry (Every Riemann manifold
can be embedded into some Rn) and the theory of partial differential equations (1957: Theorem
of de Giorgi and Nash, answering one of Hilbert's problems). In the sixties he was candidate for
the Fields medal, the Mathematics equivalent of the Nobel Prize. According to John Milnor,
"Nash's prize work is an ingenious but not surprising application of well-known methods, while
his subsequent mathematical work is far more rich and important" [Milnor 1998]. Nash was
suffering from paranoid schizophrenia for a long time.

3. Computing Mixed Nash equilibria

It is relatively easy to check whether a bunch of mixed strategies, one for every player, forms a
Nash equilibrium: According to the Indifference Theorem, all we have to do is to find for every
player all pure strategy best responses to the mixed strategies of the others, and check whether all
pure strategies in the support of that player belong to them. But the question is how to find them.
And this is in general not easy, even when the normal form is given. Without that, if the game is
described by its extensive form, even calculating the normal form may be not feasible, since it
may be far too large. An example of such a game is DMA Soccer II. In the following we assume
that the normal form is given.

What should be done first is to eliminate all strictly dominated pure strategies from the normal
form:

Fact: A strictly dominated pure strategy never occurs in the support of a Nash equilibrium mixed
strategy, but a weakly dominated strategy may.

As mentioned above, for the special two-player zero-sum games, the problem to find Nash
equilibria in mixed strategies can be formulated as a Linear Program. And for such problems,
George Dantzig developed around 1947 a solution method, the so-called "Simplex Algorithm."
This method is in practice quite fast, although there are a few artificial cases where the
algorithms needs long. But later, other algorithms have been found that always find a solution in
time bounded by a polynomial in the number of rows and columns.

For general games, if the numbers of rows and columns are small, the Indifference Theorem can
be used. Then it suffices to solve a few systems of linear equations to obtain all mixed Nash
equilibria. We will describe details below for two-player games and bimatrices of sizes 2 × 2, 2 ×
n, 3 × 3, and also for three players with two options each.

This method gets infeasible for large number of options. Although in a 2 player game where the
normal form has 10 rows and columns, we need to solve systems with up to 10 equations and 10
variables, which is not too difficult using technology, the number of such systems we need to
solve raises already to more than 1,000,000. More sophisticated approaches are needed. For 2-
player games, the best approach so far is an algorithm found by Lemke and Howson. By the way,
they also showed that under that conditions the number of Nash equilibria must be odd ---except
in degenerate cases. Still, this algorithm has exponential running time in the worst case
[Lemke/Howson 1964].

Finally Brown's fictitious play can be used to get approximate values of mixed Nash equilibria.
However, the method does not always work. Moreover convergence may be slow---even after
say 50000 iteration you may not be within say 1% of the probabilities. And if there are more than
one mixed Nash equilibria, it may be that Brown's process converges only to one of them in most
cases.

Here is an applet where you can compute mixed Nash equilibria in zero-sum games. For non-
zero-sum games you can use this one.

In the following we will discuss methods for several small cases.

3.1 Small 2-player zero-sum games
2 × n zero-sum games

If Ann mixes between two choices, the probability p for her first move obviously implies that the
probability for her second move must be 1-p. For such a mix, Ann's payoff provided Beth plays
some of her pure strategies is a linear function in p, with the cases p=0 and p=1 being the payoffs
in the pure cases. So for Beth's ith move we draw the straight line from (0,A1,i) to (1,A2,i). We
then draw the curve always following the lowest of these lines. The largest value of this curve is
the security level for Ann, and the corresponding p-value belongs to Ann's Nash equilibrium
mix.

Look at the following example:

B1 B2 B3 B4

3.5 1 0.5
A1 2

1.25 0.5 2
A2 3

The Nash equilibrium p is

about 0.55, and Ann's guaranteed payoff for this mix is a little more than 1.5. Since the resulting
point lies at the intersection point of the straight lines belonging to B1 and B3, Beth will mix
between these two pure strategies.

3 × n zero-sum games

A similar approach, but now in 3 dimensions, works for the 3 × n case. Ann's mix would be p1,
p2, p3 with p1+p2+p3=1. Any such triple of numbers corresponds to a point on a triangle, using
barycentric coordinates. These are calculated as the ratio of shortest distance of the point to one
side over the shortest distance of the third point of the triangle to that same side. Now we build
straight lines perpendicular to the triangle at the three points, and hang a triangle at these pillars
at heights A1,i, A2,i, A3,i for every pure strategy i of Beth. These triangles are intersecting. The
point on the triangle having most "air" above it until it meets the lowest ceiling is Ann's maximin
mix, and the height until this lowest ceiling there is Ann's security level for mixed strategies.

In the example to the right the triangle and the three pillars are drawn in white. The matrix is
 B1 B2 B3

A1 5 7 4

A2 3 5 6

A2 6 4 5

which is just the matrix of CRASH ROCK SCISSORS PAPER discussed above with 5 added to
each one of Ann's payoff. Grab the object and turn it around.

3.2 Small non zero-sum games

3.2.1 The 2×2 Case

The simplest normal forms are for two-players games where each player has exactly two
strategies. That is, we are facing a payoff matrix of the form

Left Right

Up A1,1, B1,1 A1,2, B1,2

Down A2,1, B2,1 A2,2, B2,2

There are three cases of mixed Nash equilibria: One where Ann mixes and Beth doesn't, one
where Beth mixes and Ann doesn't, and one where both mix.

Let us look at this third case first. Assume that Ann chooses "Up" with probability p, 0 < p < 1
and "Down" with probability 1-p. In the same way, Beth chooses "Left" with probability q, 0 < q
< 1 and "Right" with probability 1-q. The Indifference Theorem above implies that Ann faces the
same expected payoffs when playing Up or Down provided Beth keeps mixing. These expected
payoffs are q·A1,1+(1-q)·A1,2 and q·A2,1+(1-q)·A2,2. Therefore

q·A1,1+(1-q)·A1,2 = q·A2,1+(1-q)·A2,2
Similar, in a Nash equilibrium Beth must face the same payoffs with both strategies when facing
Ann mixing as described above. Therefore

p·B1,1+(1-p)·B2,1 = p·B1,2+(1-p)·B2,2

We have two linear equations with two variables p and q. It is even simpler than the general case,
since p does not occur in the first equation and q not in the second. They are not interrelated, and
can therefore be solved separately. For the first we get

q·(A1,1-A1,2)+A1,2 = q·(A2,1-A2,2)+A2,2

or

q = (A2,2-A1,2)/A1,1+A2,2-A1,2-A2,1)

In the same way, the second equation can be solved for p as

p = (B2,2-B2,1)/(B1,1+B2,2-B2,1-B1,2).

Of course both these values have to be between 0 and 1 in order to get a mixed Nash equilibrium.

The other two cases are easy to analyze: Assume Ann mixes, plays "Up" with probability p and
"Down" with probability 1-p, but that Beth plays the pure strategy "Left". Then the Indifference
Theorem implies that A1,1=A2,1. Then both "Up" versus "Left" as well as "Down" versus "Left"
are pure Nash equilibria, and every value of p between 0 and 1 would produce a mixed strategy
for Ann that would form a Nash equilibrium with "Left". Therefore we would have infinitely
many mixed Nash equilibria, with two pure ones as extreme cases. The other cases are similar.
So ordinarily we would have at most one mixed Nash equilibrium, with both Ann and Beth
really mixing, or we would have infinitely many of them.

3.2.2 The 3×3 Case

The general payoff bimatrix in this case is:

Left Middle Right

Up A1,1, B1,1 A1,2, B1,2 A1,3, B1,3

Middle A2,1, B2,1 A2,2, B2,2 A2,3, B2,3

 Down A3,1, B3,1 A3,2, B3,2 A3,3, B3,3

We have one case where Ann mixes between all three strategies, three cases where Ann mixes
just between two of them, and three cases where Ann uses a pure strategy, and the same holds for
Beth. Thus we can in principle pair any of Ann's seven cases with any of Beth's seven cases to
get 49 possible patterns for Nash equilibria.

Let us start with the most interesting pattern where both Ann and Beth mix between all their
three strategies. Assume that Ann chooses "Up" with probability p1, "Middle" with probability
p2, and "Down" with probability 1-p1-p2. In the same way, Beth chooses "Left" with probability
q1, "Middle" with probability q2, and "Right" with probability 1-q1-q2. The Indifference
Theorem above gives us two double equations, namely

q1A1,1+q2A1,2+(1-q1-q2)A1,3 = q1A2,1+q2A2,2+(1-q1-q2)A2,3 = q1A3,1+q2A3,2+(1-q1-q2)A3,3

p1B1,1+p2B2,1+(1-p1-p2)B3,1 = p1B1,2+p2B2,2+(1-p1-p2)B3,2 = p1B1,3+p2B2,3+(1-p1-p2)B3,3

Each of these double equations can be broken into two equations. So what we really have is a
system of four linear equations in the four variables p1, p2, q1, and q2.

q1A1,1+q2A1,2+(1-q1-q2)A1,3 = q1A2,1+q2A2,2+(1-q1-q2)A2,3

q1A1,1+q2A1,2+(1-q1-q2)A1,3 = q1A3,1+q2A3,2+(1-q1-q2)A3,3

p1B1,1+p2B2,1+(1-p1-p2)B3,1 = p1B1,2+p2B2,2+(1-p1-p2)B3,2

p1B1,1+p2B2,1+(1-p1-p2)B3,1 = p1B1,3+p2B2,3+(1-p1-p2)B3,3

q1 and q2 occur in the first and the second equation, and p1 and p2 in the third and the fourth, so
the first two equations can be solved separately from the last two. We therefore can express
q1 and q2 in fairly complicated expressions by the coefficients A1,1, ... A3,3, and p1 and p2 in a
similar way by B1,1, ... B3,3. In general we get one solution.

The next case is where Ann mixes between two strategies, say between "Up" and "Middle" with
probabilities p and 1-p, and Beth between all three as before. We get three equations with three
variables

q1A1,1+q2A1,2+(1-q1-q2)A1,3 = q1A2,1+q2A2,2+(1-q1-q2)A2,3

pB1,1+(1-p)B2,1 = pB1,2+(1-p)B2,2

pB1,1+(1-p)B2,1 = pB1,3+(1-p)B2,3
If both Ann and Beth mix between two strategies, essentially the formulas for the 2×2 case can
be used.

In the same way as above, if one player mixes and the other plays a pure strategy in a Nash
equilibrium, some of the payoff coefficients of the matrix, in a row or a column, are identical,
and the mixing player could use any mix between the strategies involved to get a Nash
equilibrium.

Mixed Strategies

To introduce the idea of mixed strategies we will consider the Battle of the Disciplines at Lower
State University. The president of the school has received a gift for a new building. He has
asked the Dean of the Sciences, Al Einstein, and the Dean of the Performing Arts, Vaughn
Karian, what sort of building to construct: a performance hall or a new lab building. The
president has stipulated that the two must agree on the project or nothing will be built.

The payoff table for the Deans is

Vaughn Karian

Hall Lab

Hall 1, 2 0, 0
Al Einstein
Lab 0, 0 2, 1

Which project will be built? To answer the question, begin by trying to find a solution to the
game.

Are there dominant strategies? No.

Are there dominated strategies? No.
Is/are there Nash Equilibria? The 'best responses' are

Einstein's Best Response to Karian Karian's Best Response to Einstein

bE(HK)=H bK(HE)=H

bE(LK)=L bK(LE)=L

There are two Nash equilibria, but is either of them more likely to be the outcome of the game?
What strategies could be played by Al? H, or L, or flip a coin? Using the coin to decide which
strategy to play is called a mixed strategy. Previously we worked only with pure strategies. A
pure strategy would be for Al to play Hall with no uncertainty.

Suppose Al plays Lab 2/3 of the time and Hall 1/3 of the time. Vaughn plays Hall 2/3 of the
time and Lab 1/3 of the time. We can construct a 2-way table to represent the probability of the
different outcomes.

Vaughn Karian

Hall Lab

Hall 2/9 1/9 1/3

Al Einstein
Lab 4/9 2/9 2/3

2/3 1/3

Was anything gained by playing a mixed strategy? The last column corresponds to Vaughn's
expected payoff from playing the mixed strategy against Al's pure strategies. The last row shows
Al's expected payoffs from playing the mixed strategy against Vaughn's pure strategies. There is
still no dominant strategy.

Vaughn
Karian

V's mix against E's

Hall Lab
pure strategy

Hall 1, 2 0, 0 2/3, 4/3

Al
Einstein Lab 0, 0 2, 1 2/3, 1/3

Al's mix against K's pure

1/3, 2/3 4/3, 2/3
strategy

It doesn't seem anything is gained from the mixed strategy, since the mixed strategy doesn't
dominate either Hall or Lab for either player. But wait. Does it make sense for Al to play a pure
strategy against Vaughn's mixed strategy? If Al plays a pure strategy of Lab against Vaughn's
mixed strategy then Al's expected payoff is 2/3 ( = 0x2/3 + 2x1/3 ); playing a pure strategy of
Hall against Vaughn's mixed strategy is no better for Al, 2/3 (=1x2/3 + 0x1/3). The symmetry of
the game tells us that we would reach a similar conclusion for Vaughn's use of a pure strategy
against mixed play by Al. So, we conclude that there is a third Nash equilibrium: Both play a
mixed strategy.

What is the expected value of Al's utility from playing the game when both of them use the
proposed mixed strategy?

Al: (2/9)x1 + (1/9)x0 + (4/9)x0 + (2/9)x2 = 6/9

Similarly for Vaughn

Vaughn: (2/9)x2 + (1/9)x0 + (4/9)x0 + (2/9)x1 = 6/9

Redrawing the payoff table

Vaughn Karian

Hall Lab V's mix

Hall 1, 2 0, 0 2/3, 4/3

Al Einstein
Lab 0, 0 2, 1 2/3, 1/3

Al's mix 1/3, 2/3 4/3, 2/3 2/3, 2/3

These payoffs can be better for them than playing a pure strategy against an opponent's mixed
strategy. Given that neither of the pure strategy equilibria is a focal point of the game, the
solution to the game is for both of them to play a mixed strategy.

Consider a second example.

To kill time while waiting to be contacted by extraterrestrials, Ursa Maigeur and Paul Ariss are
playing the following normal form game:

Paul Ariss

Left Right

Left 1, 0 0, 1
Ursa Maigeur
Right 0, 1 1, 0
There is no solution to this game in pure strategies. It is possible to find a Nash equilibrium in
mixed strategies. Suppose that Paul and Ursa both play Left with probability 1/2. When they do
this we can expand the payoff table as follows:

Paul Ariss

Left Right Mix

1/2,
Left 1, 0 0, 1
1/2

1/2,
Ursa Maigeur Right 0, 1 1, 0
1/2

1/2,
Mix 1/2, 1/2 1/2, 1/2
1/2

Paul would be indifferent between Left, Right and Mix in response to a play of Mix by Ursa.
The same is true for Ursa. There is an intersection of their conjectures at {Mix, Mix}, so this is
the Nash equilibrium.

Consider a third example.

A Day at the Mall: Nole's Soles and Grace's Laces

At the local mall there are two shoe stores, Nole's Soles and Grace's Laces. Their strategies
include the window displays to be used in their respective stores. The decision is made
simultaneously each Monday morning. They then must live with their choice for the rest of the
week. The payoff matrix, or strategic form of the game, is shown below. The numbers are
hundreds of shoppers stopping in to examine the goods.

Grace's Laces

U M1 M2 D

U 1, 0 4, 2 2, 4 3, 1
Nole's
M 2, 4 2, 0 2, 2 2, 1
Soles
D 4, 2 1, 4 2, 0 3, 1
Suppose that for aesthetic reasons Grace will never use window displays of type M1 or D.
Otherwise, the two store owners play strategies with the probabilities shown along the margins
of the 2-way table shown below. The probabilities in the cells are the joint probabilities resulting
from the corresponding strategy pairs.

Grace's Laces
U M1 M2 D
U .08 0 .12 0 .2
Nole's
M .12 0 .18 0 .3
Soles
D .2 0 .3 0 .5
.4 0 .6 0

The expected payoffs when both managers are playing mixed strategies are

Payoff
Nole .08x1+.12x2+.2x4+.12x2+.18x2+.3x2=2.32

Grace .12x4+.2x2+.12x4+.18x2=1.72

Before explaining the importance of these expected payoffs in the solution of the game we need
a bit more background.

Definition: The support of a mixed strategy is the set of pure strategies used to construct the
mixed strategy.

Implication: Grace's mixed strategy is a best response to Nole's play if and only if each of the
pure strategies in its support is a best response to Nole's play. Since each pure strategy is a best
response, any (probability) weighted average of them is also a best response.

Look again at Nole's and Grace's original game again:

Grace's Laces

U M1 M2 D

U 1, 0 4, 2 2, 4 3, 1
Nole's
M 2, 4 2, 0 2, 2 2, 1
Soles
D 4, 2 1, 4 2, 0 3, 1
Their best response plays are

Best Responses
Nole Grace
b (UG) = D
N
b (UN) = M2
G

bN(M1G) = U bG(MN) = U
bN(M2G) = U, M, D bG(DN) = M1
bN(DG) = U, D

From which we can see that there is just one Nash equilibrium, shown by the shaded cell in the
payoff matrix.

Consider a specific best response of Nole: bN(DG) = U, D. To a play of D by Grace, Nole's best
response would be either U or D. In fact, in response to a play of D by Grace, if Nole plays U
with probability p and D with probability 1-p, then this weighted average is a best response to
DG since it beats playing a pure strategy of M: Note 3p + 3(1-p) > 2.

Can you see that whenever Grace plays M2, any probability weighted average of U, M, and D is
as good as any pure strategy for Nole? Can you explain why Grace will never construct a mixed
strategy involving D? Pick some probabilities and demonstrate it for yourself.

So, we have shown that a mixed strategy can be a best response. There must be a reason to
construct such best response mixed strategies in order to make it worth the effort. Indeed, there
are three reasons to make the effort.

The Meaning of playing a mixed strategy.

Three Reasons to Play a Mixed Strategy

Reason 1. A pure strategy that is undominated by other pure strategies may be dominated by a
mixed strategy.

Reason 2. Playing a mixed strategy can keep your opponent off balance. The worst case payoff
of a mixed strategy may be better than the worst case payoff of a pure strategy.

Reason 3. In a game without a pure strategy Nash equilibrium, a mixed strategy may result in a
Nash equilibrium.
 Finding Mixed Strategy Equilibria

Question 1.

Wumpus

Run Hide

Run 60 , 20 0, 0

Hunter

Hide 0, 0 20 , 60

What is the mixed strategy equilibrium of the above game?

(Denote by p be the probability of "Run" for the Wumpus, and q the probability of "Run" for the
hunter.)

p=1/4, q=1/4

p=1/2, q=1/2

p=1/4, q=3/4

p=3/4, q=1/4

Question 2.
Your accounting department announces that due to an error in its procedures, the numbers in the
game from question 1 are wrong, and actually each number should be multiplied by 2. Does this
change the equilibrium?
 Yes

No

Maybe

It depends

Question 3.
The deities Mars and Venus often do battle to create the weather conditions on Earth. Venus
prefers extreme temperatures (especially heat), while Mars prefers temperate conditions. The
payoffs (expressed in Points of Wrath) are given below.

Venus

Warm Chill

Warm 20 , 0 0 , 10

Mars

Chill 0 , 90 20 , 0

What is the unique mixed-strategy equilibrium of the above game?

(Let p be the probability of "Warm" for Mars, and q the probability of "Warm" for Venus.)

p=9/10, q=1/2

p=1/2, q=1/10
 p=1/2, q=1/2

p=1/10, q=1/10

Question 4.
In the above game, who earns more Points of Wrath (on average) in equilibrium, Mars or Venus?

Mars

Venus

Same

It depends

GRADE

Finding Mixed Strategy Equilibria

Question 1.

Wumpus

Run Hide

Hunter Run 60 , 20 0, 0
 Hide 0, 0 20 , 60

What is the mixed strategy equilibrium of the above game?

(Denote by p be the probability of "Run" for the Wumpus, and q the probability of "Run" for the
hunter.)

p=1/4, q=1/4

p=1/2, q=1/2

p=1/4, q=3/4

p=3/4, q=1/4

Question 2.
Your accounting department announces that due to an error in its procedures, the numbers in the
game from question 1 are wrong, and actually each number should be multiplied by 2. Does this
change the equilibrium?

Yes

No

Maybe

It depends
Question 3.
The deities Mars and Venus often do battle to create the weather conditions on Earth. Venus
prefers extreme temperatures (especially heat), while Mars prefers temperate conditions. The
payoffs (expressed in Points of Wrath) are given below.

Venus

Warm Chill

Warm 20 , 0 0 , 10

Mars

Chill 0 , 90 20 , 0

What is the unique mixed-strategy equilibrium of the above game?

(Let p be the probability of "Warm" for Mars, and q the probability of "Warm" for Venus.)

p=9/10, q=1/2

p=1/2, q=1/10

p=1/2, q=1/2

p=1/10, q=1/10

Question 4.
In the above game, who earns more Points of Wrath (on average) in equilibrium, Mars or Venus?

Mars
 Venus

Same

It depends

Nash Equilibrium in Pure Strategies

Nash equilibrium is one of the central solution concepts for games. The basic idea of a Nash
equilibrium is that if each player chooses their part of the Nash equilbrium strategy, then no
other player has a reason to deviate to another strategy. A simple example is a coordination
game, such as the one in the figure below.

Figure 1: Coordination game payoffs

Nash Equilibrium in Pure Strategies

In this game, both (L, l) and (R, r) are Nash equilibria. If Player 1 chooses L then Player 2 gets 1
by playing l and 0 by playing r; if Player 1 chooses R then Player 2 gets 2 by playing r and 0 by
playing l. The two stratgies L and R for Player 1 and the two strategies l and r for Player 2 are
called "pure strategies" and the strategy pairs (L, l) and (R, r) are called "pure strategy
equilibria."

Nash Equilibrium in Mixed Strategies

Some games, such as Rock-Paper-Scissors, don't have a pure strategy equilibrium. In this game,
if Player 1 chooses R, Player 2 should choose p, but if Player 2 chooses p, Player 1 should
choose S. This continues with Player 2 choosing r in response to the choice S by Player 1, and so
forth.

Figure 1: Rock-Paper-Scissors game payoffs

In games like Rock-Paper-Scissors, a player will want to randomize over several actions. If a
player randomly chooses a pure strategy, we say that the player is using a "mixed strategy." In a
pure strategy a player chooses an action for sure, whereas in a mixed strategy, he chooses a
probability distribution over the set of actions available to him.

A pure strategy defines a specific move or action that a player will follow in every possible
attainable situation in a game. Such moves may not be random, or drawn from a distribution, as
in the case of mixed strategies.

A strategy consisting of possible moves and a probability distribution (collection of weights)

which corresponds to how frequently each move is to be played. A player would only use a
mixed strategy when she is indifferent between several pure strategies, and when keeping the
opponent guessing is desirable - that is, when the opponent can benefit from knowing the next
move.

We start with a constant-sum game: for every possible outcome of the game, the utility of
Player 1 plus the utility of Player 2, adds to a constant. For example, if two firms are
competing for market shares, then .

The possible outcomes are best represented in a table.

We can solve Battle of the Networks as follows: from Network 1's point of view, it is better to
show a sitcom, whether Network 2 shows a sitcom or sports. The strategy ``Show a Sitcom'' is
said to dominate the strategy ``Show Sports'' for Network 1. So Network 1 will show a sitcom.
Similarly, Network 2 is better off showing sports whether Network 1 shows a sitcom or sports. In
other words, Network 2 also has a dominating strategy. So Network 2 shows sports. The
resulting outcome, namely 51% viewer share to Network 1 and 49% to Network 2, is
an equilibrium, since neither of the two players in this game can unilaterally improve their
outcome. If Network 1 were to switch from sitcom to sports, its viewer share would drop 5%,
from 51% to 46%. If Network 2 were to switch from sports to sitcom, its viewer share would
also drop, from 49% to 44%. Each network is getting the best viewer share it can, given the
competition it is up against.

In a general 2-person game, Strategy i for Player 1 dominates Strategy k for Player 1
if for every j. Similarly, Strategy j for Player 2 dominates Strategy for Player 2
if for every i.

In a general 2-person game, Strategy i for Player 1 and Strategy j for Player 2 is an equilibrium if
the corresponding outcome has the following property: is the largest element in
column j and is the largest element in row i. When such a pair i,j exists, a pure strategy for
each player provides an optimal solution to the game. When no such a pair exists, the players
must play mixed strategies to maximize their gains (see Section 1.1.2).

There is an easy way to relate constant-sum games and zero-sum games. Let and be the
payoffs of players 1 and 2, respectively, in a constant-sum game: . Now consider a
new set of payoffs of the form

Clearly, , so we now have a zero-sum game. Using the relation , we

can rewrite the payoffs and as:
These are positive, linear transformations of utility, and so, according to the expected utility
theorem (see 45-749), they have no effect on decisions. For the Battle of the Networks example,
the new table becomes

Each firm has two strategies, either Stay put, or Adopt the new technology. Firm 1 has an
incentive to adopt the new technology: in the event Firm 2 stays put, then Firm 1 gets the
competitive advantage a, and in the event Firm 2 adopts the new technology, then Firm 1 erases
its competitive disadvantage -a. So, whatever Firm 2's decision is, Firm 1 is better off adopting
the new technology. This is Firm 1's dominant strategy. Of course, the situation for Firm 2 is
identical. So the equilibrium of Competitive Advantage is for both firms to adopt the new
technology. As a result, both firms get a payoff of 0. This may seem like a paradox, since the
payoffs would have been the same if both firms had stayed put. But, in Competitive Advantage,
neither firm can afford to stay put. The firms in this game are driven to adopt any technology that
comes along. Take, for example, the hospital industry. Magnetic Resonance Imaging (MRI) is a
new technology that enhances conventional X rays. It allows doctors to see body damage in ways
that were not previously possible. Once MRI became available, any hospital that installed an
MRI unit gained a competitive advantage over other hospitals in the area. From a public policy
standpoint, it may not make much sense for every hospital to have its own MRI unit. These units
are expensive to buy and to operate. They can eat up millions of dollars. Often, one MRI unit
could handle the traffic of several hospitals. But an individual hospital would be at a competitive
disadvantage if it did not have its own MRI. As long as hospitals play Competitive Advantage,
they are going to adopt every new technology that comes along.

The two-person games we have encountered so far have had unique pure strategy equilibria.
However, a two-person zero-sum game may have multiple equilibria. For example, consider the
game:
Each player can play indifferently strategy A or C and so there are four pure strategy equilibria,
corresponding to the four corners of the above table. Note that these equilibria all have the same
payoff. This is not a coincidence. It can be shown that this is always the case. Every equilibrium
of a 2-person zero-sum game has the same payoff. For pure strategy equilibria, there is a simple

proof: Suppose two equilibria had payoffs (a,-a) and (b,-b) where . If these two solutions
lied on the same row or column, we would get an immediate contradiction to the definition of an
equilibrium. Let (a,-a) lie in row i and column k and let (b,-b) lie in row jand column . The
subtable corresponding to these two rows and columns looks as follows

Since (a,-a) is an equilibrium, we must have (from Player 1) and (from Player
2). Similarly, since (b,-b) is an equilibrium, we must have and . Putting these
inequalities together, we get

This implies that a = b, which completes the proof.

It is clear that advertising on television is a powerful marketing tool. If Company 1 switches

from not advertising to advertising on television, its profits go up 20%, everything else being
equal, when Company 2 does not advertise on TV, and they go up 35% when Company 2
advertises on TV; the same is true if the roles are reversed. In other words, advertising on
television is a dominant strategy for each of the companies. Hence the equilibrium is when both
companies advertise on television, with a payoff of $27 million. We say that an
outcome is efficient if there is no other outcome that pays both players at
least as much, and one or both players strictly more. Namely, the pair i,j is efficient if there is no
pair k,l satisfying and and . Obviously, the
equilibrium in the cigarette advertizing game is not efficient. There is one efficient outcome,
however, in this game: that is when neither company advertises on TV. Then both companies
enjoy a payoff of $50 million. The ban of cigarette advertizing on television in 1971 only left the
strategy ``No TV Ads'' and forced the industry into the efficient outcome! In large part, this is
why profits went up in 1971.

It often happens in variable-sum games that the solution is not efficient. One of the biggest
differences between constant-sum and variable-sum games is that solutions to the former are
always efficient, whereas solutions to the latter rarely are.

Strategy set
A player's strategy set defines what strategies are available for them to play.
A player has a finite strategy set if they have a number of discrete strategies available to them.
For instance, in a single game of Rock-paper-scissors, each player has the finite strategy set
{rock, paper, scissors}.
A strategy set is infinite otherwise. For instance, an auction with mandated bid increments may
have an infinite number of discrete strategies in the strategy set {$10, $20, $30, ...}.
Alternatively, the Cake cutting game has a bounded continuum of strategies in the strategy set
{Cut anywhere between zero percent and 100 percent of the cake}.
In a dynamic game, the strategy set consists of the possible rules a player could give to
a robot or agent on how to play the game. For instance, in the Ultimatum game, the strategy set
for the second player would consist of every possible rule for which offers to accept and which
to reject.
In a Bayesian game, the strategy set is similar to that in a dynamic game. It consists of rules for
what action to take for any possible private information.
Choosing a strategy set
In applied game theory, the definition of the strategy sets is an important part of the art of
making a game simultaneously solvable and meaningful. The game theorist can use knowledge
of the overall problem to limit the strategy spaces, and ease the solution.
For instance, strictly speaking in the Ultimatum game a player can have strategies such as: Reject
offers of ($1, $3, $5, ..., $19), accept offers of ($0, $2, $4, ..., $20). Including all such strategies
makes for a very large strategy space and a somewhat difficult problem. A game theorist might
instead believe they can limit the strategy set to: {Reject any offer ≤ x, accept any offer > x; for x
in ($0, $1, $2, ..., $20)}.
Pure and mixed strategies[edit source]

A pure strategy provides a complete definition of how a player will play a game. In particular, it
determines the move a player will make for any situation he or she could face. A
player's strategy set is the set of pure strategies available to that player.
A mixed strategy is an assignment of a probability to each pure strategy. This allows for a
player to randomly select a pure strategy. Since probabilities are continuous, there are infinitely
many mixed strategies available to a player, even if their strategy set is finite.
Of course, one can regard a pure strategy as a degenerate case of a mixed strategy, in which that
particular pure strategy is selected with probability 1 and every other strategy with probability 0.
A totally mixed strategy is a mixed strategy in which the player assigns a strictly positive
probability to every pure strategy. (Totally mixed strategies are important for equilibrium
refinement such as trembling hand perfect equilibrium.)
Mixed strategy[edit source]

Illustration[edit source]
A B
Consider the payoff matrix pictured to the right (known as
a coordination game). Here one player chooses the row and the other
chooses a column. The row player receives the first payoff, the A 1, 1 0, 0
column player the second. If row opts to play A with probability 1
(i.e. play A for sure), then he is said to be playing a pure strategy. If B 0, 0 1, 1
column opts to flip a coin and play A if the coin lands heads and B if
the coin lands tails, then she is said to be playing a mixed strategy,
and not a pure strategy. Pure coordination game
Significance[edit source]
In his famous paper, John Forbes Nash proved that there is an equilibrium for every finite game.
One can divide Nash equilibria into two types. Pure strategy Nash equilibria are Nash equilibria
where all players are playing pure strategies.Mixed strategy Nash equilibria are equilibria where
at least one player is playing a mixed strategy. While Nash proved that every finite game has a
Nash equilibrium, not all have pure strategy Nash equilibria. For an example of a game that does
not have a Nash equilibrium in pure strategies, see Matching pennies. However, many games do
have pure strategy Nash equilibria (e.g. theCoordination game, the Prisoner's dilemma, the Stag
hunt). Further, games can have both pure strategy and mixed strategy equilibria.
“The lesson content has been compiled from various sources in public domain including but not limited to the
internet for the convenience of the users. The university has no proprietary right on the same.”

Rai Technology University Campus

Dhodballapur Nelmangala Road, SH -74, Off Highway 207, Dhodballapur Taluk, Bangalore - 561204
E-mail: info@raitechuniversity.in | Web: www.raitechuniversity.in