Lec 18 Unidirectional Search PDF

Optimization Techniques
Multi-variable Unconstrained Optimization:

Unidirectional Search
Dr. Nasir M Mirza

Email: nasirmm@yahoo.com
In this Lecture
In this lecture we will discuss two important methods
that deal with Multi-variable Unconstrained
Optimization:
Multi Dimensional Plots and
Uni-directional Search
Example 4.2
The following function of two variables has distinct local minimum points and
inflection point:
f = (x - 2)2 + (x - 22)2
Let us draw a mesh grid
graph in three dimensions
using MATLAB.
% matlab function for 3D plot
clear all
[X,Y] = meshgrid(-2:.1:2);
Z = ( X - 2.0).^2 + (X - 2.*Y.*Y).^2 ;
mesh(X, Y, Z) %interpolated
axis tight; hold on
plot3(x,y,z,'.','MarkerSize',15)
SURFACE
PLOT using
MATLAB

clear all
[X,Y] = meshgrid(-2:.1:2);
Z = ( X - 2.0).^2 + (X - 2.*Y.*Y).^2 ;
surf(X,Y,Z,'FaceColor','red','EdgeColor','none')
camlight left; lighting phong
Example 4.2 continued

Object function is: f = (x - 2)2+ (x - 22)2
f
= 2( x 2) + 2( x 2 y 2 ) = 4 + 4 x 4 y 2
x
f
= 8 y ( x 2 y 2 ) = 8 xy + 16 y 3
y
gradient vector =
f
2 f
2
x
H= 2
f
yx
4
4
x
4
y
=
8 xy + 16 y
2 f
8y
xy 4
=
2
2
f 8 y 8 x + 48 y
y 2
CONTOUR
PLOT using
MATLAB
Next step is to
draw a contour
graph and identify
points of interest.
(Here we see (2, 1)T and (2, 1)T as
important points.
clear all
[X,Y] = meshgrid(-2:.05:5);
Z = ( X - 2.0).^2 + (X - 2.*Y.*Y).^2 ;
contour(X,Y,Z, 3000)
SURFACE
PLOT using
MATLAB

clear all
[X,Y] = meshgrid(-2:.05:5);
Z = ( X - 2.0).^2 + (X - 2.*Y.*Y).^2 ;
surf(X,Y,Z,'FaceColor','red','EdgeColor','none')
hold on
Example 4.2 continued

Two types of contour plots showing possible maximum or minimum points
Example 4.2
Optimality condition (first order):
f
= 2( x 2) + 2( x 2 y 2 ) = 4 + 4 x 4 y 2 0
x
f
= 8 y ( x 2 y 2 ) = 8 xy + 16 y 3 0
y
Possible solutions (x, y) are following:

(1, 0) ; (2, -1); (2, 1)
Let us check with second optimality condition for each point:
For point (1, 0):
4 0
H =
0 8
Principal minors are:

A1 = a11= 4, A2 = det(A)= -32
It is then inflection point and Function value = 2.
Example 4.2
Let us check with second optimality condition for point (2, -1)
4 8
H =
8 32

A1 = a11= 4, A2 = det(A)= 128 64 = 64
It is then minimum point and Function value = 0.
Let us check with second optimality condition for point (2, +1)
4 8
H =
8 32

A1 = a11= 4, A2 = det(A)= 128 64 = 64
Example 4.3
The following function of two variables has only one minimum point
but two distinct local maximum points and two inflection points:
f(x) = 25x2 12x4 6xy + 25y2 24x2y2 12y4.
Let us first draw 3D graph:

clear all
[X,Y] = meshgrid(-1.5:.05:1.5);
Z = 25.*X.^2-12.*X.^4-6.*X.*Y
+25.*Y.^2 -24.*(X.^2).*(Y.^2) 12.*Y.^4.
surf(X,Y,Z,'FaceColor','red','Edge
Color','none')
hold on
Example 4.3
% matlab function for this graph

clear all
[X,Y] = meshgrid(-1.5:.1:1.5);
Z = 25.*X.^2-12.*X.^4-6.*X.*Y +25.*Y.^2 -24.*(X.^2).*(Y.^2) - 12.*Y.^4.
surf(X, Y, Z);
colormap(jet);
Example 4.3

clear all
[X,Y] = meshgrid(1.5:.01:1.5);
Z = 25.*X.^2-12.*X.^4-6.*X.*Y
+25.*Y.^2 -24.*(X.^2).*(Y.^2) 12.*Y.^4.
contour(X, Y, Z, 200);
Example 4.3
Object function is: f(x) = 25x2 12x4 6xy + 25y2 24x2y2 12y4.
f
= 50 x 48 x 3 6 y 48 xy 2
x
f
= 6 x + 50 y 48 x 2 y 48 y 3
y
2 f
2
x
H= 2
f
yx
2 f
2
2
xy 50 144 x 48 y
=
2
f
6 96 xy
y 2
2
2
50 48 x 144 y
6 96 xy
Example 4.3
Optimality condition (first order):
f
= 50 x 48 x 3 6 y 48 xy 2 = 0
x
f
= 6 x + 50 y 48 x 2 y 48 y 3 = 0
y
Possible solutions (x, y) are following:

(-0.764, 0.764) ; (-0.677, -0.677); (0, 0); (0.764, -0.764) ; (0.677, 0.677);
Let us check with second optimality condition for each point:
For point (-0.764, 0.764):
62 50
H =
50 62

A1 = a11= -62, A2 = det(A)= 1344
It is then maximum point and
Function value = 16.33
Example 4.3
Let us check with second optimality condition for point (-0.677, -0.677)
38 50
H =
50 38

A1 = a11= -38, A2 = det(A)= -1056
It is then inflection point and Function value = 10.08
Let us check with second optimality condition for point (0, 0)
50 6
H =
6 50

A1 = a11= 50, A2 = det(A)= 2464
Example 4.3
Let us check with second optimality condition for point (+0.677, +0.677)
38 50
H =
50 38

A1 = a11= -38, A2 = det(A)= -1056
It is then inflection point and Function value = 10.08
Let us check with second optimality condition for point (0.764, -0.764)
62 50
H =
50 62

A1 = a11= -62, A2 = det(A)= 1364
It is then maximum point and
Function value = 16.3
Example 4.3
The graph shows
these points on
contour plot as
maximum,
minimum and
inflection points;
Maximum
Inflection
Minimum
Inflection
Maximum
Unidirectional Search Method

A unidirectional search is a one-dimensional search
performed by comparing function values only along a
specific direction.
Usually a unidirectional search is performed from a
point x(t) in a specific direction s(t).
Only points that lie on a line passing through x(t) in
direction of s(t) are allowed.
Any arbitrary point on that line can be expressed as
x(a) = x(t) + as(t).
The parameter a is a scalar quantity and it is measure
of distance of point x(a) from x(t).
Unidirectional Search Method

The above equation is a vector equation in multidimensional space.
Hence when a is known the x(a) can be estimated.
Any positive or negative value of a will create x(a) on
the line for search.
When a is zero, the current point x(t) is obtained.
When we want minimum point, we convert objective
function in terms of single variable a by putting all xi as
x(a) = x(t) + as(t).
Once optimal a is found the corresponding point is found.
Example 3.2.1
Consider the objective function:
Minimize: f(x, y) = (x 10)2 + (y 10)2
Step 1: First draw the contour plot of this function. The contour
plot has lines where any two points on the line have the same
function value.
The optimal point is a minimum at (10, 10)T. The function value

is zero at this point.
Step 2: Let us say that a current point selected is x(t) = (2, 1)T.
and the function value in a search direction s(t) = (2, 5)T from
the current point.
x(a) = x(t) + as(t) ; vector equation
x(a) = 2 + a2 ; y(a) = 1 + a5; scalar equations
Example 3.2.1
Now we find the equation for the straight line passing
through (2, 1)T and direction (2, 5)T as:
(x 2)/2 = (y 1)/5 ;
y 1 = (5/2)(x 2).
Now let us find a point on this line as x = 3 then y = 3.5

and we can estimate a from above equations as a = 0.5
If the point on line is with x = 9 and y = 18.5,
then a = 3.5
The function value for (3, 3.5)T is 91.25
And at point (9, 18.5)T the value is 73.25.
Then at a = (3.5 + 0.5)/2 = 2, then x = 6, y = 11 and
function value is 17. So, approx. opt. is (6, 11)T.
Example 3.3.1
Consider the Himmelblau function:
Minimize:
f(x, y) = (x2 + y 11)2 + (x + y2
7)2
in the interval 0 < x < 5;
Step 1: This function is plotted as
Figure. It appears from the plot that

a minimum point is at (3, 2)T.

clear all
[X,Y] = meshgrid(0:.05:5);
Z = ( X.*X + Y - 11.0).^2 + (X + Y.*Y - 7.0).^2 ;
surf(X, Y, Z);
colormap(jet);
Example 3.3.1: contour lines

A contour line is a collection of
points having identical function
values.
The contour plot for the object
function is shown here.
The continual decrease in the
function value of the successive
contour lines as they approach a
point means that the point is
minimum point.
% matlab function for contour plot
clear all
axis([0 5 0 5])
hold on
[X,Y] = meshgrid(-2:.05:5);
Z = ( X.*X + Y - 11.0).^2 + (X + Y.*Y - 7.0).^2 ;
contour(X, Y, Z, 200)
Example 3.3.1
Step 1: We now choose an initial
point x(0) = (1, 1)T
Size reduction parameter = (2, 2)T
Also choose e = 0.001
Step 2: We create a two
dimensional hypercube (a square)
around x(0) such that
X(1) = (0, 0)T , x(2) = (2, 0)T ,
X(3) = (0, 2)T , x(4) = (2, 2)T
Step 3: The function values at these
points are:
f(0) = 106 ; f(1) = 170 ; f(2) = 74
f(3) = 90 ; f(4) = 26
The minimum is at (2, 2)T. Let us say this as x* =(2, 2)T
Example 3.3.1
Step 4: Since x* is not x(0), we
now set x(0) = (2, 2)T and got to
step 2.
Step 2: We create a new square
around this x(0) with = 1 such
that
X(1) = (1, 1)T , x(2) = (3, 1)T ,
X(3) = (1, 3)T , x(4) = (3, 3)T
Step 3: The function values at
these points are:
f(0) = 26 ; f(1) = 106 ; f(2) = 10
f(3) = 58 ; f(4) = 26
The minimum is at (3, 1)T. The function

value is 10.
Let us say this as x* =(3, 1)T
Example 3.3.1
Step 4: Since x* is not x(0), we
now set x(0) = (3, 1)T and go to
step 2.
around this x(0) with = 1 such
that
X(1) = (2, 0)T , x(2) = (4, 0)T ,
X(3) = (2, 2)T , x(4) = (4, 2)T
these points are:
f(0) = 10 ; f(1) = 74 ; f(2) = 34
f(3) = 26 ; f(4) = 50
The minimum is at (3, 1)T where function

value is 10. It is same as x(0).
Let us say this as x* =(3, 1)T
Example 3.3.1
Step 4: Since x* is same as x(0),
we now set x(0) = (3, 1)T and go
to step 2 after reducing step size =
0.5
around this x(0)=(3, 1)T with = 0.5
such that
X(1) = (2.5, 0.5)T , x(2) = (3.5, 0.5)T
,
X(3) = (2.5, 1.5)T , x(4) = (3.5, 1.5)T
these points are computed and
minimum is at x(4) = (3.5, 1.5)T
The minimum is at (3.5, 1.5)T
And function value is 9.125.
Let us say this as x* =(3.5, 1.5)T
Example 3.3.1
Step 4: Since x* is not same as
x(0), we now set x(0) = (3.5, 1.5)T
and go to step 2 with step size =
0.5
around this x(0)=(3, 1)T with = 0.5
such that
X(1) = (3, 1)T , x(2) = (4, 1)T ,
X(3) = (3, 2)T , x(4) = (4, 2)T
these points are
f(0) =9.125; f(1) = 10 ;
The minimum is at (3, 2)T
f(2) = 40; f(3) = 0; f(4) = 50
This is the best minimum so far as f=0.
the minimum is f(3) = 0.0
Example 3.3.1
It is important to see that although
we have found a minimum point but
the algorithm does not terminate at
this step. We may continue.
It is clear that the convergence
depends upon the initial cube size
and location and the chosen size
reduction parameter.
Small size may lead to premature
convergence and large may never
lead to convergence.
The minimum is at (3, 2)T

This is the best minimum so far as f=0.
Example: Suppose f(x, y) = 2xy + 2x x2 2y2

Using the steepest ascent method to find the next point if we are
moving from point (-1, 1).
f
f
= 2 y + 2 2x
= 2x 4 y
x
y
f
1
,
1
)
x
2(1) + 2 2(1) 6
At (1,1), f = f
=
=
(1,1) 2(1) 4(1) 6

y
Let g (h) = f (-1 + 6h,1 6h)

Next step is to find h that maximize g(h)
f ( x) = 2 xy + 2 x x 2 2 y 2
g (h) = f (-1 + 6h,1 6h) = ... = 7 + 72h 180h 2
Setting g ' (h) = 0 yields
72 360h = 0 h = 0.2
If h = 0.2 maximizes g(h), then x = -1+6(0.2) = 0.2 and y = 16(0.2) = -0.2 would maximize f(x, y).
So moving along the direction of gradient from point (-1,
1), we would reach the optimum point (which is our next
point) at (0.2, -0.2).

Lec 18 Unidirectional Search PDF

Uploaded by

Copyright:

Available Formats

Lec 18 Unidirectional Search PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec 18 Unidirectional Search PDF

Uploaded by

Copyright:

Available Formats

Optimization Techniques

Multi-variable Unconstrained Optimization:

Dr. Nasir M Mirza

% matlab function for 3D plot

Example 4.2 continued

% matlab function for 3D plot

Example 4.2 continued

Possible solutions (x, y) are following:

Principal minors are:

Principal minors are:

Principal minors are:

Let us first draw 3D graph:

% matlab function for this graph

% matlab function for 3D plot

Possible solutions (x, y) are following:

Principal minors are:

Principal minors are:

Let us check with second optimality condition for point (0, 0)

Principal minors are:

Principal minors are:

Principal minors are:

Unidirectional Search Method

Unidirectional Search Method

The optimal point is a minimum at (10, 10)T. The function value

x(a) = 2 + a2 ; y(a) = 1 + a5; scalar equations

Now let us find a point on this line as x = 3 then y = 3.5

Step 1: This function is plotted as

Figure. It appears from the plot that

% matlab function for 3D plot

Example 3.3.1: contour lines

The minimum is at (3, 1)T. The function

The minimum is at (3, 1)T where function

The minimum is at (3.5, 1.5)T

And function value is 9.125.

Let us say this as x* =(3.5, 1.5)T

The minimum is at (3, 2)T

f(2) = 40; f(3) = 0; f(4) = 50

This is the best minimum so far as f=0.

the minimum is f(3) = 0.0

The minimum is at (3, 2)T

Example: Suppose f(x, y) = 2xy + 2x x2 2y2

(1,1) 2(1) 4(1) 6

Let g (h) = f (-1 + 6h,1 6h)

You might also like