Chapter 6 PDF

E1
251 Linear and Nonlinear

Op2miza2on

Chapter 6: Convex op/miza/on problems
and duality
1
6.1 Convex op/miza/on problem: defini/on
Defini/on: convex set
A set Ω is called a convex set if, for every x, y ∈Ω, we

have tx + (1− t)y ∈Ω for all t ∈[0,1].
Defini/on: convex op/miza/on problem
A problem of minimizing a convex function within a

convex set is called convex optimization problem
2
6.2 Convexity of func/ons on convex sets
Theorem 6.2A (compare with Th. 4.2A)
A funtion f (x) is convex in convex set Ω if and only if for

every pair of vectors x and y in Ω, we have
f (y) ≥ f (x) + (y − x)T ∇f (x)
Theorem 6.2B (compare with Th. 4.2B)
A function f (x) is convex in convex set Ω if and only

if its Hessian F(x) is positive semi-definite for all x ∈Ω.
3
6.3 Minimiza/on of convex func/ons with convex constraints
6.3.1 Three types of constraints that lead to convex feasible set:
1) h(x) = 0, where h is an affine function
2) h(x) ≤ 0, where h is a convex function
3) h(x) ≥ 0, where h is a concave function
4
6.3.2 Local minimum is global minimum
Theorem 6.3A (compare with Th. 4.1A)

For a convex function f (x), if x* is a local minimum
in convex set Ω, it is also a global minimum in Ω.
Proof:
1) if x* is not a global minimum, then there exists a y, such that
f (y) < f (x* ).
2) For any point in the line {α y + (1 − α )x*; 0 < α < 1}, which
is included in Ω because of its convexity, we have
by convexity of f (x) that
f (α y + (1 − α )x* ) ≤ α f (y) + (1− α ) f (x* )
!## #"### $ !###"###$
T1 T2 5
3) Since f (y) < f (x* ) by assumption, T2 < f (x* ). This implies
T1 = f (α y + (1 − α )x* ) < f (x* ).
4) For α aribitrarily small, this contradicts the the fact that

x* is a local minimum.
5) Hence, for a convex function f (x) having a local minimum

at x* , there is no such y satisfying f (y) < f (x* ).
Hence, a local minimum is also a global minimum.
6
Corollaries (6.3B and 6.3C)
For a convex optimization problem, if more than one point,

satisfy FONC-Min, their values of the objective function
should be equal.
For a convex optimization problem, any local maximum

should be a non-interior point.
7
Theorem 6.3D
For a convex oprtimization problem, If a point x* satify
FONC (KKT) for minimum, it will also satisfy SONC.
Proof:
The augmented Hessian
m m
L(x, λ , µ ) = F(x) + ∑ λi H(x) + ∑ µi G(x),
i=1 i=1
is positive semi-definite due to the following:

(i) F(x) is positive semi-definite
(ii) Hi s are zero matrices (affine functions)
(iii) Gi s are positive semi-definite
(iv) µi s are non-negative.
Hence because of Th. 5.3E, SONC (Th. 5.3C)
is satisfied. 8
6.4 The dual problem of a convex op/miza/on problem
6.4.1 Definition of dual function
The primal op/miza/on problem
Minimize f (x)
Subject to h(x) = 0 (! n → ! m affine function)
and g(x) ≤ 0 (! n → ! m convex function)
The Lagrangian func/on
⎡ u ∈" m ⎤
L(x, y) ! f (x) + u h(x) + v g(x) where y = ⎢
T T
⎥ ∈" m+ p
⎢⎣ v ∈" p
⎥⎦
9
The dual func/on defini/on
q(y) ! infx ( L(x, y)) where y ∈" m+ p
The dual op/miza/on problem
Maximize q(y) subject to v ≥ 0 (y = [u v]T )
The domain of the dual func/on
D ! { y q(y) > −∞}
Properties of q and D
q is concave and D is convex.

10
6.5 The duality rela/ons
6.5.1 The weak duality relation
q(y) ≤ f (x) for any arbitrary x and y.
6.5.2 The strong duality relation
If (x* , y* ) satisfy KKT conditions, x* and y* solve the primal and

dual problems respectively, the values of the objective function at
the solution points are equal.
11
6.5.3 The converse of strong duality for restricted
problems
The restricted primal problem : Minimize the strictly convex
function f (x) subject to g(x) ≤ 0 (! n → ! m strictly convex)
The Lagrangian : L(x, y) ! f (x) + yT g(x), q(y) ! infx ( L(x, y))
The dual problem : Maximize q(y) subject to y ≥ 0
Converse of strong duality :

Suppose y maximizes q(y) with corresponding x such that
q(y) = L(x, y), x solves the primal problem.
12
Note: The converse of strong duality is also applicable
to the following more general problem:
Minimize f (x)
Subject to h(x) = 0 (! n → ! m affine function)
and g(x) ≤ 0 (! n → ! m convex function),
where at least one of the functions in {f (x) g(x)}
is strictly convex.
This is possible because the proof depends only

on the strict convexity of L(x, y).
13

Chapter 6 PDF

Uploaded by

Copyright:

Available Formats

Chapter 6 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 6 PDF

Uploaded by

Copyright:

Available Formats

E1

251 Linear and Nonlinear

Deﬁni/on: convex set

A set Ω is called a convex set if, for every x, y ∈Ω, we

Deﬁni/on: convex op/miza/on problem

A problem of minimizing a convex function within a

A funtion f (x) is convex in convex set Ω if and only if for

A function f (x) is convex in convex set Ω if and only

6.3.1 Three types of constraints that lead to convex feasible set:

1) h(x) = 0, where h is an affine function

2) h(x) ≤ 0, where h is a convex function

3) h(x) ≥ 0, where h is a concave function

Theorem 6.3A (compare with Th. 4.1A)

4) For α aribitrarily small, this contradicts the the fact that

5) Hence, for a convex function f (x) having a local minimum

For a convex optimization problem, if more than one point,

For a convex optimization problem, any local maximum

is positive semi-definite due to the following:

The primal op/miza/on problem

The Lagrangian func/on

The dual op/miza/on problem

Maximize q(y) subject to v ≥ 0 (y = [u v]T )

The domain of the dual func/on

D ! { y q(y) > −∞}

q is concave and D is convex.

6.5.1 The weak duality relation

q(y) ≤ f (x) for any arbitrary x and y.

6.5.2 The strong duality relation

If (x* , y* ) satisfy KKT conditions, x* and y* solve the primal and

The Lagrangian : L(x, y) ! f (x) + yT g(x), q(y) ! infx ( L(x, y))

The dual problem : Maximize q(y) subject to y ≥ 0

Converse of strong duality :

This is possible because the proof depends only

You might also like