Lecture 8

Strong Duality Results

September 22, 2008

 Lecture 8

Outline

• Slater Condition and its Variations

• Convex Objective with Linear Inequality Constraints

• Quadratic Objective over Quadratic Constraints

• Representation Issue

• Multiple Dual Choices

Convex Optimization 1
 Lecture 8

General Convex Problem

Primal Problem
minimize f (x)
subject to gj (x) ≤ 0, j = 1, . . . , m
aTi x = bi, i = 1, . . . , r
x∈X
Assumption used throughout the lecture
Assumption 1.
• The objective f and all gj are convex

• The set X ⊆ Rn is nonempty and convex

• The optimal value f ∗ is finite

Convex Optimization 2
 Lecture 8

Slater Condition for Inequality Constrained Problem

Convex Primal Problem

minimize f (x)
subject to gj (x) ≤ 0, j = 1, . . . , m
x∈X
Slater Condition There exists a vector x̄ ∈ X ∩ domf such that
gj (x̄) < 0 for all j = 1, . . . , m.
Observation: The Slater condition is a condition on the constraint set only
when domf = Rn
Theorem Let Assumption 1 and the Slater condition hold. Then:
• There is no duality gap, i.e., q ∗ = f ∗
• The set of dual optimal solutions is nonempty and bounded

Convex Optimization 3
 Lecture 8

Proof
Consider the set V ⊂ Rm × R given by
V = {(u, w) | g(x)  u, f (x) ≤ w, x ∈ X}
The Slater condition is illustrated in the figures below.

The set V is convex by the convexity of f , gj0 s, and X .

Convex Optimization 4
 Lecture 8

Proof continues
The vector (0, f ∗) is not in the interior of the set V . Suppose it is, i.e.,
(0, f ∗) ∈ intV . Then, there exists an  > 0 such that (0, f ∗ − ) ∈ V
contradicting the optimality of f ∗.
Thus, either (0, f ∗) ∈ bdV or (0, f ∗) 6∈ V . By the Supporting Hyperplane
Theorem, there exists a hyperplane passing through (0, f ∗) and supporting
the set V : there exists (µ, µ0) ∈ Rm × R with (µ, µ0) 6= 0 such that
µT u + µ0 w ≥ µ0 f ∗ for all (u, w) ∈ V (1)
This relation implies that µ  0 and µ0 ≥ 0.
Suppose that µ0 = 0. Then, µ 6= 0 and relation (1) reduces to
inf µT u = 0
(u,v)∈V

On the other hand, by the definition of the set V , since µ  0 and µ 6= 0,

we have
inf µT u = inf µT g(x) ≤ µT g(x̄) < 0 - a contradiction
(u,v)∈V x∈X

Convex Optimization 5
 Lecture 8

Hence, µ0 > 0 (the supporting hyperplane is nonvertical), and by dividing

with µ0 in Eq. (1), we obtain

inf {µ̃T u + w} ≥ f ∗
(u,v)∈V

with µ̃  0. Therefore,
n o
q(µ̃) = inf f (x) + µ̃ g(x) ≥ f ∗
T
with µ̃  0
x∈X
implying that q ∗ ≥ f . By the weak duality [q ∗ ≤ f ∗], it follows that
∗

q ∗ = f ∗ and µ̃ is a dual optimal solution.

Convex Optimization 6
 Lecture 8

We now show that the set of dual optimal solutions is bounded. For any
dual optimal µ̃  0, we have
n o
∗ T
q = q(µ̃) = inf f (x) + µ̃ g(x)
x∈X
≤ f (x̄) + µ̃T g(x̄)
m
X
≤ f (x̄) + max {gj (x̄)} µ̃j
1≤j≤m
j=1
Pm ∗
Therefore, min1≤j≤m {−gj (x̄)} j=1 µ̃j ≤ f (x̄) − q implying that
m
X f (x̄) − q ∗
kµ̃k ≤ µ̃j ≤
j=1 min1≤j≤m {−gj (x̄)}

Convex Optimization 7
 Lecture 8

Example

l∞-Norm Minimization

minimize kAx − bk∞

Equivalent to

minimize t
subject to aTj x − bj − t ≤ 0, j = 1, . . . , m
bj − aTj x − t ≤ 0, j = 1, . . . , m
(x, t) ∈ Rn × R
The vector (x̄, t̄ ) given by
x̄ = 0 and t̄ =  + max |bj | for some  > 0
1≤j≤m
satisfies the Slater condition
We refer to a vector satisfying the Slater condition as a Slater vector

Convex Optimization 8
 Lecture 8

Convex Objective Over Linear Constraints

Def. A vector x0 is in the relative interior of X when there exists a ball
Br (x0) = {x ∈ Rn | kx − x0k ≤ r} such that
Br (x0) ∩ aff (X) ⊆ X
The relative interior of a convex set X is the set of points that are interior
with respect to the affine hull of X .
Primal Problem with Linear Inequality Constraints
minimize f (x)
subject to Ax ≤ b,
x∈X
Theorem Let Assumption 1 hold. Assume that there exists a vector x̃ such
that
Ax̃ ≤ b and x̃ ∈ relint(X) ∩ relint(domf )
Then
• There is no duality gap (q ∗ = f ∗) and

• A dual optimal solution exists

Convex Optimization 9
 Lecture 8

Implications

• When domf = Rn, the relative interior condition has to do only with
the constraint set.

• The “relative interior” condition is always satisfied

(a) When domf = Rn and X = Rn:
convex function over Rn and linear constraints
(b) LP’s: X = Rn and f (x) = cT x.

• In these cases, by the preceding theorem:

• Strong duality holds and optimal dual solution exists

Convex Optimization 10
 Lecture 8

Examples

Examples not satisfying the “relative interior” condition

√
minimize e− x1x2
subject to x1 ≤ 0, x ∈ R2
Here, X = R2 and the feasible set C is C = {x | x1 = 0, x2 ∈ R}. The
domain of f is the set {x | x1 ≥ 0, x2 ≥ 0}, thus,
relint(X) ∩ relint(domf ) = {x | x1 > 0, x2 > 0}.
However, none of the feasible vectors lies in relint(X ∩ domf ). Hence, the
condition fails. Furthermore, there is a gap - Homework.
√
Consider the same constraint set with an objective f (x) = − x1. The
condition is not satisfied. In this case, there is no gap, but a dual optimal
solution does not exist.

Convex Optimization 11
 Lecture 8

Slater for a General Convex Problem

minimize f (x)
subject to gj (x) ≤ 0, j = 1, . . . , m
Ax = b, Dx  d
x∈X
∗
Theorem Let f be finite, and assume that:
• There is a feasible vector satisfying the Slater condition, i.e., there is
x̄ ∈ Rn such that
g(x̄) ≺ 0, Ax̄ = b, Dx̄  d, x̄ ∈ X ∩ domf
• There is a vector satisfying the linear constraints and belongs to the
relative interior of X , i.e., there is x̃ ∈ Rn such that
Ax̃ = b, Dx̃  d, x̃ ∈ relint(X) ∩ relint(domf )
Then:
• There is no duality gap, i.e., q ∗ = f ∗
• The set of dual optimal solutions is nonempty
Note: Conditions only on the constraint set
Theorem says nothing about the existence of primal optimal solutions

Convex Optimization 12
 Lecture 8

Quadratic Objective over Quadratic Constraint

minimize xT Q0x + aT0 x + b0

subject to xT Qj x + aTj x + bj ≤ 0, j = 1, . . . , m
x ∈ Rn
Theorem Let each Qi be symmetric positive semidefinite matrix. Let f ∗
be finite. Then, there is no duality gap [q ∗ = f ∗] and the primal optimal
set is nonempty.
Note:
• The theorem says nothing about the existence of dual optimal solutions
• Opposed to the Slater and the relative interior condition, which say
nothing about the existence of the primal optimal solutions
Example
minimize kxk
subject to Ax = b
x ∈ Rn

Convex Optimization 13
 Lecture 8

Representation Issue
minimize −x2
subject to kxk ≤ x1
x ∈ X, X = {(x1, x2) | x2 ≥ 0}
• The relaxation of the inequality constraint results in a dual problem, for
which the dual value q(µ) is −∞ for any µ ≥ 0 (verify yourself).
Thus, q ∗ = −∞, while f ∗ = 0.
However, a closer look into the constraint set reveals
C = {(x1, x2) | x1 ≥ 0, x2 = 0}
Thus the problem is equivalent to
minimize −x2
subject to x1 ≥ 0, x2 = 0
x ∈ R2
• There is no gap for this problem!!! Why?
The duality gap issue is closely related to the “representation” of the
constraints [the model for the constraints]

Convex Optimization 14
 Lecture 8

Multiple Choices for a Dual Problem

minimize f (x)
subject to gj (x) ≤ 0, j = 1, . . . , m
Ax = b, Dx  d
x∈X
• There are multiple choices for the set of “inequalities” to be relaxed

• How to choose the “right one”? There is no general rule

• Keep box constraints, “sign” constraints if any
• Keep constraints for which the dual function can be easily evaluated

• Will for all of them “no duality-gap” relation hold?

• What do we know about duality gaps when “multiple dual choices”

exist?

Convex Optimization 15
 Lecture 8

Relax-All Rule

minimize f (x)
subject to gj (x) ≤ 0, j = 1, . . . , m
x ∈ Rn
A linear equality is represented by two linear inequalities.
Theorem Let f ∗ be finite. Consider a dual corresponding to the relaxation
of all the constraints, and assume that there is no duality gap. Then, there
is no duality gap when partially relaxing the constraints.

Convex Optimization 16