Math. Program., Ser.

B 97: 177–207 (2003)

Digital Object Identifier (DOI) 10.1007/s10107-003-0442-2

Zhi-Quan Luo

Applications of convex optimization in signal processing

and digital communication∗
Received: January 22, 2003 / Accepted: April 29, 2003
Published online: May 28, 2003 – © Springer-Verlag 2003

Abstract. In the last two decades, the mathematical programming community has witnessed some spectacular
advances in interior point methods and robust optimization. These advances have recently started to signifi-
cantly impact various fields of applied sciences and engineering where computational efficiency is essential.
This paper focuses on two such fields: digital signal processing and communication. In the past, the widely
used optimization methods in both fields had been the gradient descent or least squares methods, both of which
are known to suffer from the usual headaches of stepsize selection, algorithm initialization and local minima.
With the recent advances in conic and robust optimization, the opportunity is ripe to use the newly developed
interior point optimization techniques and highly efficient software tools to help advance the fields of signal
processing and digital communication. This paper surveys recent successes of applying interior point and
robust optimization to solve some core problems in these two fields. The successful applications considered
in this paper include adaptive filtering, robust beamforming, design and analysis of multi-user communication
system, channel equalization, decoding and detection. Throughout, our emphasis is on how to exploit the
hidden convexity, convex reformulation of semi-infinite constraints, analysis of convergence, complexity and
performance, as well as efficient practical implementation.

1. Introduction

Over the last two decades, there have been significant advances in the research of interior
point methods [61, 92] and conic optimization [85]. Powerful optimization models and
efficient algorithmic tools as well as software [58] have been produced. Recently these
advances have begun to significantly impact various applied science and engineering
fields, such as mechanical structure design [11], VLSI circuit design [18, 77], systems
and control [13], discrete optimization [35, 36, 91], statistics and probability [12, 31],
where efficient optimization is essential. The goal of this paper is to review some existing
as well as new optimization models and tools for convex conic and robust optimization,
and to survey recent successes in applying these modern optimization tools to solve
several core problems in signal processing and communication.
The application of modern optimization methods to signal processing and digital
communication is well motivated. In the past thirty years, the work-horse algorithms in
the field of digital signal processing and communication have been the gradient descent

Z-Q. Luo: Department of Electrical and Computer Engineering, McMaster University, Hamilton, Ontario,
L8S 4K1, Canada, e-mail: luozq@mcmaster.ca
New address after April 1, 2003: Department of Electrical and Computer Engineering, University of Minne-
sota, Minneapolis, MN 55455, USA
∗This research was supported in part by the Natural Sciences and Engineering Research Council of Canada,
Grant No. OPG0090391, and by the Canada Research Chair program.
178 Z.-Q. Luo

and the least squares algorithms. While these algorithms have served their purpose
well, they suffer from slow convergence and sensitivity to the algorithm initialization
and stepsize selection, especially when applied to ill-conditioned or nonconvex
problem formulations. This is unfortunate, since many design and implementation
problems in signal processing and digital communication naturally lead to nonconvex
optimization formulations, the solution of which by the gradient descent algorithm
usually works poorly. Examples of such problems include channel equalization [26,57],
maximum likelihood detection [56], pulse shaping filter design [23, 24], sidelobe
control in digital beamforming [80], to name just a few. The main problem with
applying the least squares or the gradient descent algorithms directly to the nonconvex
formulations is slow convergence and local minima. One powerful way to avoid
these problems is to derive an exact convex reformulation or a convex relaxation of
the original nonconvex formulation. Once a convex reformulation or relaxation is
obtained, we can be guaranteed of finding the globally optimal design efficiently without
the usual headaches of stepsize selection, algorithm initialization and local minima.
Another challenging problem in signal processing and communication is the robust-
ness of the obtained solution against data perturbation and implementation errors. These
errors can be caused by measurement noise, small sample size, model mismatch or the
finite arithmetic of digital hardware. The pervasive nature of these errors in signal pro-
cessing and communication underscores the need and importance of robust solutions.
Without properly modelling the required robustness, the “optimal designs” obtained
from noisy data can be useless, especially when truncated for implementation on digital
hardware. Unfortunately, due to the lack of tractable mathematical models and optimiza-
tion techniques, most of the previous research in signal processing and communication
do not address the robustness issue directly nor satisfactorily.
In recent years, the mathematical programming community has witnessed extraor-
dinary advances in interior point methods [61], Semidefinite Programming (SDP) [76],
Second Order Cone programming (SOCP) [50], and robust optimization techniques [8].
These new advances and the related software tools [70, 74, 86] have allowed us to
model many problems which were previously considered intractable as convex pro-
grams and solve them much more efficiently than the classical gradient descent algo-
rithm. Boyd and Vandenberghe [76] were among the first to successfully apply the
interior point optimization methods to the real world engineering problems includ-
ing robust control (see the book by Boyd, El-Ghaoui, Feron and Balakrishnan [13]).
Previously, the control community had mainly been looking for closed form solu-
tions for their problems. With the availability of efficient numerical tools (such as
those based on interior point methods), many challenging control problems can now
be efficiently solved numerically with high accuracy (not in closed form). This new
approach has significantly broadened the class of control problems that are considered
solvable. These exciting studies show the great potential of convex reformulation of
classical engineering design problems and their subsequent solution by interior point
methods.
Equally exciting are the recent applications of convex optimization techniques in
signal processing and communication. Successful examples of this kind include detec-
tion and estimation [5, 27, 47, 69], circuit design [77], channel equalization [26, 45, 57],
filter design [4, 23, 24, 75, 87], digital beamforming [46, 80], and communication
Applications of convex optimization in signal processing and digital communication 179

system design [52, 56, 82, 89, 94]. In the case where the underlying signal process-
ing and communication problems defy a convex reformulation, a high quality convex
relaxation solution may be sought (see the quasi-maximum likelihood detection exam-
ple in Section 4). These examples illustrate the great impact of modern optimization
techniques on the fields of signal processing and digital communication. In fact, the
impact is felt in both directions: the process of formulating practical signal process-
ing and communications problems also helps to inspire optimizers to further strengthen
and generalize their existing theory and methods. For example, the existing robust opti-
mization theory of Ben-Tal and Nemirovski [8] explicitly models the uncertainty in
the input data and seeks a solution which is robust against input data perturbations.
In practice there is an additional source of perturbation caused by truncation error
when the obtained solution is to be implemented on digital hardware. Without prop-
erly modelling this implementation error, the obtained solution may not be sufficiently
robust (see the robust magnitude filter design problem in Section 3). The work of Ben-
Tal and Nemirovski [8–10] does consider implementation error, but instead models
it as a form of input data perturbation. It turns out that we can develop more gen-
eral robust optimization models [55] which explicitly account for both the data per-
turbation and the implementation error. Such extensions (under ellipsoidal uncertainty
set) can be derived easily using the so called S-procedure [90] or using the results
of [9, 10, 55].
Here and throughout, vectors are in lower letters, and matrices are in capital letters.
The vector (or matrix) transpose is expressed by superscript ‘ T ’, while the Hermitian
matrix transpose is denoted by superscript ‘ H ’. The set of n by n symmetric matrices
is denoted by S n ; and the set of n by n real positive semidefinite matrices is denoted
by S+ n . The set of n by n Hermitian matrices is denoted by Hn ; and the set of n by

n Hermitian positive semidefinite matrices is denoted by H+ n . The second order cone

{(t, x) ∈ R | t ≥ x} where ‘ · ’ represents the Euclidean norm, is denoted by

n

SOC(n). For two given matrices A and B, we use ‘A 
B’ (‘A  B’) to indicate
that A − B is positive (semi)-definite, and A • B := i,j Aij Bij = Tr (AB ) to
T

indicate the matrix inner product. The Frobenius norm of A is denoted by AF =

Tr (AAT ).

2. Review of conic optimization and robust optimization

2.1. Conic optimization model and interior point method

Consider a primal-dual pair of linear conic optimization problems:

minimize C • X
(1)
subject to AX = b, X ∈ C

and

maximize bT y
(2)
subject to A∗ y + S = C, S ∈ C ∗
180 Z.-Q. Luo

where A is a linear operator mapping an Euclidean space onto another Euclidean space,
A∗ denotes the adjoint of A, C signifies a pointed, closed convex cone, and C ∗ is its dual
cone. The problems (1)–(2) include many well known special cases such as

• Linear Programming (LP): C = R+

n,


n
• Second Order Cone programming (SOCP): C = SOC(ni ),
i=1
• Semidefinite Programming (SDP): C = S+
n or (Hn ).
+

For ease of exposition, we will focus on the SDP case with C = S+ n . The other cases

can be treated similarly (in fact they are special case of SDP). In practice, sometimes it
is more convenient to work with the so-called rotated Second Order Cone: {(t, s, x) ∈
Rn | ts ≥ x2 , t ≥ 0, s ≥ 0}. This cone is equivalent to the standard SOC(n) via a
simple linear transformation.
Assume that the feasible regions of the SDP pair (1)–(2) have nonempty interiors.
Then we can define the central path of (1)–(2) as {(X(µ), S(µ))} satisfying

A∗ y(µ) + S(µ) = c
AX(µ) = b (3)
X(µ)S(µ) = µI

where µ is a positive parameter. By driving µ → 0 and under mild assumptions, the

central path converges to an optimal primal-dual solution pair for (1)–(2). Notice that
the central path condition (3) is exactly the necessary and sufficient optimality condition
for the following convex problem:

minimize C • X − µ log det(X)

(4)
subject to AX = b, X ∈ S+n.

In other words, the points on the central path corresponds to the optimal solution of (4)
and the associated optimal dual solution. Here the function − log det(X) is called the
barrier function for the positive semidefinite matrix cone S+ n.

Many interior point algorithms follow (approximately) the central path to achieve
optimality. As a result, the iterates are required to remain in a neighborhood of the central
path which can be defined as:


N (γ ) = (X, y, S) | AX = b, A∗ y + S = c, X  0, S  0,

X•S X•S
X1/2 SX 1/2 − I F ≤ γ
n n

With this definition, a generic interior point path-following algorithm can be stated as
follows.
Applications of convex optimization in signal processing and digital communication 181

GENERIC PATH-FOLLOWING ALGORITHM

Given a strictly feasible primal-dual pair (X 0 , y 0 , S 0 ) ∈ N (γ ) with 0 < γ < 1.

Let k = 0.
REPEAT (main iteration)
Let X = Xk , y = y k , S = S k and µk = X • S/n.
Compute a search direction (X k , y k , S k ).
Compute the largest step tk such that
(X + t k X k , y + t k y k , S + t k S k ) ∈ N (γ ).
Set Xk+1 = X + t k X k , y k+1 = y + t k y k , S k+1 = S + t k S k .
Set k = k + 1.
UNTIL convergence.

There are many choices for the search direction (X, y, S). For example, we
can take it as the solution of the following linear system of equations:

A∗ y + S = C − S − A∗ Ay
AX = b (5)
HP (XS + XS) = µI − HP (XS)

where P is a nonsingular matrix and

1
HP (U ) = (P U P −1 + (P U P −1 )T ).
2
Different choices of P lead to different search directions. For example, P = I corre-
sponds to the so-called AHO direction [85].
The
√ standard analysis of path-following interior point methods shows that a total
of O( n log µ0 /) main iterations are required to reduce the duality gap X • S to less
than . Each main iteration involves solving the linear system of equations (5) whose
size depends on the underlying cone C. If C = R+ n (linear programming), the linear

system is of size O(n), implying  each main iteration has an arithmetic complexity of
In the case where C = ni=1 SOC(ni ) (SOCP),
O(n3 ).

the linear system (5) will have
size O( i ni ), so the complexity of solving (5) is O(( i ni )3 ). For the SDP case where
C = S+ n , the size of the linear system (5) is O(n2 ), so the amount of work required to

solve (5) is O(n6 ). Combining the estimates of the number of main iterations with the
complexity estimate per each iteration yields the overall complexity of interior point
methods. In general, the computational effort required to solve SDP is more than that
of SOCP, which in turn is more than that of LP. However, the expressive power of these
optimization models rank in the reverse order.

2.2. Robust optimization

Robust optimization models in mathematical programming have received much atten-

tion recently; see, e.g. [7, 8, 29]. In this subsection we will briefly review some of these
models and some extensions.
182 Z.-Q. Luo

Consider a convex optimization for the form:

minimize f0 (x)
(6)
subject to fi (x) ≤ 0, i = 1, 2, ..., m,
where each fi is convex. In many engineering design applications, the data defining the
constraint and the objective functions may be inexact, corrupted by noise or may fluctu-
ate with time. In such cases, the traditional approach is simply to use the nominal form
of each fi in the formulation of the design problem (6). However, an optimal solution
for the nominal formulation (6) may yield poor performance or become infeasible when
each fi is perturbed. In other words, optimal solutions for (6) may be misleading or even
useless in practice. A more appropriate thing to do is to seek a high quality solution which
can remain feasible and deliver high quality performance in all possible realizations of
unknown perturbation. This principle was formulated rigorously in [7, 8, 29]. Specif-
ically, the data perturbation can be modelled using a parameter vector δ, with δ = 0
representing the nominal unperturbed situation. In other words, we consider a family of
perturbed functions parameterized by δ: fi (x; δ), with δ taken from an uncertainty set
 containing the origin. Then a robustly feasible solution x is the one that satisfies
fi (x; δ) ≤ 0, ∀ δ ∈  or equivalently max fi (x; δ) ≤ 0.
δ∈

Thus, a robustly feasible solution x is, in a sense, strongly feasible, since it is required
to satisfy all slightly perturbed version of the nominal constraint fi (x; 0) = fi (x) ≤ 0.
The robust optimal solution can now be defined as a robust feasible solution which min-
imizes the worst case objective value maxδ∈ f0 (x; δ). This gives rise to the following
formulation:
minimize maxδ∈ f0 (x; δ)
(7)
subject to fi (x; δ) ≤ 0, ∀ δ ∈ , i = 1, 2, ..., m.
Let us assume the perturbation vector δ enters the objective and the constraint func-
tions fi in such a way that preserves convexity, i.e., each fi (x; δ) remains a convex
function for each δ ∈ . As a result, the robust counterpart (7) of the original (nominal
case) convex problem (6) remains convex since its constraints are convex (for each i
and δ) and the objective function maxδ∈ f0 (x; δ) is also convex.
Much of the research in robust optimization is focussed on finding a finite represen-
tation of the feasible region of (7) which is defined in terms of infinitely many constraints
(one for each δ ∈ ). Assume that the uncertainty parameter δ can be partitioned as
δ = (δ0 , δ1 , δ2 ..., δm )T and that the uncertainty set has a Cartesian product structure
 = 0 × 1 × · · · × m , with δi ∈ i . Moreover, assume that δ enters fi (x; δ) in an
affine manner. Under these assumptions, it is possible to characterize the robust feasible
set of many well known classes of optimization problems in a finite way. In particular,
consider the robust linear programming model proposed by Ben-Tal and Nemirovskii [8]:
minimize maxc≤0 (c + c)T x
subject to (ai + ai )T x ≥ (bi + bi ), (8)
for all (ai , bi ) ≤ i , i = 1, 2, ..., m,
Applications of convex optimization in signal processing and digital communication 183

where each i > 0 is a pre-specified scalar. In the above formulation, we have δi =

(ai , bi ) and i = {(ai , bi ) | (ai , bi ) ≤ i }. It is known that the above
robust LP can be reformulated as a SOCP [8]. The references [7, 8, 29] have shown that
the robust counterpart of some other well known convex optimization problems can also
be reformulated in a finite way as a conic optimization problem, often as an SOCP or
SDP.
An important tool in converting a semi-infinite problem to one with finitely many
constraints is the following well-known S-lemma of Yakubovich [90].
Proposition 1 (S-lemma, level set). Suppose p(x) = c̄ +2b̄T x +x T Āx is a quadratic
function with p(x̄) > 0 for some x̄ ∈ Rn , and let  := {x | p(x) ≥ 0}. Then
q(x) = c + 2bT x + x T Ax ≥ 0 for all x ∈ 
if and only if there exists s ≥ 0 such that
q(x) − sp(x) ≥ 0 for all x ∈ Rn .
Recently, a matrix version of the S-lemma has been established [55] for the following
robust QMI:
C + X T B + B T X + X T AX  0 for all X with I − X T DX  0. (9)
Here, the perturbation parameter X enters the LMI in a quadratic manner. The case
where A = 0 (so the perturbation to the LMI is affine) has been previously considered
in [7].
Proposition 2. The robust QMI (9) is equivalent to
     
C BT I 0
∈ Z Z−s  0, s ≥ 0 . (10)
B A 0 −D
This shows that the robust QMI (9) holds if and only if the data matrices (A, B, C, D)
satisfy a certain LMI relation.
Another popular technique to convert certain semi-infinite constraints into a single
convex constraint is the Positive Real Lemma [6] (and the closely related Kalman-
Yakubovich-Popov [KYP] Lemma). For discrete time Finite Impulse Response (FIR)
systems, this lemma can be expressed as follows [5, 24, 27].
Proposition 3. Let {rm ∈ C, −M + 1 ≤ m ≤ M − 1}
be a sequence satisfying
M−1
r−m = r̄m , and define its Fourier transform as R(ejθ ) = m=−M+1 rm e
jmθ . Then

R(ejθ ) ≥ 0 for all θ ∈ [0, 2π) if and only if there exists an X ∈ H+ M×M
such that

M−1−m
Tr (X) = r0 and =0 X+m, = rm , for 1 ≤ m ≤ M − 1.
The importance of Proposition 3 lies in the fact that in signal processing applications
one often encouters design constraints of the spectral mask type which are specified in
terms of the lower and upper bounds on the Fourier transform of the FIR system to be
designed. Such semi-infinite constraints can be appropriately transformed into an LMI
using Proposition 3. Recently there have been some useful extensions of Proposition 3
which show that the condition R(ejθ ) ≥ 0 for all θ ∈ [α, β] ⊂ [0, 2π ) can also be
represented as an LMI (see Proposition 5 and [5, 23]).
184 Z.-Q. Luo

3. Efficient optimization methods for digital signal processing

In this section, we describe some successful applications of convex optimization methods

to digital signal processing. As it turns out, the theory of interior point methods and robust
optimization needs to be appropriately modified or extended to account for the effect
of noise and non-convexity often encountered in signal processing applications. This
is especially true for real-time signal processing applications where data are collected
dynamically and are corrupted by noise. Previously the robust optimization techniques
and interior point algorithms have only been studied in the noise-free deterministic and
convex setting. The major technical issues to be studied are

(a) Convergence (in the stochastic sense) and convergence speed;

(b) Robustness to modelling error and noise statistics;
(c) Numerical stability.

3.1. Stochastic convergence and rate of convergence of IPLS (Interior Point Least
Square) or its variants in a time-varying environment

Consider a discrete-time linear system as follows:

yi = xiT w∗ + vi , i = 1, 2, . . .

where yi ∈ R is the sequence of observations, xi ∈ RM denotes the sequence of input

vectors of size M, w∗ ∈ RM is the unknown deterministic parameter vector or filter that
we wish to estimate, and vi is the additive measurement noise. The parameter estimation
problem is to identify the parameter vector w∗ from input/output pairs {xi , yi }. When
the estimate is to be updated sequentially with the arrival of new data, the problem is
often referred to as adaptive filtering [40]. Define the mean-squared error of a particular
estimate w (at time n) as
n
1 2 1 T
Fn (w) = yi − xiT w = y yn − 2w T pxy (n) + w T Rxx (n)w (11)
n n n
i=1

where
n n
1 1
yn = [y1 , y2 , . . . , yn ]T , pxy (n) = xi yi , Rxx (n) = xi xiT .
n n
i=1 i=1

The least squares estimate of the parameter vector w∗ is the estimate that minimizes
Fn . By solving ∇Fn (w) = 0 we find the well-known optimum linear filter wnls =
Rxx (n)−1 pxy (n).
The above problem is dynamic in that the data are collected one at a time. The tradi-
tional approach for identifying w∗ is the Recursive Leaset Square (RLS) method which
works well asymptotically but requires proper initialization in the transient phase. It
turns out that we can use the concept of analytic center (cf. (3)–(4)) to devise an effi-
cient Interior Point Least Square (IPLS) algorithm which not only enjoys an optimal
Applications of convex optimization in signal processing and digital communication 185

asymptotic convergence rate but also fast transient convergence. To develop the IPLS
algorithm, let us define the following convex feasibility region:

n = w ∈ R | Fn (w) ≤ τn , w ≤ R
M 2 2
, (12)

where τn > 0 is an appropriately chosen scalar and R > 0 is a fixed number. Any vector
w ∈ n is said to be feasible at time n. Obviously, we would like τn to approach zero,
because then any feasible vector w will approximately minimize Fn (w). The additional
inequality w2 ≤ R 2 is important since it ensures that the region n is bounded and
provides the “correct” regularization.
For the above convex feasible region n , we define its analytic center as the vector
wna such that the following logarithmic barrier function
φn (w) = − log(τn − Fn (w)) − log(R 2 − w2 ), w∈ n

is minimized. It remains to specify how τn is updated in each iteration. We choose

R
τn
Fn (wn−1
a
) + β √ ∇Fn (wn−1
a
)2 , (13)
2
where β > 0 is a constant chosen by the user. An important consequence of (13) is that
a
wn−1 remains feasible with respect to the updated region n so that wn−1 a can be used
as a starting point for a Newton procedure to find the analytic center a of
√ w n n . The
condition Fn (wn−1a ) ≤ τ is easily verified with (13). The (R/ 2) term in τ provides a
n n
normalization that is required to show the convergence of the algorithm. The parameter
β is typically used in interior point methods to control the “aggressiveness” with which
constraints are generated. For example, a small β corresponds to a small allowed slack,
a
i.e., wn−1 is left close to the boundary of n . By increasing β we allow more slack and
hence define n more conservatively.
A major advantage of the analytic center based adaptive filtering algorithm is its fast
transient convergence. Generally speaking, the convergence behaviour of an estimation
algorithm is influenced by two factors: the speed of the statistical averaging process and
the decay rate of the initialization parameters. The former is usually dictated by the law
of large numbers and therefore exhibits O(1/n) type of convergence rate. The latter,
which is called the transient behavior, is dependent on how fast the estimation algorithm
can phase out the effect of initialization and is therefore more algorithm specific. Fast
transient convergence is important for accurate estimation of time-varying parameters.
It has been shown [2] that our new adaptive filtering algorithm enjoys a geometric rate of
convergence in the transient phase while the traditional Recursive Least Squares (RLS)
algorithm converges only at the rate of O(1/n).
The details of the Interior Point Least Squares (IPLS) algorithm for adaptive filtering
are given below.
IPLS: An Analytic Center Based Adaptive Filtering Procedure
Step 1: Initialization. Select R > 0, β > 0. Let F0 (w) = 0 and 0 = {w2 ≤ R 2 }.
Let w0a = 0.
Step 2: Updating. For n ≥ 1, update the feasible region n using the new data sample
(xn , yn ) and the new value τn according to (13).
186 Z.-Q. Luo

Step 3: Recentering. Compute the analytic center wna of n by taking Newton steps
a . Set n := n + 1 and return to Step 2.
starting from the previous center wn−1

It has been shown [2, 3] the updating of τn and the computation of an approximate
analytic center wn of n can be efficiently performed with per sample complexity of
O(M 2.2 ). In fact, a constant number of Newton updating iterations are needed at each
recentering step. It was also shown that wn → w∗ as n → ∞. The effectiveness of
IPLS algorithm was demonstrated in a channel tracking application in a Code Division
Multiple Access (CDMA) uplink application [3].

3.2. Robust beamforming

In recent decades, adaptive beamforming has been widely used in wireless commu-
nications, microphone array speech processing, radar, sonar, medical imaging, radio
astronomy, and other areas. Such techniques employ multiple sensors (or antennas) for
signal transmission/reception to improve the system performance. Since the output from
multiple sensors possess distinct spatial correlations depending on the angle of signal
arrival, we can appropriately process the sensor outputs to separate signals arriving from
different directions. This signal processing step is called beamforming.
Mathematically, the output of a narrowband beamformer is given by

y(k) = wH x(k)

where k is the time index, x(k) = [x1 (k), . . . , xM (k)]T ∈ CM×1 is the complex vector
of array observations, w = [w1 , . . . , wM ]T ∈ CM×1 is the complex vector of beam-
former weights, M is the number of array sensors. The observation (training snapshot)
vector is given by

x(k) = s(k) + i(k) + n(k) = s(k)a + i(k) + n(k) (14)

where s(k), i(k), and n(k) are the desired signal, interference, and noise components,
respectively. Here, s(k) is the signal waveform, and a is the signal steering vector. The

x1 x2 xM−1 xM

w1 w2 w M−1 wM

Fig. 1. Beamforming using a linear array of antennas

Applications of convex optimization in signal processing and digital communication 187

optimal weight vector can be found by the maximization of the Signal-to-Interference-

plus-Noise Ratio (SINR) [59]

σs2 |wH a|2

SINR = (15)
wH Ri+n w
where

Ri+n = E (i(t) + n(t)) (i(t) + n(t))H (16)

is the M × M interference-plus-noise covariance matrix and σs2 is the signal power. It is

easy to find the solution for the weight vector by maintaining a distortionless response
towards the desired signal and minimizing the output interference-plus-noise power [59].
Hence, the maximization of (15) is equivalent to [59]

min wH Ri+n w subject to wH a = 1 . (17)

w

In practice, Ri+n is not available and is usually approximated by the data covariance
matrix R. Moreover, since the steering vector a is typically estimated from data and
therefore known only approximately, we are led to consider the following robust formu-
lation of (17):

min wH Rw subject to |wH (a + a)| ≥ 1 for all a ≤ , (18)

w

where | · | denotes the magnitude of a complex number. Note that (18) represents a
modified version of (17): we impose distortionless response constraints for all steering
vectors within a distance  from a, instead of requiring distortionless response for just a
single steering vector a. Unfortunately, the optimization problem (18) involves infinitely
many nonconvex quadratic constraints, which suggests that it might be an intractable
problem. Therefore, the robust convex optimization framework (6)–(7) of Ben-Tal and
Nemirovskii does not apply. Surprisingly, it turns out we can reformulate this problem
exactly as a convex SOCP [80].
Proposition 4. The robust beamforming problem (18) can be transformed into the fol-
lowing SOCP:

min wH Rw subject to wH a ≥ w + 1 , Im wH a = 0 . (19)
w

which can be solved via interior point methods with a complexity of O(M 3.5 ).
The argument for Proposition 4 is simple. First, we use triangle inequality to show that
the nonconvex semi-infinite problem (18) is equivalent to:

min wH Rw subject to |wH a| − w ≥ 1 . (20)

w

The nonlinear constraint in (20) is still nonconvex due to the absolute value operation on
the left hand side. An important observation is that the cost function in (20) is unchanged
when w undergoes an arbitrary phase rotation. Therefore, if w0 is an optimal solution to
188 Z.-Q. Luo

(20), we can always rotate, without affecting the objective function value, the phase of
w0 so that wH a is real. Thus, we can, without any loss of generality, choose w such that
 
Re wH a ≥ 0, Im wH a = 0.

It is not yet known if one can extend Proposition 4 by incorporating other robust con-
straints in (18) to account for the placement of nulls and sidelobe control. The answer
to this is quite relevant to practical digital beamforming applications. However, mathe-
matically, introducing new robust nonconvex quadratic constraints in (18) may destroy
the hidden structure of SOCP. It is not clear how one can reformulate the problem while
retaining the SOCP structure.
Prior to the discovery of the SOCP formulation given in Proposition 4, the same
robust optimization technique was used to design a powerful SOCP based blind mul-
tiuser detector [20]. A closely related (but independent) work [51] used the standard
robust LP model (8) to formulate the robust beamforming problem by replacing the
constraint set of (17) with Re (wH (a + a)) ≥ 1, for all a ≤ . While this formu-
lation does not appear to correspond to the maximization of the worst case SINR (15),
it can be shown, surprisingly, to be also equivalent to the same SOCP (19). Similarly,
the robust approach [15] offered an interesting solution to beamforming using quadratic
programming, although no explicit attempt was made to ensure the worst case SINR is
maximized. Several recent work have further extended the robust beamforming method
(18) to nonstationary and general rank models [46, 48, 64, 67, 81].

Example: Signal look direction mismatch

We now present a working example illustrating the effectiveness of the robust beam-
forming approach. Let us assume a uniform linear array with M = 10 omnidirectional
sensors spaced half a wavelength apart. For each scenario, 100 simulation runs are used to
obtain each simulated point. We assume two interfering sources with plane wavefronts
and the Directions Of Arrival (DOA’s) 30◦ and 50◦ , respectively. Both the presumed
and actual signal spatial signatures are plane waves impinging from the DOA’s 3◦ and
5◦ , respectively. This corresponds to a 2◦ mismatch in the signal look direction. The
Interference-to-Noise Ratio (INR) in a single sensor is equal to 15 dB and the signal
is always present in the training data cell. Four methods are compared in terms of the
mean output SINR: the proposed robust beamformer (19), the SMI beamformer [65], the
LSMI beamformer [19] and the eigenspace-based beamformer [16]. The optimal theo-
retic SINR is also shown in the figures. The SeDuMi convex optimization MATLAB
toolbox [70] has been used to compute the weight vector of our robust beamformer,
where the constant  = 3 has been chosen assuming that the nominal steering vector is
normalized so that aH a = M (= 10). The diagonal loading factor ξ = 0.5 is taken in the
LSMI beamformer. Furthermore, diagonal loading with the same parameter is applied
to our robust technique as well, but only in the case when the data covariance matrix R
is rank deficient (i.e., in the case when the number of snapshots N is less than M).
Fig. 2(a) shows the performance of the methods tested versus the number of training
snapshots N for the fixed SNR = −10 dB. The performance of these algorithms versus
Applications of convex optimization in signal processing and digital communication 189

15
0

−1 10

−2
5
OUTPUT SINR (DB)

−3
OUTPUT SINR (DB)

−4 0

−5
−5
−6

−7 −10
OPTIMAL SINR OPTIMAL SINR
−8 PROPOSED ROBUST BEAMFORMER PROPOSED ROBUST BEAMFORMER
SMI BEAMFORMER −15 SMI BEAMFORMER
−9 LSMI BEAMFORMER LSMI BEAMFORMER
EIGENSPACE−BASED BEAMFORMER EIGENSPACE−BASED BEAMFORMER
−10 −20
0 20 40 60 80 100 −20 −15 −10 −5 0 5
NUMBER OF SNAPSHOTS SNR (DB)

(a) Output SINR versus training sample (b) Output SINR versus SNR.
size N .
Fig. 2. Performance comparison of different robust beamformers

the SNR for the fixed training data size N = 30 is shown in Fig. 2(b). These figures show
a substantial performance gain of the new robust methods as compared to the existing
approaches.

3.3. Magnitude filter design

In the design of finite impulse response (FIR) filters, one often encounters a spectral mask
constraint on the magnitude of the frequency response of the filter (e.g., [63, 66, 68]).
That is, for given L(ejω ) and U (ejω ), constrain the (possibly complex) filter coefficients
gk so that

L(ejω ) ≤ |G(ejω )| ≤ U (ejω ) for all 0 ≤ ω < 2π, (21)

√
or

determine that the constraint cannot be satisfied. Here, j = −1 and G(ejω ) =
−jωk is the frequency response of the filter. The mask functions L(ejω ) and U (ejω )
k gk e
are typically given by industrial standards or by the overall system design specification
of a particular application. Fig. 3 shows a graphical example of a spectral mask constraint
(the dashed lines) for a pulse shaping filter (length L = 48) suggested by a wireless
communication standard (IS95 [42]). A spectral mask constraint can be rather awkward
to accommodate into general optimization-based filter design techniques for two rea-
sons. First, it is semi-infinite in the sense that there are two inequality constraints for
every ω ∈ [0, 2π ). Second, the set of feasible filter coefficients is in general non-convex
due to the lower bound on |G(ejω )|. In order to efficiently solve filter design problems
employing such constraints, we must find a way in which (21) can be represented in a
finite and convex manner.
There are two established approaches [63] to deal with the problem of non-con-
vexity of (21). The first is to enforce additional constraints on the parameters gk so
that G(ejω ) has ‘linear phase’ (i.e., gk possesses a symmetry property). In that case
190 Z.-Q. Luo

0
Relative Magnitude, dB

−10

−20

−30

−40

−50

−60
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
cycles-per-sample f = ω
2π ,
Fig. 3. An example of spectral mask constraint: the frequency magnitude response of the pulse shaping filter
suggested by the IS95 standard

|G(ejω )| becomes a linear function of approximately half the gk ’s, and hence (21) can be
reduced to two semi-infinite linear (and hence convex) constraints. The second approach
to deal with non-convexity is to reformulate (21) in terms of the autocorrelation of the
filter [17, 24, 41, 87, 88]. In particular, if

rm = gk ḡk−m
k

represents the autocorrelation of the
filter, then taking Fourier transform on both sides
of above equation yields R(ejω ) = m rm e−jωm = |G(ejω )|2 , and hence (21) becomes

L(ejω )2 ≤ R(ejω ) ≤ U (ejω )2 for all 0 ≤ ω < 2π, (22)

which amounts to two semi-infinite linear constraints on rm . (Observe that r−m = r̄m
and hence R(ejω ) is real.) Hence, by reformulating the mask constraint in terms of
rm , m ≥ 0, we obtain convex (in fact linear) constraints. Note that the constraint that
R(ejω ) ≥ L(ejω )2 ≥ 0 is sufficient to ensure that a filter gk can be extracted (though not
uniquely) from a designed autocorrelation rm via spectral factorization (also known as
Riesz-Féjer Theorem) [88]. Therefore the nonconvex feasibility problem (21) is com-
pletely equivalent to the convex (linear) feasibility problem (22).
The problem of representing (21) or (22) in a finite manner is more challenging. One
standard, but ad-hoc, approach is to approximate the constraints by discretizing them
uniformly in frequency and enforcing the 2N linear constraints

L(ejωi )2 +  ≤ R(ejωi ) ≤ U (ejωi )2 −  for ωi = 2π i/N , i = 0, 1, . . . , N − 1,

(23)

where N and  are chosen heuristically. For a fixed N , one must choose  to be small
enough so that the over-constraining of the problem at frequencies ωi does not result in
significant performance loss, yet one must choose  to be large enough for satisfaction
Applications of convex optimization in signal processing and digital communication 191

of (23) to guarantees satisfaction of (22) for all 0 ≤ ω < 2π . Recently, a dual parame-
terization method [21] is introduced to treat the semi-infinite mask constraint. However,
this approach may still result in non-convex design problems and thus is subject to risks
of local optima.
If the lower and upper spectral bounds L(ejω ) and U (ejω ) are constant over [0, 2π ],
then we can invoke the Positive Real Lemma (Proposition 3) to transform the spec-
tral mask constraints into some equivalent finite LMI constraints. However, the bounds
L(ejω ) and U (ejω ) are typically piecewise constant over [0, 2π ], so the Positive Real
Lemma does not apply. Fortunately, it turns out that one can derive LMI formulations
of general constraints of the form Re (R(ejω )) ≥ Re (A(ejω )) for all ω ∈ [α, β], where

MA −1
A(ejω ) = k=0 ak e−jωk is a fixed trigonometric polynomial and Re (·) denotes the
real part. Since these LMI formulations apply to segments of the unit circle they natu-
rally incorporate piecewise constant lower/upper bounds in (22) and can be considered
as generalizations of the Positive Real Lemma (cf. Proposition 3). Thus, there exists a
precise finite LMI representation of a large class of spectral mask constraints that results
in convex design problems [5, 23].
Proposition 5. Let 0 ≤ α < β <
2π . Let r = (r−M+1 , · · · , rM−1 )T ∈ C2M−1 and
its Fourier transform as R(e ) = M−1
jω
m=−M+1 rm e
jmω . Then Re (R(ejω )) ≥ 0 for all
(M−1)×(M−1)
ω ∈ [α, β] if and only if there exist X ∈ H+M×M
, Z ∈ H+ , ξ ∈ R such that

r + ξ je0 = L∗ (X) + ^ ∗ (Z; α, β), (24)

where L∗ and ^ ∗ are some linear operators, and e0 is the M-th column of the (2M −
1) × (2M − 1) identity matrix.
Proposition 5 provides a theoretically satisfying characterization of the mask con-
straint which avoids the heuristic approximation of discretization techniques, yet gen-
erates practically competitive design algorithms [23].

Fig. 4. Discrete-time model of baseband PAM communication channel

We can apply this LMI formulation to design several families of FIR filters, includ-
ing those which generate robust ‘chip’ waveforms for digital wireless telephony systems
based on code division multiple access [22, 23]. Specifically, consider a match filtered
baseband PAM (Pulse Amplitude Modulation) digital communication system described
by Figure 4, where the integer K > 0 denotes the oversampling factor and {gk } denotes
the pulse shaping

filter (or chip waveform) to be designed. The data estimates after match
filtering d̂n = k ḡk−Kn ŝk , can be written as
isi
d̂n = cn−q dq + ηnd , (25)
q
192 Z.-Q. Luo

where cqisi = i ci ri−Kq is the equivalent channel from an inter-symbol interference


(ISI) perspective,

rm
k gk ḡk+m is the autocorrelation sequence of the filter gk , and
ηnd = k ḡk−Kn ηk is the effect of the noise on d̂n . A common design goal is to find a
pulse shaping filter {gk } which minimizes the spectral occupation of the communication
scheme subject to the constraint that the filters are self-orthogonal at translations of
integer multiples of K. The orthogonality constraint ensures that there is no inter-sym-
bol interference in a distortionless channel, and that the receiver filter neither amplifies
nor correlates the white noise component of the external interference. Given positive
integers K, L and spectral mask functions L(ejω ), U (ejω ), and assuming the channel
is distortionless, we wish to find a filter vector (possibly complex) {gk } of length L
satisfying
L−1
gk ḡk−K = δ(),  = 0, 1, . . . , (L − 1)/K, (26a)
k=K
L(ejω ) ≤ |G(ejω )| ≤ U (ejω ), for all 0 ≤ ω < 2π, (26b)

where δ(·) denotes the Kronecker Delta function, x denotes the greatest integer ≤ x.
Unfortunately, both these constraints are non-convex in the parameters gk . However, we
can parameterize the problem in terms of the autocorrelation rm , resulting in the linear
constraints

rK = δ , for  = 0, 1, . . . , (L − 1)/K, (27a)

L (e ) ≤ R(e ) ≤ U (e ),
2 jω jω 2 jω
for all 0 ≤ ω < 2π. (27b)

Notice that by Proposition 5 the constraint (27b) can be reformulated as a finite LMI,
so the overall design problem (27a)–(27b) can be cast as a LMI feasibility problem.

Example. We design a filter to compete with the filter specified for the synthesis of the
chip waveform in the IS95 standard [42]. The standard requires a filter with a ±1.5dB
ripple in the pass-band f ∈ [0, fp ], and 40dB attenuation in the stop-band f ∈ [fs , 1/2],
where fp = 590/(1228.8K) and fs = 740/(1228.8K). The filter chosen in the stan-
dard has linear phase, K = 4 and L = 48, and hence fp ≈ 0.12 and fs ≈ 0.15. Whilst
that filter satisfies the spectral mask, it does not satisfy the orthogonality constraints
(26a). Hence, the IS95 filter can induce substantial ‘inter-chip’ interference even when
the physical channel is benign. Therefore, we seek a minimal length filter such that
both the frequency response mask is satisfied and the filter is orthogonal. A globally
optimal solution to this problem was found using an SDP formulation coupled with a
binary search technique [24]. Each SDP was solved using SeDuMi [70] in about just
under 7 minutes on a 400 MHz Pentium II PC. The above procedure resulted in a length
L = 51, so orthogonality is achieved for the price of a mild increase in filter length (from
L = 48). The frequency response of the designed filter is shown in Fig. 5 (compare with
Fig. 3 of the nonorthogonal filter suggested by the IS95 standard).

The formulation in the above example (see (27)) was based on a nominally ideal chan-
nel model. In practice, robustness to channel perturbations will be required. Fortunately,
robustness to quite diverse class of perturbations can be enforced using a semidefinite
Applications of convex optimization in signal processing and digital communication 193

−10
|G(ej 2πf )|2 , dB

−20

−30

−40

−50

−60
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
f = ω
2π , cycles-per-sample
Fig. 5. Relative power spectra (in decibels) of the designed orthogonal filter (L = 51) with the spectral mask
from the IS95 standard

programming framework [22]. Moreover, for computational reason, it is more efficient

to deal with the dual formulation of the cone of autocorrelation sequences [4]. This
is due to the fact that the dual cone can be easily parameterized by O(n) parameters,
while the primal cone (cf. Proposition 3) requires O(n2 ) parameters. This results in a
substantial reduction in the arithmetic operation count (from O(n6 ) to O(n3 )) per each
interior point iteration.

3.4. Robust magnitude filter design

In this subsection, we study the problem of designing magnitude filters which are robust
to quantization errors. To motivate our robust model, let us consider the design of a
linear phase FIR (Finite Impulse Response) filter for digital signal processing. Here, for
a linear phase FIR filter h = (h1 ..., hn ) ∈ Rn (symmetric part ignored), the frequency
response is (essentially)
H (ejω ) = ejnω (h1 + h2 cos ω + · · · + hn cos (nω)) = ejnω (cos ω)T h,
where cos ω = (1, cos ω, · · · , cos (nω))T . The FIR filter usually must satisfy a given
spectral envelope constraint (typically specified by design requirements or industry stan-
dards, see Figure 3).
L(ejω ) ≤ (cos ω)T h ≤ U (ejω ), ∀ ω ∈ [0, π ] (28)
Finding a discrete h (say, 4-bit integer vector) satisfying (28) is NP-hard. Ignoring
discrete structure of h, we can find a h satisfying (28) in polynomial time [23] by exactly
reformulating (28) as a LMI system using Proposition 5. However, rounding such a solu-
tion to the nearest discrete h may degrade performance significantly. Our design strategy
is then to first discretize the frequency [0, π ], then find a solution robust to discretization
and rounding errors. This leads to the following notion of robustly feasible solution:
L(ejωi ) ≤ (cos ωi + i )T (h + h) ≤ U (ejωi ), for all i  ≤ , h ≤ δ,
(29)
where i accounts for discretization error, while h models the rounding errors.
194 Z.-Q. Luo

The above example motivates us to consider the following formulation of a robust

linear program:
minimize max (c + c)T (x + x)
x≤δ,c≤0
subject to (ai + ai )T (x + x) ≥ (bi + bi ), (30)
for all (ai , bi ) ≤ i , x ≤ δ, i = 1, 2, ..., m.
Here two types of perturbation are considered. First, the problem data ({ai }, {bi }, c)
might be affected by unpredictable perturbation (e.g., measurement error). Second, the
optimal solution x opt is subject to implementation errors caused by the finite precision
arithmetic of digital hardware. That is, we have x actual := x opt + x, where x acutal
is the actually implemented solution. To ensure x actual remains feasible and delivers a
performance comparable to that of x opt , we need to make sure that x opt is robust against
both types of perturbations. This is essentially the motivation of the above robust linear
programming model. Notice that the above model (30) is more general than the robust
LP formulation (8) proposed by Ben-Tal and Nemirovskii [8] in that the latter only
considers perturbation error in the data ({ai }, {bi }, c).
We now reformulate the robust linear program (30) as a semidefinite program. We
say the solution x is robustly feasible if, for all i = 1, 2, ..., m,
(ai + ai )T (x + x) ≥ (bi + bi ),
for all (ai , bi ) ≤ i , x ≤ δ, i = 1, 2, ..., m.
It can be shown [8] that x is robustly feasible if and only if

aiT (x + x) − bi − i x + x2 + 1 ≥ 0, ∀ x ≤ δ, i = 1, 2, ..., m. (31)
Constraint (31) can be formulated as
   
T (x + x) − b )I x + x
(a
 i i  i   0,
  T 1 ∀ x ≤ δ, i = 1, 2, ..., m.
i (x + x) 1 ai (x + x) − bi
T

(32)
Now the objective function can also be modelled by introducing an additional variable t
to be minimized, and at the same time set as a constraint t − (c + c)T (x + x) ≥ 0, for
all c ≤ 0 and x ≤ δ. Then the objective can be modelled by t − cT (x + x) ≥
0 x + x, for all x ≤ δ, which is equivalent to
 
(t − cT (x + x))I 0 (x + x)
 0, ∀ x ≤ δ. (33)
0 (x + x)T t − cT (x + x)
Using Proposition 2, we can show that (32) is equivalent to the following: there exists
a µi ≥ 0 such that
    
x I  
 (ai x − bi )I i 1 i 0 
T
0 0 0
     2   0.
 i x T 1 a T x − b i 1 a T  − µ i 0 δ 0 (34)
  i
1
2 i 0 0 −I
i I 0 2 ai 0
Applications of convex optimization in signal processing and digital communication 195

Similarly, (33) holds for all x ≤ δ if and only if there is a µ0 ≥ 0 such that
   
(t − cT x)I 0 x 0 I 0 0 0
 0 x T t − cT x − 21 cT  − µ0  0 δ 2 0   0. (35)
0 I − 21 c 0 0 0 −I

Therefore, the robust linear programming model becomes a semidefinite program.

Proposition 6. The robust linear program (30) can be reformulated as the following
SDP:

minimize t, subject to (34) and (35).

Figures 6 and 7 show a design example obtained using the robust formulation (29)
and the nonrobust LMI formulation of spectral mask constraint (28), respectively. In
both cases, the dotted lines denote the spectral mask constraints which the designed
filter must satisfy, the solid curves correspond to the magnitude responses of optimal
robust and non-robust filters designed from the formulation (29) with δ = 2−10 and

Magnitude Passband Magnitude Stopband Magnitude

1.4 1.08 0.1
optimal solution optimal solution optimal solution
1.2 10−bit quantization 10−bit quantization 10−bit quantization
1.06
worst case worst case worst case
0.08
1 1.04

0.8 1.02 0.06

1
0.6
0.04
0.98
0.4
0.96 0.02
0.2
0.94
0 0
0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Frequency (× π rad/sample) Normalized Frequency (× π rad/sample) Normalized Frequency (× π rad/sample)

(a) Magnitude responses (b) Detailed passband re- (c) Detailed stopband re-
sponses sponses
Fig. 6. Magnitude responses of robust FIR filter (n = 22) subject to different quantization errors

Magnitude Passband Magnitude Stopband Magnitude

1.4 1.08 0.1
optimal solution optimal solution optimal solution
1.2 10−bit quantization 10−bit quantization 10−bit quantization
1.06
worst case worst case worst case
0.08
1 1.04

0.8 1.02 0.06

1
0.6
0.04
0.98
0.4
0.96 0.02
0.2
0.94
0 0
0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Frequency (× π rad/sample) Normalized Frequency (× π rad/sample) Normalized Frequency (× π rad/sample)

(a) Magnitude responses (b) Detailed passband re- (c) Detailed stopband re-
sponses sponses
Fig. 7. Magnitude responses of nonrobust FIR filter (n = 19) subject to quantization errors
196 Z.-Q. Luo

0 respectively, the dash-dot and dashed curves correspond to the 10-bit truncation of
the optimal filters. To satisfy the spectral mask specification, the minimum filter length
is 22 for the robust filter and 19 for the nonrobust filter. We consider both the nearest
neighbor quantization and the farthest neighbor quantization (or worst case rounding).
As shown in Figs. 6 and 7, when the optimal coefficients are truncated to 10 bits, the
quantized versions of the robust filter still satisfy the spectral mask constraints, while
the resulting nonrobust filters violate both the passband and the stopband spectral mask
specifications.

4. Efficient optimization methods for digital communication

Conic and robust optimization techniques have also been successfully applied to some of
the fundamental system design and real time processing problems in the area of digital
communication. In this section, we describe two such applications arising from multi-
user communication and wireless Multi-Input Multi-Output (MIMO) communication.

4.1. The application of SDP to transceiver design for a multi-user communication

system

Consider a quasi-synchronous vector multiple access scheme with two users whose data
vectors s1 and s2 , are uncorrelated (see Figure 8). The channel matrices H1 and H2 ,
which are of size p × n, are assumed to be known, and n is a zero mean additive Gauss-
ian noise vector which is uncorrelated with s1 and s2 and has known correlation matrix
E(nnH ) = R. With square transmitter precoding matrices F1 and F2 , the received signal
takes the form

x = H1 F1 s1 + H2 F2 s2 + n. (36)

In our development, each data block si will be treated as white with identity correlation
matrix. From the received signal x, we wish to extract the transmitted signals si , i = 1, 2.
This can be accomplished in various ways. A popular approach is to use a linear receiver
Gi whereby the equalized signal Gi x is quantized according to the finite alphabet of si ,
with Gi , i = 1, 2, being the block (matrix) equalizers.
The above vector multiple access channel model arises naturally in the so-called
generalized multi-carrier block transmission scheme [84]. When the data blocks are
appropriately padded with cyclic prefix, the channel matrices Hi become circulant and
can be diagonalized by Fourier transforms, resulting the well known OFDM (Orthogonal
Frequency Division Multiplexing) scheme. We will develop an SDP formulation of the
MMSE transceiver design problem for general Hj and R as well as a SOCP formula-
tion for the diagonalized OFDM channels. We point out that similar models to that in
Fig. 8 have been considered in [94] where the capacity region for the above multi-access
communication channel is evaluated using the tool of semidefinite programming.
Applications of convex optimization in signal processing and digital communication 197

Transmitters Channel Matrices Receivers

s1
F1 H1 G1
Noise n

x
+ received
signal

s2
F2 H2 G2

Fig. 8. Two user multiple access scheme (uplink)

SDP formulation of MMSE Transceiver Design

For the system in Figure 8, let ei denote the error vector (before making the hard decision)
for user i, i = 1, 2. Then

e1 = G1 x − s1 = G1 (H1 F1 s1 + H2 F2 s2 + n) − s1
= (G1 H1 F1 − I ) s1 + G1 H2 F2 s2 + G1 n.

Assuming statistical independence of signals and noises, this further implies that

E(e1 e1H ) = (G1 H1 F1 − I ) (G1 H1 F1 − I )H + (G1 H2 F2 ) (G1 H2 F2 )H + G1 RGH1

(37)

where E(·) represents statistical expectation and we have used the fact that E(nnH ) = R
and E(si siH ) = I , i = 1, 2. Similarly, we have

E(e2 e2H ) = (G2 H2 F2 − I ) (G2 H2 F2 − I )H + (G2 H1 F1 ) (G2 H1 F1 )H + G2 RGH

2 .
(38)

As is always the case in practice, there are power constraints on the transmitting matrix
filters:

Tr F1 F1H ≤ p1 , Tr F2 F2H ≤ p2 (39)

where p1 > 0 and p2 > 0 are user-specified bounds on the transmitting power for each
user. Our goal is to design a set of transmitting matrix filters Fi satisfying the power
constraints (39) and a set of matrix equalizers Gi such that the total mean squared error
MSE = Tr(E(e1 e1H )) + Tr(E(e2 e2H )) is minimized. In other words, we aim to solve

minimizeF1 ,F2 ,G1 ,G2 Tr(E(e1 e1H )) + Tr(E(e2 e2H ))

(40)
subject to Tr(F1 F1H ) ≤ p1 , Tr(F2 F2H ) ≤ p2 ,

where Tr(E(e1 e1H )) and Tr(E(e2 e2H )) are given by (37) and (38) respectively. The
receiver filters G1 and G2 in (40) are unconstrained. The objective function of (40) is a
198 Z.-Q. Luo

fourth order polynomial in Gi , Fi , i = 1, 2. It can be easily checked (even for the case
where the block length n is one; i.e., each Gi , Fi is a scalar) that the Hessian matrix
of this fourth-order polynomial is not positive semidefinite. Therefore, the objective
function of (40) is nonconvex, and hence it can be difficult to minimize due to the usual
difficulties with local solutions and the selection of a stepsize and starting point. In what
follows, we will reformulate (40) as a convex semidefinite program.
As the first step, we can eliminate G1 and G2 in (40) by first minimizing the total
MSE with respect to G1 and G2 assuming F1 and F2 are fixed. The resulting receivers
are the so called linear Minimum Mean Squared Error (MMSE) receivers:

G1 = F1H H1H W, G2 = F2H H2H W (41)

with
−1
W = H1 F1 F1H H1H + H2 F2 F2H H2H + R , (42)

Substituting the MMSE equalizer (41) into (37) results in the following minimized (with
respect to Gi ) mean square error:

E(e1 e1H ) = −F1H H1H W H1 F1 + I, E(e2 e2H ) = −F2H H2H W H2 F2 + I. (43)

Substituting (43) into the total MSE and using the definition of W (42) gives rise to

MSE = Tr (W R) + n, (44)

Now let us further define two new matrix variables U1 = F1 F1H and U2 = F2 F2H . Then
the MMSE (44) can be expressed as

MSE = Tr (H1 U1 H1H + H2 U2 H2H + R)−1 R + n

and the power constraints (39) can be expressed as Tr(U1 ) ≤ p1 and Tr(U2 ) ≤ p2 .
Consequently, the optimal joint MMSE transmitter-receiver design problem can be stated
as
 
minimizeU1 ,U2 g(U1 , U2 ) := Tr (H1 U1 H1H + H2 U2 H2H + R)−1 R
subject to Tr(U1 ) ≤ p1 , Tr(U2 ) ≤ p2 , (45)
U1  0, U2  0.

Using the auxiliary matrix variable W (42) and the fact that R  0, we can rewrite (45)
in the following alternative (but equivalent) form:

minimizeW,U1 ,U2 Tr (W R)
subject to Tr(U1 ) ≤ p1 , Tr(U2 ) ≤ p2 ,
(46)
W  (H1 U1 H1H + H2 U2 H2H + R)−1
U1  0, U2  0.
Applications of convex optimization in signal processing and digital communication 199

Notice that the constraint

W  (H1 U1 H1H + H2 U2 H2H + R)−1

can be replaced, via Schur’s complement, by the following equivalent linear matrix
inequality
 
W I
 0. (47)
I H1 U1 H1H + H2 U2 H2H + R

Therefore, we obtain an equivalent Semidefinite Programming (SDP) formulation. This

SDP formulation makes it possible to efficiently solve the optimal transmitter design
problem using interior point methods [61, 76]. The advantage of the SDP formulation
over the formulation (40) is that the former is convex while the latter is not. The arith-
metic complexity of the interior point methods for solving the SDP is O(n6.5 log(1/)),
where  > 0 is the solution accuracy [61, 76]. Once the optimal U1 and U2 have been
determined, they can be factorized (e.g., Cholesky factorization) as U1 = F1 F1H and
U2 = F2 F2H to obtain optimal MMSE transmitter matrices F1 and F2 .

Diagonal designs

When the channel matrices H1 and H2 are diagonal (as in OFDM systems) and the noise
covariance matrix R is also diagonal, we can show (see Proposition 7 below or [52]) that
the optimal transmitters are also diagonal and can be computed more efficiently (faster
than solving the SDP described earlier).
Proposition 7. If the channel matrices H1 and H2 are diagonal and the noise covariance
matrix R is diagonal, then the optimal transmitters U1 and U2 are also diagonal. Con-
sequently, the MMSE transceivers for a multi-user OFDM system can be implemented
by optimally allocating power to each subcarrier for all the users.
Proposition 7 should be good news to practitioners since it says that in ‘diagonal’sce-
narios, there is no need to implement full precoder matrices because diagonal precoders
are optimal. Notice that diagonal precoders simply represent power loading/subcarrier
allocation at the transmitters. Therefore, Proposition 7 implies that the MMSE trans-
ceivers for a multi-user OFDM system can be implemented by optimally assigning
subcarriers and allocating power to them.
Another important implication of Proposition 7 is the significant simplification in
the computation of the optimal MMSE transceivers. In particular, Proposition 7 suggests
that we only need to search among all the diagonal transmitters in order to achieve the
minimum MSE. Therefore, if H1 , H2 and R are diagonal, it is only necessary to solve
(48) below rather than the SDP described in the previous section. Before we state this
formally, we point out that when the channel matrices Hj have been diagonalized using
the FFT (Fast Fourier Transform) and IFFT (Inverse FFT), the ith diagonal element is
Hj (i), where Hj (i) is the frequency response of user j ’s channel at the ith point on
the FFT grid, ωi = 2πi/n. Define the diagonal entries of U1 , U2 by u1 = diag (U1 ),
200 Z.-Q. Luo

u2 = diag (U2 ). Then using u1 , u2 as the new variables to be optimized, and letting
R = diag{ρi2 }, the reduced optimization problem becomes
n
−1
minimizeu1 ,u2 ρi2 |h1 (i)|2 u1 (i) + |h2 (i)|2 u2 (i) + ρi2
i=1
n n (48)
subject to u1 (i) ≤ p1 , u2 (i) ≤ p2 ,
i=1 i=1
u1 (i) ≥ 0, u2 (i) ≥ 0, i = 1, 2, ..., n.
Introducing an auxiliary vector w, we can transform (48) into the following (rotated)
second order cone program:
n
minimizew,u1 ,u2 ρi2 w(i)
i=1
n n
subject to u1 (i) ≤ p1 , u2 (i) ≤ p2 , (49)
i=1  i=1 
w(i) |h1 (i)|2 u1 (i) + |h2 (i)|2 u2 (i) + ρi2 ≥ 1,
u1 (i) ≥ 0, u2 (i) ≥ 0, i = 1, 2, ..., n.
There exist highly efficient (general purpose) interior point methods [61] to solve
the above second order cone program with total computational complexity of
O(n3.5 log(1/)), where  > 0 is the solution accuracy. This is a significant improve-
ment from the complexity of O(n6.5 log(1/)) if we solve the MMSE transceiver design
problem as an SDP.
Extensions to the sum-capacity formulation and to the MIMO channel as well as
broadcast channel models have been obtained in a number of recent papers [34, 52, 62].
Another related work is given in [94].

4.2. Efficient optimal/sub-optimal detection methods for space-time processing in

wireless communication
Recently the use of multiple antennas at both the transmitter and the receiver has been
shown to provide several-fold increase in capacity. In a rich scattering environment such
as indoor wireless scenario, the channel gains between any pair of transmitter antenna
and receiver antenna can be assumed independent. In this case, the capacity increase
is shown [71] to be proportional to the number of transmit antennas. To reap the ben-
efits brought by the MIMO (Multi-Input-Multi-Output) channel, one must develop an
efficient decoding algorithm used in such a system, especially for cases where channel
matrix is ill conditioned. The existing V-Blast type of receiver algorithms [32] developed
at Bell Labs suffers from significant performance loss when the channel matrix is ill
conditioned.
Central to the MIMO maximum likelihood channel detection and decoding is the
following constrained optimization problem
min y − H x2 (50)
x∈V m
Applications of convex optimization in signal processing and digital communication 201

where H ∈ C n×m denotes the (known) channel matrix, y ∈ C n is the received channel
output, x is the unknown (to be determined) transmitted information symbol vector from
the signal constellation set V m . Usually the constellation set V is either a finite lattice
set (QAM signalling) or a set of points uniformly distributed on the unit circle over the
complex plane (PSK signalling):

ej φi , with φi = 2π i/M, ∀ i = 1, . . . , M.

The current popular method to approximately solve (50) is with the Sphere Decoder
[30, 79]. This method enjoys a cubic complexity on average [39], but is known to have
poor performance in low SNR (signal to noise ratio) region.
Consider the case of PSK signalling (which is a popular choice in practice due to
its constant modulus and superior performance). The discrete nature of the set {φi }
makes problem (50) intractable. We can use a continuous relaxation of the set {φi } to
include all possible angles in [0, 2π]. In other words, we relax the constraint x ∈ V m to
x ∈ U m = { x | |xi | = 1, ∀ i = 1, . . . , m}, so problem (50) becomes:

min y − H x2 (51)

x∈U m

In the combinatorial optimization context (which roughly corresponds to the case of

Binary PSK modulation of M = 2), the above relaxation was first considered by Burer-
Monteiro-Zhang [14] as a low-rank semidefinite programming relaxation of a Boolean
quadratic maximization problem. In addition to applying this relaxation to the graph
partitioning problem, Burer-Monteiro-Zhang also analyzed the structure of the local

Performance comparison, PSK Decoder (PSKD) vs Sphere Decoder (SD)

0
10
PSKD, bpsk, size = 10
SD, bpsk, size = 10
−1
PSKD, bpsk, size = 30
10 SD, bpsk, size = 30
PSKD, qpsk, size = 10
SD, qpsk, size = 10
−2
10

−3
10
BER

−4
10

−5
10

−6
10

−7
10

−8
10
2 4 6 8 10 12 14 16 18
SNR, dB

Fig. 9. Bit error performance comparison of PSK Decoder and the Sphere Decoder for uncoded channel with
m = n = 10 and m = n = 30 for binary PSK modulation and m = n = 10 for 4-PSK modulation
202 Z.-Q. Luo

maxima of (51), and showed that for M = 2 each local minimizer of (51) is also a
global minimizer. For general M, we have the following result [53].
Proposition 8. Every local minimizer x̂ of (51) is a 21 -minimizer of (51) in the sense
that
1
(f (x̂) − fmin )/(fmax − fmin ) ≤ ,
2
where f (x) := y − H x2 , fmax and fmin represent the global maximum and minimum
value of f over U m .
Proposition 8 suggests that finding a local solution of (51) will provide a good approx-
imate solution for the original maximum likelihood detection problem (50). This is the
approach adopted first by [1, 56, 72, 73, 83] in the context of CDMA multiuser detec-
tion of binary symbols and further extended to the current MIMO channel model for
general PSK modulation by [53]. In particular, given the channel matrix H and the vec-
tor of received signals y, we need to minimize the relaxed objective function f (x) in
(51). While various options exist, we have used the coordinate descent method (similar
to [1]). The choice of a starting point x0 can be random or obtained as an approximate
solution to H x = y by running a fixed number of iterations of the conjugate gradient
method, Gauss-Jacobi or Gauss-Seidel iterative method. Since each iteration of these
methods have a complexity of O(k 2 ), where k = max{m, n}, the overall complexity of
the initialization process does not exceed O(k 2 ).
We have compared the performance of the new PSK decoder and the Sphere
Decoder [79] in terms of BER and computational complexity for different dimensions

Complexity comparison, PSK Decoder vs Sphere Decoder, QPSK

10
SNR = 5 dB
SNR = 10 dB
9 SNR = 13 dB

7
Flops (SD) / Flops (PSK)

0
0 5 10 15 20 25 30 35 40 45 50
Problem size, m = n

Fig. 10. Complexity comparison, ratio of the Sphere Decoder flops over the PSK Decoder flops for
SNR = 5, 10 and 13 dB, 4-PSK modulation
Applications of convex optimization in signal processing and digital communication 203

and SNR levels. Notice that PSK decoder works only for PSK constellations, while the
Sphere Decoder only for QAM signals. Therefore, the comparison of the two methods
is possible only for binary PSK and 4-PSK constellations since these are the cases that
can be handled by both decoders. Figure 9 shows the BER comparison of the Sphere
Decoder and the PSK Decoder for the case of equal number of transmitters and receiv-
ers m = n = 10 and m = n = 30 with binary PSK signalling, and for the case of
m = n = 10 for 4-PSK signalling. Both decoders exhibit similar performance in terms
of BER. However, the two decoders are significantly different in their complexity. The
running time of both decoders depends on the problem size and the type of constella-
tion. For example, Figure 10 shows the ratio of floating-point operations for the Sphere
Decoder over the new PSK Decoder as a function of m (we set m = n). Three different
SNRs (= 5, 10 and 13 dB) are considered and the modulation is 4-PSK. For SNR=10
dB, Sphere Decoder performs faster for problem sizes up to m = n = 24, after which the
the new PSK decoder becomes substantially faster. For SNR = 5 dB and with 4-PSK
modulation, the new PSK Decoder becomes faster for dimensions m = n = 12 and
above. The efficiency of our PSK decoder can also be seen from CPU time: decoding
100 4-PSK symbols takes 0.29 sec on a 1.5 GHz Pentium IV PC.

5. Concluding remarks

The recent development and application of modern optimization techniques has made
it possible to solve many of the core problems in signal processing and digital commu-
nication efficiently. This is remarkable because not long ago many of these problems
were still considered difficult or intractable by signal processing and communication
engineers. Encouraged by these successes, some of the mainstream signal processing
and communications conferences have started to sponsor special sessions and tutorials
to feature the role of optimization and its application potential. This is good news for
the mathematical programming community because it not only brings recognition to
our field but also will provide new impetus for the continued innovation in algorithm
design and numerical software development in mathematical programming. Moreover,
the interplay between these fields will ultimately benefit both sides: the modern opti-
mization techniques and related software can help solve some well known difficult
engineering problems efficiently, and at the same time, the process of formulating and
solving practical engineering problems will also inspire optimizers to refine and extend
the current theory and algorithms. The latter point is best represented by the robust beam-
forming example and the robust magnitude filter design example in Section 3, where the
existing robust convex optimization techniques were found to be inadequate and had be
generalized.
This paper presented a few examples where the use of conic and robust optimization
techniques played an essential role in their formulation and numerical solution. Needless
to say, the selected examples only reflect author’s limited experience in this burgeon-
ing research area, and they by no means represent all the currently known engineering
applications of optimization. Indeed, the list of important applications of convex opti-
mization (e.g., linear/quadratic programming, SOCP/SDP and robust optimization) in
electrical engineering is long and fast growing. They include blind channel equaliza-
204 Z.-Q. Luo

tion [26, 54, 57], statistical estimation and moment problems [4, 12, 27, 31, 69], VLSI
layout and circuit design [14, 18, 77], designing optimal communication systems (e.g.,
minimum bit error rate) [25, 44, 45, 82, 89, 94], and robust Kalman filtering [33, 47].
Another fascinating application of convex optimization is in multi-user information the-
ory where SDP duality theory and interior point algorithms have been used to establish
the dual correspondence of the capacity region of a multi-access channel and the so called
“dirty paper” region of a vector broadcast channel [43,78,93]. Undoubtedly, much more
remains to be done to harness the power of convex optimization in the field of signal
processing and communication.

Acknowledgements. The author wishes to thank his collaborators who have helped make many of the results
presented in this paper possible. These include Kaywan Afkhamie, Robert Cui, Tim Davidson, Zhi Ding,
Minyue Fu, Alex Gershman, Georgios Giannakis, Nick Kisialiou, Lingjie Li, Jing Liu, Boris Maricic, Mei
Meng, Sharahm Shahbazpanahi, Jos Sturm, Sergiy Vorobyov, Max Wong and Shuzhong Zhang.

References

[1] Abdi, M., El Nahas, H., Jard, A., Moulines, E.: Semidefinite Positive Relaxation of the Maximum-Like-
lihood Criterion Applied to Multiuser Detection in a CDMA Context. IEEE Signal Process. Lett. 9,
165–167 (2002)
[2] Afkhamie, K., Luo, Z.-Q., Wong, K.M.: Adaptive Linear Filtering Using Interior Point Optimization
Techniques. IEEE Transactions on Signal Process. 48(6), 1637–1648 (2000)
[3] Afkhamie, K.H., Luo, Z.-Q., Wong, K.M.: MMSE Decision-Feedback Equalization with Short Training
Sequences: An Application of Interior Point Least Squares. IEEE Transactions on Signal Process. 49,
1543–1555 (2001)
[4] Alkire, B., Vandenberghe, L.: Handling Nonnegative Constraints In Spectral Estimation. Proceedings of
the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers 1, 202–206 (2000)
[5] Alkire, B., Vandenberghe, L.: Interior-Point Methods For Magnitude Filter Design. Proceedings of 2001
IEEE International Conference on Acoustics, Speech, and Signal Processing 6, 3821–3824 (2001)
[6] Anderson, B.D.O., Hitz, K.L., Diem, N.D.: Recursive Algorithm for Spectral Factorization. IEEE Trans-
actions Circuits Syst. CAS–21, 742–750 (1974)
[7] Ben-Tal, A., El Ghaoui, L., Nemirovskii, A.: Robust Semidefinite Programming. In: Handbook of Semi-
definite Programming, edited by Wolkowicz, H., Saigal, R. and Vandenberghe, L., Kluwer Academic
Publishers, March 2000
[8] Ben-Tal, A., Nemirovskii, A.: Robust Convex Optimization. Math. Operations Res. 23, 769–805 (1998)
[9] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms and Engi-
neering Applications. SIAM-MPS Series on Optimization, SIAM publications, Philadelphia, 2001
[10] Ben-Tal, A., Nemirovski, A.: Robust Optimization Methodology and Applications. Math. Programming,
Ser. B 92, 453–480 (2002)
[11] Ben-Tal, A., Nemirovskii, A.: Structural Design. In: Handbook of Semindefinite Programming, The-
ory, Algorithms, and Applications, edited by Wolkowicz, H., Saigal, R. and Vandenberghe, L., Kluwer
Academic Press, 2000
[12] Bertsimas, D., Sethuraman, J.: Moment Problems and Semidefinite Optimization. In: Handbook of Sem-
indefinite Programming, Theory, Algorithms, and Applications, edited by Wolkowicz, H., Saigal, R. and
Vandenberghe, L., Kluwer Academic Press, 2000
[13] Boyd, S., El Ghaoui, L., Feron, E., Balakrishnan, V.: Linear Matrix Inequalities in System and Control
Theory. SIAM Studies in Applied Mathematics, SIAM, Philadelphia, 1994
[14] Burer, S., Monteiro, R.D.C., Zhang,Y.: Rank-Two Relaxation Heuristics For Max-Cut And Other Binary
Quadratic Programs. SIAM J. Optim. 12, 503–521 (2001)
[15] Cantoni, A., Lin, X.G., Teo, K.L.: A New Approach to the Optimization of Robust Antenna Array
Processors. IEEE Transactions on Antennas and Propagation 41(4), 403–411 (1993)
[16] Chang, L., Yeh, C.C.: Performance of DMI and Eigenspace-based Beamformers. IEEE Transactions
Antennas Propagation 40, 1336–1347 (1992)
[17] Chen, X., Parks, T.W.: Design of Optimal Minimum Phase FIR Filters by Direct Factorization. Signal
Process. 10, 369–383 (1986)
Applications of convex optimization in signal processing and digital communication 205

[18] Choi, C., Ye, Y.: Application of Semidefinite Programming to Circuit Partitioning. In: Pardalos, P. eds.,
Approximation and Complexity in Numerical Optimization, pp. 130–136, KluwerAcademics Publishers,
2000
[19] Cox, H., Zeskind, R.M., Owen, M.H.: Robust Adaptive Beamforming. IEEE Transactions on Acoustic
Speech Signal Processing 35, 1365–1376 (1987)
[20] Cui, S., Maricic, B., Luo, Z.-Q., Ding, Z.: Robust Blind Multiuser Detection Against Signature Waveform
Mismatch. Submitted to IEEE Transactions on Wireless Communication
[21] Dam, H.H., Teo, K.L., Nordebo, S., Cantoni, A.: The Dual Parameterization Approach to Optimal Least
Square FIR Filter Design Subject to Maximum Error Constraints. IEEE Transactions Signal Process.
48(8), 2314–2320 (2000)
[22] Davidson, T.N.: Efficient Design of Waveforms for Robust Pulse Amplitude Modulation. IEEE Trans-
actions on Signal Process. 4(12), 3098–3111 (2001)
[23] Davidson, T.N., Luo, Z.-Q., Sturm, J.: Linear Matrix Inequality Formulation of Spectral Mask Con-
straints. IEEE Transactions on Signal Process. 50, 2702–2715 (2002)
[24] Davidson, T.N., Luo, Z.-Q., Wong, K.M.: Spectrally-Efficient Orthogonal Pulse Shape Design via Semi-
definite Programming. IEEE Transactions on Signal Process. 48(5), 1433–1445 (2000)
[25] Ding, Y.-W., Davidson, T.N., Luo, Z.-Q., Wong, K.M.: Minimum BER Block Precoders for Zero-forcing
Equalization. Accepted for publication in IEEE Transactions on Signal Process.
[26] Ding, Z., Luo, Z.-Q.: A Fast Linear Programming Algorithm for Blind Equalization. IEEE Transactions
on Commun. 48, 1432–1436 (2000)
[27] Dumitrescu, B., Tabus, I., Stoica, P.: On the Parameterization of Positive Real Sequences and MA
Parameter Estimation. IEEE Transactions on Signal Process. 49(11), 2630–2639 (2001)
[28] El Ghaoui, L., Lebret, H.: Robust Solutions To Least-Squares Problems With Uncertain Data. SIAM J.
Matrix Anal. Appl. 1997
[29] El Ghaoui, L., Oustry, F., Lebret, H.: Robust Solutions to Uncertain Semidefinite Programs. SIAM J.
Optim. 9, 1 (1998)
[30] Fincke, U., Pohst, M.: Improved Methods for Calculating Vectors of Short Length in a Lattice. Including
a Complexity Analysis. Math. Comput. 44, 463–471 (1985)
[31] Fedorov. V., Lee, J.: Design of Experiments in Statistics. In: Handbook of Semindefinite Programming,
Theory,Algorithms, andApplications, edited by Wolkowicz, H., Saigal, R. and Vandenberghe, L., Kluwer
Academic Press, 2000
[32] Foschini, G.J.: Layered Space-Time Architecture for Wireless Communications in a Fading Environment
When Using Multi-Element Antennas. Bell Labs. Tech. J. 41–59 (1996)
[33] Fu, M., Souza, C., Luo, Z.-Q.: Finite Horizon Robust Kalman Filter Design. IEEE Transactions on Signal
Process. 49, 2103–2112 (2001)
[34] Gohary, R.H., Davidson, T.N., Luo, Z.-Q.: On Efficient Design of Vector Broadcast Communications
Systems, Manuscript, Department of Electrical and Computer Engineering, McMaster University, Ham-
ilton, Ontario L8S 4K1, 2002
[35] Goemans, M.X., Rendl, R.: Combinatorial Optimization. In: Handbook of Semindefinite Programming,
Theory,Algorithms, andApplications, edited by Wolkowicz, H., Saigal, R. and Vandenberghe, L., Kluwer
Academic Press, 2000
[36] Goemans, M.X., Williamson, D.P.: Improved Approximation Algorithms for Maximum Cut and Satisfi-
ability Problem Using Semi-definite Programming. J. Association Comput. Machinery 42, 1115–1145
(1995)
[37] Goldfarb, D., Iyengar, G.: Robust Convex Quadratic Constrained Programs. Manuscript, Department of
Industrial Engineering and Operations Research, Columbia University, July, 2002
[38] Hassibi, B., Hochwald, B.M.: Cayley Differential Unitary Space-Time Codes. IEEE Transactions on
Inf. Theor. 48(6), 1485–1503 (2002)
[39] Hassibi, B., Vikalo, H.: On the Expected Complexity of Sphere Decoding. Proceedings of the Thirty-Fifth
Asilomar Conference on Signals, Systems and Computers 2, 1051–1055 (2001)
[40] Haykin, S.: Adaptive Filter Theory. Third Edition, Prentice Hall, New Jersey, 1998
[41] Herrmann, O., Schuessler, W.: Design of Nonrecursive Digital Filters with Minimum Phase. Electronics
Lett. 6(11), 329–330 (1970)
[42] Proposed EIA/TIA Interim Standard. Wideband Spread Spectrum Digital Cellular System Dual-Mode
Mobile Station-Base Station Compatibility Standard. Technical Report TR45.5, QUALCOMM, Inc.,
San Diego, CA, USA, Apr. 1992
[43] Jindal, N., Vishwanath, S., Goldsmith, A.: On the Duality of Gaussian Multiple-Access and Broadcast
Channels. Submitted to IEEE Transactions on Inf. Theor. August 2002
[44] Jindal, N., Goldsmith, A.: Capacity and Optimal Power Allocation for Fading Broadcast Channels with
Minimum Rates. Submitted to IEEE Transactions on Inf. Theor. Sept. 2001
206 Z.-Q. Luo

[45] Kosut, R., Chung, W., Johnson, C., Boyd, S.: On Achieving Reduced Error Propagation Sensitivity
In DFE Design Via Convex Optimization. Proceedings of the 39th IEEE Conference on Decision and
Control 5, 4320–4323 (2000)
[46] Lebret, H., Boyd, S.: Antenna Array Pattern Synthesis via Convex Optimization. IEEE Transactions on
Signal Process. 45(3), 526–532 (1997)
[47] Li, L., Luo, Z.-Q., Davidson, T.N., Wong, K.M., Bossé, E.: Robust Filtering via Semidefinite Program-
ming with Applications to Target Tracking. SIAM J. Optim. 12, 740–755 (2002)
[48] Liu, J., Gershman, A., Luo, Z.-Q.: Adaptive Beamforming with Sidelobe Control: A Second Order Cone
Programming Approach. Accepted for publication in IEEE Signal Processing Letters
[49] Liu, Y., Luo, Z.-Q.: Design of Robust IIR Magnitude Filters via Semidefinite Programming, to appear
in Proceedings of ICASSP 2003, Hong Kong
[50] Lobo, M., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of Second-Order Cone Programming.
Linear Algebra and its Appl. 284, 193–228 (1998)
[51] Lorenz, R., Boyd, S.: Robust Minimum Variance Beamforming. Submitted to IEEE Transactions on
Signal Process.
[52] Luo, Z.-Q., Davidson, T.N., Giannakis, G.B., Wong, K.M.: Transceiver Optimization for Multiple Access
through ISI Channels. Accepted for publication in IEEE Transactions on Signal Process.
[53] Luo, Z.-Q., Luo, X.-D., Kisialiou, M.: An Efficient Quasi-Maximum Likelihood Decoder for PSK Sig-
nals, to appear in Proceedings of ICASSP’2003, Hong Kong
[54] Luo, Z.-Q., Meng, M., Wong, K.M., Zhang, J.-K.: A Fractionally Spaced Blind equalizer Based on
Linear Programming. IEEE Transactions on Signal Process. 50, 1650–1660 (2002)
[55] Luo, Z.-Q., Sturm, J., Zhang, S.: Multivariate Nonnegative Quadratic Mappings, Manuscript, Department
of Electrical and Computer Engineering, McMaster University, 2002
[56] Ma, W.-K., Davidson, T.N., Wong, K.M., Luo, Z.-Q., Ching, P.-C.: Quasi-maximum-likelihood Mul-
tiuser Detection Using Semi-definite Relaxation. IEEE Transactions on Signal Process. 50, 912–922
(2002)
[57] Maricic, B., Luo, Z.-Q., Davidson, T.N.: Blind Constant Modulus Equalization via Convex Optimization,
IEEE Transactions on Signal Process. 51(3), 805–818 (2003)
[58] Mittelmann, H.D.: An Independent Benchmarking of SDP and SOCP Solvers, preprint, Department of
Mathematics, Arizona State University, 2001
[59] Monzingo, R.A., Miller, T.W.: Introduction to Adaptive Arrays, Wiley, NY, 1980
[60] Nesterov, Y.: Squared Functional Systems And Optimization Problems. In: High Performance Optimi-
zation (H. Frenk et al., eds.), Kluwer Academic Publishers 405–440 (2000)
[61] Nesterov,Y., Nemirovskii,A.: Interior Point Polynomial Methods in Convex Programming SIAM Studies
in Applied Mathematics 13, Philadelphia, PA, 1994
[62] Ohno, S., Giannakis, G., Luo, Z.-Q.: Multi-Carrier Multiple Access is Sum-Rate Optimal for Block
Transmissions over Circulant ISI Channels. Submitted to IEEE Transactions on Information Theory
[63] Parks, T.W., Burrus, C.S.: Digital Filter Design. J. Wiley & Sons, New York, 1987
[64] Pesavento, M., Gershman, A., Luo, Z.-Q.: Robust Array Interpolation Using Second-Order Cone Pro-
gramming. IEEE Signal Process. Lett. 9, 8–11 (2002)
[65] Reed, I.S., Mallett, J.D., Brennan, L.E.: Rapid Convergence Rate in Adaptive Arrays. IEEE Transactions
Aerospace and Electron. Syst. 10, 853–863 (1974)
[66] Selesnick, I.W., Lang, M., Burrus, C.S.: Constrained Least Square Design for FIR Filters Without Spec-
ified Transition Bands. IEEE Transactions Signal Process. 44(8), 1879–1892 (1996)
[67] Shahbazpanahi, S., Gershman, A., Luo, Z.-Q., Wong, K.M.: Robust Adaptive Beamforming for Gen-
eral-rank Signal Models, Submitted to IEEE Transactions on Signal Process.
[68] Steiglitz, K., Parks, T.W., Kaiser, J.F.: METEOR: A Constraint-based FIR Filter Design Program. IEEE
Transactions Signal Process. 40(8), 1901–1909 (1992)
[69] Stoica, P., McKelvey, T., Mari, J.: MA Estimation In Polynomial Time. IEEE Transactions on Signal
Process. 48(7), 1999–2012 (2000)
[70] Sturm, J.F.: Using SeDuMi 1.02, A Matlab Toolbox For Optimization Over Symmetric Cones. Op-
tim. Meth. Software 11–12, 625–653 (1999). See http://fewcal.kub.nl/sturm/soft-
ware/sedumi.html for updates
[71] Telatar, I.E.: Capacity of Multi-Antenna Gaussian Channels. European Transactions on Telecommuni-
cation 10, 585–595 (1999)
[72] Tan, P., Rasmussen, L.: The Application of Semidefinite Programming for Detection in CDMA. IEEE
J. Selected Areas Commun. 19, 1142–1149 (2001)
[73] Tan, P., Rasmussen, L., Lim, T.: Constrained Maximum Likelihood Detection in CDMA. IEEE Trans-
actions Commun. 49, 142–153 (2001)
[74] Toh, K.C., Todd, M.J., Tutuncu, R.H.: SDPT3 – a Matlab Software Packgage for Semidefinite Program-
ming, version 3.0, Technical Report, National University of Singapore, June 2001
Applications of convex optimization in signal processing and digital communication 207

[75] Tuqan, J., Vaidyanathan, P.P.: Globally Optimal Two Channel FIR Orthonormal Filter Banks Adapted to
the Input Signal Statistics. IEEE Transactions Signal Process. 48, 2822–2833 (2000)
[76] Vandenberghe, L., Boyd, S.: Semidefinite Programming. SIAM Rev. 31, 49–95 (1996)
[77] Vandenberghe, Boyd, S., El Gamal, A.: Optimizing Dominant Time Constant in RC Circuits IEEE
Transactions on Computer-Aided Design of Integrated Circuits and Systems 17, 110–125 (1998)
[78] Vishwanath, S., Jindal, N., Goldsmith,A.: Duality,Achievable Rates, and Sum-Rate Capacity of Gaussian
MIMO Broadcast Channels. Submitted to IEEE Transactions on Information Theory, August, 2002
[79] Viterbo, E., Bouros, J.: A Universal Lattice Code Decoder For Fading Channels. IEEE Transactions Inf.
Theor. 45(5), 1639–1642 (1999)
[80] Vorobyov, S., Gershman, A., Luo, Z.-Q.: Robust Adaptive Beamforming Using Worst-Case Performance
Optimization: A Solution to the Signal Mismatch Problem. IEEE Transactions on Signal Process. 51(2),
33–323 (2003)
[81] Vorobyov, S., Gershman, A., Luo, Z.-Q., Ma, N.: Adaptive Beamforming with Joint Robustness Against
Mismatched Steering Vector and Inteference Nonstationarity. Accepted for publication in IEEE Signal
Processing Letters
[82] Wang, X., Lu, W.S., Antoniou, A.: Constrained Minimum-BER Multiuser Detection. IEEE Transactions
on Signal Process. 48(10), 2903–2909 (2000)
[83] Wang, X., Lu, W.S., Antoniou, A.: A Near-optimal Multiuser Detector for CDMA Channels Using
Semidefinte Programming Relaxation. Proceedings of 2001 IEEE Intenational Sympsium on Circuits
and Systems 4, 298–301 (2001)
[84] Wang, Z., Giannakis, G.: Wireless Multicarrier Communications – Where Fourier Meets Shannon. IEEE
Signal Process. Mag. 29–48 (2000)
[85] Wolkowicz, H., Saigal, R., Vandenberghe, L.: Handbook of Semidefinite Programming: Theory, Algo-
rithms and Applications, Kluwer Academic Press, pp. 163–188, 1999
[86] Wu, S.-P., Boyd, S.: SDPSOL: A Parser/Solver For Semidefinite Programs with Matrix Structure. in:
Recent Advances in LMI Methods for Control, L. El Ghaoui and S.-I. Niculescu, editors, SIAM, chapter
4, pp. 79–91, 2000
[87] Wu, S.-P., Boyd, S., Vandenberghe, L.: FIR Filter Design via Semidefinite Programming and Spectral
Factorization. Proceedings of the 35th IEEE Conference on Decision and Control 1, 271–276 (1996)
[88] Wu, S.-P., Boyd, S., Vandenberghe, L.: FIR Filter Design via Spectral Factorization and Convex Opti-
mization. Applied and Computational Control, Signals, and Circuits, B. Datta, Ed., Vol. 1. Birkhauser,
Boston, May 1999
[89] Xiao, L., Johansson, M., Hindi, H., Boyd, S., Goldsmith, A.: Joint Optimization of Communication
Rates and Linear Systems, Proceedings 2001 Conference on Decision and Control, December 2001
[90] Yakubovich, V.A.: S-Procedure In Nonlinear Control Theory. Vestnik Leningrad University 4, 73–93
(1977) English translation; original Russian publication in Vestnik Leningradskogo Universiteta, Seriya
Matematika 62–77 (1971)
[91] Ye, Y.: A .699 Approximation Algorithm for Max-Bisection. Math. Programming 90(1), 101–111 (2001)
[92] Ye, Y.: Interior-Point Algorithm: Theory and Analysis, John Wiley and Sons, New York, 1997
[93] Yu, W., Cioffi, J.: Sum Capacity of Gaussian Vector Broadcast Channel. Submitted IEEE Transactions
Inf. Theor., Nov. 2001
[94] Yu, W., Rhee, W., Cioffi, J.: Multiuser Transmitter Optimization for Vector Multiple Access Channels,
manuscript, STAR laboratory, Department of Electrical Engineering, Stanford University, CA 94305–
9515, 2002