Adaptive LMS-Type Filter for Cyclostationary
Signals - Full Version
Nir Shlezinger, Koby Todros, and Ron Dabora
Department of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel
Email: nirshl@post.bgu.ac.il, {todros, ron}@ee.bgu.ac.il
Abstract—Adaptive filters are employed in many signal processing and communications systems. Commonly, the design and
analysis of adaptive algorithms, such as the least mean-squares
(LMS) algorithm, is based on the assumptions that the signals are
wide-sense stationary (WSS). However, in many cases, including,
for example, interference-limited wireless communications and
power line communications, the considered signals are jointly
cyclostationary. In this paper we propose a new LMS-type
algorithm for adaptive filtering of jointly cyclostationary signals
using the time-averaged mean-squared error objective. When
the considered signals are jointly WSS, the proposed algorithm
specializes to the standard LMS algorithm. We characterize
the performance of the algorithm without assuming specific
distributions on the considered signals, and derive conditions for
convergence. We then evaluate the performance of the proposed
algorithm, called time-averaged LMS, in a simulation study
of practical channel estimation scenarios. The results show a
very good agreement between the theoretical and empirical
performance measures.
I. I NTRODUCTION
Adaptive filters play an important role in the implementation
of linear estimators. A commonly employed class of adaptation
algorithms is the class of least mean-squares (LMS)-type
algorithms [1, Ch. 9], [2, Ch. 10]. The majority of LMStype filters are designed for linear estimation of a widesense stationary (WSS) signal of interest (SOI), based on
a statistically dependent jointly WSS (JWSS) input signal.
In many practical scenarios, the considered signals are nonstationary, but rather belong to the class of wide-sense cyclostationary (WSCS) signals [3, Sec. 5-7]. For example,
most digitally modulated communications signals are WSCS
[4, Ch. 1]. Cyclostationarity is also observed in power line
communications (PLC), especially in the narrowband (NB)
frequency range (0 − 500 kHz) [5].
The work [6] studied the convergence of the LMS algorithm
for linear estimation of jointly wide-sense cyclostationary
(JWSCS) signals when no specific model relating the input
signal and the SOI is assumed. It was shown that the filter
coefficients are mean convergent when the step-size of the
algorithm approaches zero. The work [7] applied the LMS
algorithm to the identification of a linear time-invariant (LTI)
system with WSCS Gaussian input, and an output that is
corrupted by additive WSS Gaussian noise. For this scenario,
the transient, tracking, and the steady-state performance of
the LMS algorithm were studied, and convergence conditions
were derived. However, in many practical scenarios involving
WSCS signals, e.g., interference-limited communications and
NB-PLC, the relationship between the considered signals
cannot be modeled using a single LTI system with WSS noise.
This work was supported by the Ministry of Economy of Israel through the
Israeli Smart Grid Consortium.
The optimal linear estimator in the minimum mean-squared
error (MSE) sense of a WSCS SOI based on a JWSCS input
signal is the cyclic Wiener filter (CWF) [8]. The CWF achieves
the minimal MSE (MMSE) at any time instant, and is realized
as a linear periodically time-varying (LPTV) filter. A common
structure for realizing LPTV filters is the frequency shift
(FRESH) filter [8], which implements an LPTV system by
summing the outputs of multiple LTI filters, each applied
to frequency-shifted versions of the input signal. Adaptive
FRESH filters were considered in several works, e.g., [9], [10],
in which adaptive algorithms originally designed for JWSS
signals, such as the standard LMS algorithm, were applied
to the adaptation of FRESH filters. Any LPTV filter can be
implemented using a set of LTI filters [11], with possibly
infinite impulse responses. However, in practical designs,
linear estimators consist of a limited number of LTI filters.
Thus, in practice, when the number of LTI filters used for
implementing an LPTV system is smaller than the one required
for implementing the CWF, the resulting linear estimator does
not achieve the MMSE. In such cases, the time-averaged MSE
(TA-MSE) criterion is used instead of the instantaneous MSE
[8]–[10] in order to achieve a filter with static coefficients.
This motivates the development of an LMS-type algorithm for
adaptive filtering for JWSCS signals based on the minimum
TA-MSE (MTA-MSE) criterion.
Main Contributions: In this work we develop a new LMStype algorithm for adaptive linear estimation of discrete-time
(DT) signals, where the SOI and input signal are assumed to
be jointly proper-complex (JPC) [12] and JWSCS [4, Ch. 1].
Our setup accommodates various filter structures for JWSCS
signals, including LTI and LPTV filters.
We first observe that a linear MMSE (LMMSE) estimator
with time-invariant coefficients does not necessarily exist for
JWSCS signals. It follows that the TA-MSE objective is
typically preferable over the MSE since it leads to linear
estimators with time-invariant coefficients. We then derive an
adaptive algorithm based on minimizing the TA-MSE objective via stochastic approximation of the steepest descent (SD)
algorithm. The new algorithm, referred to as the time-averaged
LMS (TA-LMS), specializes to the standard LMS algorithm
when the considered signals are JWSS. Transient and steadystate performance analysis of the proposed algorithm is carried
out without imposing a specific distribution on the considered
signals. Finally, the proposed TA-LMS algorithm is applied to
a practical channel estimation problem in a simulation example
that show a very good agreement between the theoretical and
empirical performance measures.
The rest of this paper is organized as follows: In Section
II the considered estimation problem is stated. In Section
III the new TA-LMS algorithm is derived, and in Section
IV the performance analysis is carried out. In Section V
the proposed algorithm is illustrated in simulation examples.
Lastly, in Section VI, concluding remarks are provided.
filter [11], the setup (1) can realize any scalar LPTV filter of
finite dimensions.
II. P RELIMINARIES AND P ROBLEM F ORMULATION
Notations: We denote column vectors with lower-case
boldface letters, e.g., x; the k-th element (k ≥ 0) of a
vector x is denoted with (x)k . Matrices are denoted with
upper-case boldface letter, e.g., X. We denote the n × n
identity matrix with In and the all-zero n × m matrix with
0n×m . Hermitian transpose, transpose, complex conjugate, and
stochastic expectation are denoted by (·)H , (·)T , (·)∗ , and
E{·}, respectively. Let Re {x} denote the real part of the
complex number x, Tr {·} denote the trace operator, ((n))m
denote the remainder of n when divided by m, ⊗ denote
to Kronecker product, and h·iN denote time-averaging over
NP
−1
x [n − k]. The sets
N > 0 samples, e.g., hx[n]iN , N1
In this section we derive an adaptive linear estimator for
WSCS signals based on the TA-MSE objective. This objective
is motivated by the fact that when the estimator is restricted to
be in the class of linear estimators (1), an LMMSE estimator
may not exist for JWSCS signals [8]. The reason is that an
LMMSE estimator in the class of (1) exists if and only if there
exists a time-invariant vector hM M SE that satisfies
E x [n] xH [n] hM M SE = E {x [n] d∗ [n]} ,
(2)
k=0
III. A DAPTIVE L INEAR E STIMATION OF
C YCLOSTATIONARY S IGNALS BASED ON TA-MSE
∀n ∈ {0, 1, . . . , N0 − 1} , N0 . One scenario in which a
time-invariant LMMSE estimator does exist, is the scenario
considered in [7], in which x[n] is the input of an LTI system,
and d[n] is the output of the system corrupted by a WSS noise.
MTA-MSE Linear Estimator:
Define the deterministic
H
quantities:
C
[n]
,
E
x
[n]
x
[n]
x
D n
D , cx [n] , vecE(Cx [n]),
oE
2
, and
cd , E |d [n]|
, cxd , E {x [n] d∗ [n]}
N0
N0 ˜
˜
Cx , hCx [n]iN0 . Note that cd , cxd , and Cx are time-invariant
˜ ˜ while C
˜ [n] and c [n] are
˜
since
d[n] and x[n] are JWSCS,
x
x
periodic with period N0 . We assume that the following holds:
of integers and non-negative integers are denoted by Z and
N, respectively. For an n × n matrix X, let λmax (X) denote
the largest real eigenvalue of X, given that such exists, and
x = vec (X) denote the n2 × 1 column vector obtained
by stacking the columns of X one below the other. The
matrix X is recovered from its vectorized representation x via
X = vec−1 (x). For an n×1 vector y and an n2 ×1 vector x, AS1 C is non-singular.
x
2
2
let kyk denote the squared Euclidean norm yH y, and kykx
˜
The TA-MSE associated with the linear estimator (1) is
denote the weighted squared Euclidean norm yHvec−1 (x) y.
D n
oE
b
2
Lastly, for a sequence of n×n matrices {Xk }k=a , b ≥ a are
H
d
[n]−h
x[n]
J
(h)
,
E
b
Q
N0
Xk denote the matrix product Xb Xb−1 · · · Xa .
integers, let
H
= cd − 2Re cH
(3)
k=a
xd h + h Cx h.
˜
˜
˜
Cyclostationary Stochastic Processes: A DT properTherefore, the optimal filter coefficients vector in the sense of
complex (PC) multivariate process x[n] is said to be WSCS if
minimum TA-MSE, denoted by hopt , must satisfy the timeboth its mean value and autocorrelation
function
are periodic
averaged Wiener-Hopf equations (see, e.g., [9, Eq. (5)]):
with
some period, N0 , i.e., E x[n]} = E x[n + N0 ]}, and
Cx hopt = cxd ,
(4)
E x[n+l]xH[n]} = E x[n + N0 + l]xH[n + N0 ]}, see [3, Sec.
˜
˜
3.5]. A pair of JPC DT processes x1 [n] and x2 [n] are said to be resulting in the linear MTA-MSE (LMTA-MSE) estimator
JWSCS with period N0 if each process isWSCS with period dˆopt [n] , hH
opt x[n]. The following lemma states an equivaN0 and the cross-correlation function E x1 [n + l]xH
[n]}
is
lence
between
the LMMSE and LMTA-MSE estimators.
2
periodic with period N0 w.r.t. n [3, Sec. 3.6.2].
Lemma 1. If a time-invariant LMMSE estimator exists, then
Problem Formulation: We study the problem of linear
it is uniquely given by the LMTA-MSE estimator dˆopt [n].
estimation of a scalar zero-mean SOI d[n] based on an M × 1
Proof: Applying the time-averaging operator h·iN0 to both
multivariate zero-mean input signal x[n], where x[n] and d[n]
are JPC and JWSCS with period N0 . Let h denote an M × 1 sides of (2) yields (4). Hence, when the conditions of Lemma
time-invariant coefficients vector. The linear estimate of d[n] 1 are satisfied, the LMMSE estimator is also TA-MSE optimal.
Uniqueness follows from the non-singularity of Cx , that
is given by
˜
ˆ = hH x[n],
d[n]
n ≥ 0.
(1) implies strict convexity of the TA-MSE objective (3).
The property stated in Lemma 1 further motivates the use
The formulation (1) accommodates a wide range of filters,
of
the TA-MSE as an objective function.
including scalar LTI and LPTV filters. For example, when
Adaptive LMS Estimator Based on TA-MSE: The prothe input signal is a scalar signal, denoted by r[n], the vector
input signal x[n] is obtained by a multivariate mapping of r[n]. posed algorithm is obtained via instantaneous approximation
Then, a finite impulse response (FIR) LTI filter with M taps of the SD algorithm under the TA-MSE objective function (3).
is obtained by letting (x[n])k = r[n−k], k ∈ {0, 1, . . . , M − For a fixed step-size µ, the SD update equation is [2, Ch. 8.3]
1} [2, Ch. 10.5], and a scalar FRESH filter is obtained by
∂
(a)
h[n+1] = h[n]−µ ∗J (h) = h[n]+µ(cxd − Cx h[n]) , (5)
setting K, L ∈ N such that M = K · L, selecting a set of
∂h
˜
˜
K−1
h=h[n]
K cyclic frequencies {αk }k=0 , and letting (x[n])u·L+v =
jαu (n−v)
r[n−v]e
, u ∈ {0, 1, . . . , K −1}, v ∈ {0, 1, . . . , L−1} where (a) follows by applying the gradient to the
[9], [10]. As any DT LPTV system can be realized as a FRESH TA-MSE (3). Define the M × N0 matrix X [n] ,
i
x [n], x [n−1], . . . , x [n−N0 +1] , and the N0 × 1 vech
iT
tor d [n] , √1N d [n], d [n−1], . . . , d [n−N0 +1] . As Cx
0
˜
and cxd represent time-averaged correlations of the JWSCS
˜
signals, their corresponding instantaneous unbiased estimators
are given by X[n] XH [n] and X[n] d∗ [n], respectively.
As in the derivation of the standard LMS algorithm [2,
Ch. 10.2], the TA-LMS algorithm is obtained from the SD
(5) by replacing the time-averaged covariances with their
instantaneous estimates. The resulting update equation is
h[n+1] = h[n]+µX [n] d∗ [n]−XH [n] h [n] , (6)
n ≥ 0. Note that (6) specializes to the standard LMS for WSS
signals when the period is N0 = 1, i.e., the signals are JWSS.
√1
N0
h
IV. TA-LMS P ERFORMANCE A NALYSIS
We analyze the performance of the TA-LMS algorithm (6)
for linear estimation of WSCS signals.
A. Data Model and Assumptions
Since condition (2) is not necessarily satisfied, i.e., a timeinvariant LMMSE estimator does not necessarily exist, we let
hM [n] denote the possibly time-varying coefficients of the
LMMSE estimator and v[n] be the resulting estimation error.
The SOI d[n] and the LMMSE estimator are related via:
d[n] = hH
(7)
M [n]x[n] + v[n].
Note that the stationary linear data model used for the analysis
of the standard LMS algorithm [2, Ch. 10.2] is a special case
of (7), obtained by setting N0 = 1.
Since d[n] and x[n] are JWSCS with period N0 , it follows
from [13, Ch. 17.5.1] that hM[n] defines an M × 1 periodic
sequence , hM[n] = hM [n+N0 ], ∀n ∈ Z. Combined with
the fact that d[n] and x[n] are zero-mean JPC JWSCS, it
follows from (7) that v[n] is a zero-mean PC WSCS process
with period N0 . Furthermore, E{x[n]v ∗ [n]} = 0M ×1 by the
orthogonality principle [2, Ch. 4.2]. Similarly to the standard
approach used for analyzing the LMS algorithm for JWSS
signals, e.g., [1], [2], [15], we make the following assumptions:
AS2 The estimation error of the LMMSE estimator, v [n1 ],
and the input signal x [n2 ] are mutually independent ∀n1 , n2 ,
see also [2, Ch. 15.2], [15, Sec. B.2]. This is satisfied, for
H
example, if the SOI d[n] is of the form d[n] = gN
[n]x[n] +
0
z[n] where gN0 [n] is a deterministic periodically time-varying
coefficients vector and z[n] is a PC WSCS process independent
of x[n]. In this case hM[n] = gN0 [n] and v[n] = z[n].
AS3 The coefficients vector h[n] is independent of the
instantaneous input vector x[n],
[2, Ch. 16.4].
see [1, Pg. 392],
n
AS4 h[n] is independent of x [k] xH [k] k=n−N0+1 . A similar assumption was used in [14].
Note that AS3-AS4 are satisfied asymptotically, when the
filter has converged, since then the effect of the inputs on the
filter coefficients is negligible.
In addition to AS2-AS4, we make the following assumption:
n
T
o
AS5 The matrix B[n] , E X[n]XH [n] ⊗ X[n]XH [n]
is time-invariant, hence B[n] = B. We note that a similar assumption was made in the analysis of LMS with non-Gaussian
WSS inputs in [2, Eq. (24.9)]. This assumption is satisfied
when, for example, x[n1 ] and x[n2 ] are mutually independent
∀n1 6= n2 and x[n] is fourth-order cyclostationary.
We emphasize that AS1-AS5 are made only to facilitate
performance analysis and are not needed for the derivation of
the TA-LMS algorithm in Section III, which assumes only that
the signals are JPC and JWSCS.
B. Performance Measures
We begin by defining the error measures. Denote the
coefficients error vector of the TA-LMS algorithm w.r.t. the
MTA-MSE filter, hopt , obtained from (4), by h̄ [n] , hopt −
h [n], the error between the SOI and its TA-LMS estimate by
e[n] , d [n] − hH [n] x [n], and the error between the SOI and
the LMTA-MSE estimate by eopt [n] , d [n] − hH
opt x [n]. In
the transient performance analysis we characterize the timeevolutions of the expected coefficients error vector, E{h̄[n]},
and of the excess TA-MSE (ETA-MSE), defined as the difference between the TA-MSE at the TA-LMS output and at the
LMTA-MSE estimator output, i.e.,
D n
o
n
oE
2
2
ξ[n] , E |e [n]| − E |eopt [n]|
.
(8)
N0
Convergence of the algorithm is defined as follows:
Definition 1. The TA-LMS filter
is said
o to be convergent if
n
2
lim E h̄[n] = 0M ×1 and E h̄[n]
is convergent [1, Ch.
n→∞
9.4] [2, Ch. 23.4].
In the steady-state performance analysis we characterize the
asymptotic behavior of a convergent filter, via the steady-state
ETA-MSE, defined as ξs , lim ξ[n].
n→∞
C. Performance Analysis
Transient Performance: We first characterize the timeevolutions of expected coefficients error vector and of the
ETA-MSE. To that aim, let g[n] , hM [n] − hopt denote the
difference between the LMMSE filter and the LMTA-MSE
filter, and define
g̃[n] , x [n] xH [n] g [n] N0 .
(9)
Additionally, define the M × M matrices
Rx [n] , IM −µX [n] XH [n] ,
(10)
n
Q
n
n
Rx [l] for k ≤ n and Lk , IM for k > n, and
Lk ,
l=k
R̃x , E {Rx [n]} = IM − µCx .
(11)
˜
Lastly, define the N0 × 1 random vector v [n] ,
T
√1
v [n], v [n − 1], . . . , v [n − N0 + 1] , the N02 × 1 vector
N0
cv [n, l] , vec E v [n] vH [n − l] ,
(12)
the M 2 × 1 vectors
n
o
pxv[n] ,vec E g̃[n]g̃H [n] +E{X∗ [n]⊗X[n]}c∗v [n, 0], (13)
oH
o n n
n
T
, (14)
zxg [n] , E h̄ [0] R̃Tx E RTx [n]⊗g̃H [n]
X
n
o H
n
T
zxv[n] ,
cTv [n, k]E Lnn−k+1 X[n−k] ⊗XH [n] , (15)
k=1
and the M 2 × M 2 matrix
(a)
(16)
F[n] , E RTx [n] ⊗ Rx [n] ≡ F,
where (a) follows from the definition of X[n] and of Rx [n]
(10) and from AS5.
The time-evolution of the expected coefficients error vector
is characterized in the following lemma:
where B is defined in AS5. Using these definitions, we state
a sufficient condition for the TA-LMS to be convergent:
Lemma 2. The TA-LMS mean coefficients error is given by
n
E h̄ [n] = (IM −µCx ) E h̄ [0] ,
n ≥ 0.
(17)
˜
[A proof is given in Appendix A]
Theorem 2. Assume that H has at least one real-valued
positive eigenvalue,
then the TA-LMS algorithm converges
if
2
1
1
0 < µ < min
. (24a)
,
,
λmax (Cx ) λmax (A−1 B) λmax (H)
˜
If H does not have any real positive eigenvalues then the
TA-LMS converges
if
1
2
.
(24b)
,
0 < µ < min
λmax (Cx ) λmax (A−1 B)
˜
[A proof is given in Appendix C]
Next, we characterize the time-evolution of the ETA-MSE:
Theorem 1. Define ck , cx ((k))N0 . For all n ≥ N0 − 1,
the ETA-MSE of the TA-LMS
n filter isogiven by n
n
o
X
1
2
H
ξ[n] =
E h̄ [k] c +2 · Re E h̄ [0]
k
N0
k=n−N0+1
k
, (18)
× R̃x Cx ((k))N0 g ((k))N0
o
n
2
is computed via
where E h̄ [k] c
k
n
o
o
n
n
X
2
2
l−1
ck
E h̄[n] c = E h̄ [0] Fn c +µ2 pH
xv [n−l] F
k
k
l=1
)
n
X
H
H
l−1
−2µ·Re
zxg [n−l]−µ·zxv [n−l] F ck , (19)
(
l=1
and R̃x , pxv , zxg , zxv , and F are defined in (11), (13), (14),
(15), and (16), respectively.
[A proof is given in Appendix B]
Convergence and Steady-State Analysis: In order to characterize the steady-state performance and derive conditions for
convergence, we introduce two additional assumptions:
AS6 The temporal correlation of v[n] is bounded and spans
a finite interval, i.e., E{v[n+l]v ∗[n]} is bounded ∀n, l ∈ Z
and ∃Lmax > 0 s.t. E{v[n+l]v ∗[n]} = 0 for all |l| ≥ Lmax .
AS7 Define η , Lmax + N0 . All the 2N -th order moments
of x[n], N ∈ {1, 2, . . . , η + 1}, are bounded and periodic1
2N−1
with period N0 . Specifically, for any set of pairs {ki , ni }i=0
where
N−1 ki ∈ {0, 1, . . . , M − 1} and
ni ∈ N, we assume that
Q
∗
E
(x [n2i ])k2i (x [n2i+1 ])k2i+1 is bounded and equal to
i=0
N−1
Q
∗
E
(x [n2i +N0 ])k2i (x [n2i+1 +N0 ])k2i+1 . This is satisfied
i=0
when, e.g., x[n] is PC Gaussian WSCS with period N0 .
Next, define the M 2 × 1 vector
X
η
n
T
oH
zsxv[n] ,
cTv [n, k]E Lnn−k+1 X[n−k] ⊗XH [n] , (20)
k=1
where η is defined in AS7, the M 2 × M 2 matrices
A , CTx ⊗IM + (IM ⊗ Cx ) ,
(21)
˜
˜
N0−1 X
N0
1 X
cx [k] pxv ((k−l))N0
P,
N0
k=0 l=1
H l−1
F , (22)
+2zsxv ((k−l))N0
and the 2M 2 × 2M 2 matrix
1
A
−B
,
(23)
H,
2 2IM 2 0M 2 ×M 2
1 Note that in the analysis of the LMS algorithm for non-Gaussian WSS
inputs it was assumed that the fourth-order moments of the input are timeinvariant [2, Eq. (24.9)] and bounded [2, Pg. 361].
The steady-state ETA-MSE of the TA-LMS algorithm is
explicitly stated in the following:
Theorem 3. When (24) is satisfied, the steady-state ETA-MSE
of the TA-LMS algorithm
n isngiven by
−1 oo
2
,
(25)
ξs = µ · Re Tr P IM 2 − FN0
where F and P are defined in (16) and (22), respectively.
[A proof is given in Appendix D]
It can be shown that when N0 = 1, i.e., the considered
signals are JWSS, and v[n] is temporally uncorrelated, then,
the TA-LMS specializes to the standard LMS, and (25) specializes to the excess MSE of the standard LMS algorithm
with non-Gaussian inputs stated in [2, Thm. 24.1].
V. N UMERICAL E XAMPLES
In this section we evaluate the performance of the TA-LMS
algorithm and demonstrate the theoretical results presented
in Section IV for two scenarios: SN1, a scenario which
satisfies AS1-AS7; and SN2, a practical NB-PLC channel
estimation scenario. In scenario SN1 we let φ[n] be an i.i.d.
process,
uniformly
distributed over [0, 2π), and set r[n] =
1 + 0.5 sin 2πn
ejφ[n] , with period N0 = 20. The vector
N0
x[n] is constructed from r[n] by setting (x[n])k = r[n]ejαk n ,
2π
. The resulting
k ∈ {0, 1, 2}, {α0 , α1 , α2 } = {−1, 0, 1} · N
0
coefficients vector h[n] corresponds to a FRESH filter with
K = 3 branches, where each branch has L = 1 taps. Hence, the
dimension of the coefficients vector is M = 3. Furthermore,
we
set hM [n]
inthe data model (7) to satisfy (hM [n])k =
1+0.5 sin 2πn
e−|k+1| . The process v[n] in (7) is set to
N0
be a WSCS temporally uncorrelated Gaussian
process,
inde
pendent of x[n], with variance σv2 [n] = γv 1+0.5 sin 2πn
,
N0
where γv is set to achieve a signal-to-noise ratio (SNR) of 7
hE{|d[n]−v[n]|2 }iN0
. It can be
dB, defined here as SNR ,
hE{|v[n]|2 }iN0
shown that for this setting AS1-AS7 are satisfied.
In scenario SN2 we study a practical channel estimation
scenario. We consider a scalar NB-PLC channel with input
signal r[n] and output signal d[n], which is the SOI in this
scenario. The signal r[n] is set to be a passband OFDM
signal with 36 subcarriers, each modulated with a QPSK
constellation, with 4 cyclic prefix samples. As shown in [10,
Sec. II.C], r[n] is WSCS with period N0 = 40. The signal r[n]
is transmitted over an NB-PLC channel, modeled via a real
4
40
SN1, Empirical
SN1, Theoretical
SN2, Empirical
SN2, Theoretical
−2
2
−2.5
1
−3
100
120
140
0
−1
−2
SN1, Empirical
SN1, Theoretical
SN2, Empirical
SN2, Theoretical
35
30
Steady−state TAMSE [dB]
TA−MSE [dB]
3
25
−2
20
−3
15
SN1 Stability
bound
10
−4
0.2
5
0.4
0.6
0.8
SN2 Stability
bound
0
−3
−5
−4
50
100
150
200
Iteration
250
300
350
−10
0
0.2
0.4
0.6
0.8
Step−size
1
1.2
1.4
1.6
Fig. 1. TA-MSE comparison for scenarios SN1 and SN2.
Fig. 2. Steady-state TA-MSE comparison for scenarios SN1 and SN2.
LPTV channel transfer function (CTF) g[n, l] with additive
real WSCS Gaussian noise w[n], both with period N0 . Hence,
∞
P
g[n, l]r[n − l] + w[n], see [5, Sec. III.C]. Here,
d[n] =
the signals are JWSS the proposed algorithm specializes to
the standard LMS algorithm. The simulation study shows that
the theoretical analysis reliably characterizes the empirical
performance of the algorithm.
A PPENDIX
A. Proof of Lemma 2
It follows from the TA-LMS update equation (6) that
h̄[n+1] = h̄[n]−µ X [n] d∗ [n] − X [n] XH [n] h [n] . (26)
l=0
the estimator (1) represents the output of a FRESH filter
whose input is r[n], with K = 3 branches corresponding
2π
to frequency shifts {α0 , α1 , α2 } = {−1, 0, 1} · N
. At each
0
branch there is an FIR filter with L = 3 taps, and the overall
dimensionality is thus M = 9. The multivariate input signal
x[n] is constructed from r[n] according to the description
below (1). Following [5], we generate the LPTV CTF g[n, l],
n ∈ N0 , using the channel generator proposed in [16] adapted
to NB-PLC by setting the parameters as detailed in [5, Sec.
V]. The noise w[n] is simulated based on the model adopted
by the IEEE P1901.2 standard [17], with a set of typical
parameters which corresponds to low voltage site 11 (LV11)
in [17, Appendix G]. The MMSE filter hM [n] in (7) is
obtained from the orthogonality principle and v[n] is obtained
as the estimation error of hM [n]. Here, the SNR is defined
hE{|d[n]−w[n]|2 }iN0
. As NB-PLC channels are
as SNR ,
hE{|w[n]|2 }iN0
typically characterized by low SNR, we chose SNR = 0dB.
Fig. 1 depicts the theoretical TA-MSE for both scenarios
and for step-size µ = 0.01. We note that in both scenarios
there is an excellent agreement between the theoretical and
the empirical TA-MSEs. In Fig. 2, the theoretical steadystate TA-MSE is compared to its empirical value for various
step-sizes and with the theoretical stability threshold, obtained
from (24). Observing Fig. 2, we note that in both scenarios
the theoretical steady-state TA-MSE provides an accurate
characterization of the empirical steady-state TA-MSE. It is
also noted that when AS1-AS7 are satisfied, the stability of
the algorithm is accurately predicted using the step-size region
in (24), however, in scenario SN2 there is a gap between the
theoretical stability threshold and its empirical measurement,
which is due to the fact that, unlike scenario SN1, assumptions
AS1-AS7 are not satisfied here.
VI. C ONCLUSIONS
In this work, a new adaptive algorithm for linear estimation of JWSCS signals, based on the TA-MSE objective is
introduced and its performance and stability are characterized
without imposing specific distribution on the signals. When
Note that for n ≥ 0
(a)
X [n] d∗ [n] =
N0−1
1 X
x [n−k] xH [n−k] (hopt +g [n−k])
N0
+
(b)
k=0
N
0−1
X
1
N0
x [n−k] v ∗ [n−k]
k=0
= X [n] XH [n] hopt +g̃ [n]+X [n] v∗ [n] , (27)
where (a) follows by plugging (7) into the definition of d [n]
and using the definition of g[n], and (b) follows from the
definition of g̃[n] (9). Plugging (27) into (26) yields
h̄[n+1] = IM −µX[n]XH [n] h̄[n]−µ g̃[n]+X[n]v∗ [n] . (28)
Next, we show that E{g̃ [n]} = 0M ×1 , ∀n ∈ Z. Since v[n]
is zero-mean and independent of x[n] by AS2, applying the
Hermitian transpose to (7), followed by left multiplication
of both sides by x[n], and then applying the stochastic
expectation and using AS2 yields
E {x [n] d∗ [n]} = E x [n] xH [n] (hopt + g[n]) . (29)
Applying the time-averaging operator to (29) and recalling the
definition of g̃[n] (9) results in cxd = Cx hopt + E{g̃ [n]} =
˜ optimal
˜ filter, it follows that
0M ×1 . Since hopt is the TA-MSE
E{g̃ [n]} = 0M ×1 . As also E{X [n]v∗ [n]} = 0M ×1 by AS2,
applying the stochastic expectation to (28) yields the recursion
E h̄[n] = (IM −µCx ) E h̄ [n − 1] .
(30)
˜
Repeating the recursion in (30) n times yields (17).
B. Proof of Theorem 1
We begin by stating the following equalities and properties
that will be used in the sequel (see [18, Ch. 9.2]):
R1 For any matrix triplet A1 , A2 , A3 of compatible dimensions, it holds that
vec (A1 A2 A3 ) = AT3 ⊗ A1 vec (A2 ) .
(31)
R2 For any pair of square matrices A1 , A2 of identical
dimensions, it holds that
T
Tr AT1 A2 = vec (A1 ) vec (A2 ) .
(32)
R3 For any pair of matrices A1 , B1 , it holds that
T
(A1 ⊗ B1 ) = AT1 ⊗ BT1 ,
(33)
R4 For any four matrices A1 , A2 , B1 , B2 of compatible
dimensions, it holds that
A1 A2 ⊗ B1 B2 = (A1 ⊗ B1 ) (A2 ⊗ B2 ) .
(34)
In order to prove the theorem, we first derive a recursive
relationship for the mean of the weighted squared Euclidean
norm of h̄ [n], which is stated in the following proposition:
Proposition 1. For any M 2 × 1 vector q such that Q =
vec−1 {q} is Hermitian positive semi-definite, the coefficients
error vector h̄[n] satisfies the following recursion for n ≥ 0:
o
o
n
n
2
2
E h̄ [n+1] q = E h̄ [n] Fq +µ2 pH
xv [n]q
H
−2µ · Re zH
[n]−µ
·
z
[n]
q
, (35)
xg
xv
where F, pxv , zxg , and zxv are defined in (16), (13), (14),
and (15), respectively.
Proof: Let Q be the M × M Hermitian matrix obtained via
Q , vec−1 (q). Note that
h̄ [n+1]
(a)
H
2
q
H
, h̄ [n+1] Q h̄ [n+1]
2
= h̄ [n] Rx [n]QRx [n]h̄ [n]+µ2 ·kg̃ [n]+X [n] v∗ [n]kq
n
o
−2µ·Re vT [n] XH + g̃H [n] QRx [n]h̄ [n] , (36)
where F is defined in (16). Therefore,
o
n
H
E h̄ [n] Rx [n]QRx [n]h̄ [n]
n n
oo
(a)
H
= E E h̄ [n] Rx [n]QRx [n]h̄ [n] h̄ [n]
o
n
(b)
H
= E h̄ [n]E {Rx [n]QRx [n]} h̄ [n]
n
o
(c)
2
= E h̄ [n] Fq ,
(39)
where (a) follows from the law of total expectation [21, Ch.
7.4]; (b) follows from the independence assumption AS4;
and (c) follows from (38) and the definition of the weighted
Euclidean norm. Next, note that
o
n n
o o
n
2
E kg̃ [n]kq = Tr E g̃ [n] g̃H [n] Q
n
oH
(a)
= vec E g̃ [n] g̃H [n]
q,
(40)
where
(a)
follows
n
o∗
vec E g̃ [n] g̃H [n]
and
since
(32)n
oT
vec E g̃ [n] g̃H [n]
.
from
=
Similarly,
o
n
2
E kX [n] v∗ [n]kq
n
o
= Tr E v∗ [n] vT [n] XH [n] QX [n]
n
o
(a)
= Tr E v∗ [n] vT [n] E XH [n] QX [n]
T
(b)
= vec E v [n] vH [n]
vec E XH [n] QX [n]
(c) T
= cv [n, 0] E XT [n] ⊗ XH [n] q
n
o
H
(d)
q,
(41)
= E X∗ [n] ⊗ X [n] c∗v [n, 0]
where (a) follows as v[n] is independent of X[n] by AS2;
(b) follows from property R2; (c) follows from property
R1 and from the definition of cv [n, l] in (12); (d) follows
since conjugate transposition is distributive over the Kronecker
product [18, Ch. 9.2]. It follows from plugging definition (13)
into (40) and (41) That
o
o
n
n
2
2
(42)
E kg̃ [n]kq + E kX [n] v∗ [n]kq = pH
xv [n]q.
where (a) follows from the relationship (28). Since v[n] is
zero-mean and independent of X[n] by AS2, applying the
In order to obtain the third element in (37), we note that
stochastic expectation to (36) results in
repeating the recursion in (28) n times for n ≥ 1 yields
n
o
n
o
2
H
E h̄ [n+1] q = E h̄ [n] Rx [n]QRx [n]h̄ [n]
h̄[n] = L0n−1 h̄ [0]
o
o
n
n
n
X
2
2
n−1
+µ2 ·E kg̃ [n]kq +µ2 E kX [n] v∗ [n]kq
−µ
X [n−k] v∗ [n − k]+g̃ [n−k] .
Ln−k+1
n
o
k=1
−2µ·Re E vT [n] XH [n] QRx [n]h̄ [n]
n n
oo
From the definition of Lnk it follows that Rx [n]Lkn−1 = Lnk ,
H
−2µ·Re E g̃ [n] QRx [n]h̄ [n] . (37) ∀k ≤ n, it follows that
Next, we explicitly compute each of the elements in the right E vT [n] XH [n] QRx [n]h̄ [n] = E vT [n] XH [n] QLn0 h̄ [0]
n
hand side (RHS) of (37). First, from (31) it follows that
X
T
H
n
∗
n
o
−µ
E v [n] X [n] QLn−k+1 X [n−k] v [n−k]
vec E {Rx [n]QRx [n]} = E vec (Rx [n]QRx [n])
k=1
n
X
= E RTx [n] ⊗ Rx [n] q
E vT [n] XH [n] QLnn−k+1 g̃ [n−k] .
(43)
−µ
= Fq,
(38)
k=1
The first and the last elements in the RHS of (43) are zero
n
since v[n] is zero-mean and independent of {X[l]}l=0 by AS2.
As for the second element, we note that
E vT [n] XH [n] QLnn−k+1 X [n−k] v∗ [n−k]
(a)
= Tr E v∗ [n−k] vT [n]
× E XH [n] QLnn−k+1 X [n−k]
T
(b)
= Tr E v [n] vH [n−k]
H
n
× E X [n] QLn−k+1 X [n−k]
(c)
= cTv [n, k] vec E XH [n] QLnn−k+1 X [n−k]
n
o
T
(d) T
= cv [n, k] E Lnn−k+1 X [n−k] ⊗XH [n] q,
(44)
where (a) follows since v[n] and X[n] are mutually independent processes by AS2; (b) follows since
T
E v∗ [n−k] vT [n] = E v [n] vH [n−k]
; (c) follows
from property R2; (d) follows from property R1. Plugging (44)
into (43) yields
E vT [n] XH [n] QRx [n]h̄ [n]
n
o
n
T
X
= −µ
cTv [n, k] E Lnn−k+1 X [n−k] ⊗XH [n] q
k=1
(a)
= −µ · zH
xv [n]q,
(45)
where (a) follows from (15). Lastly, we write
n
o
E g̃H [n] QRx [n]h̄ [n]
o
n
(a)
= E g̃H [n] QRx [n]E h̄ [n]
n
o
T
(b)
= E Rx [n]E h̄ [n]
⊗g̃H [n] q
oT
n
(c)
q
= E Rx [n] E h̄ [n] ⊗g̃∗ [n]
T
(d)
= E {Rx [n]⊗g̃∗ [n]} E h̄ [n]
q
n
o
T
n
(e)
R̃x E h̄ [0]
=
E RTx [n]⊗g̃H [n] q
(f )
= zH
xg [n] q,
H
(46)
(47a)
and
eopt [n] = gH [n] x [n] + v [n] .
(48)
where (a) follows since by AS2, x[n] is zero-mean and
independent of v[n], and by AS3, x[n] is also independent
of h̄[n]. The ETA-MSE is obtained by time-averaging (48),
which results in
2
H
ξ[n] = E h̄ [n] x [n]
N0
D n
oE
H
. (49)
+ 2Re
E h̄ [n] x [n] xH [n] g [n]
N0
Note that the first summand in the RHS of (49) satisfies
oo
n n
2
(a)
H
H
= E E h̄ [n]x[n] xH [n]h̄[n] h̄[n]
E h̄ [n] x [n]
n
o
(b)
H
= E h̄ [n] E x[n] xH [n] h̄[n]
o
n
(c)
H
= E h̄ [n] Cx [n] h̄ [n]
o
n
(d)
2
(50)
= E h̄ [n] c [n] ,
x
where (a) follows from the law of total expectation [21, Ch.
7.4]; (b) follows from the independence assumption AS3; (c)
follows from the definition of Cx [n]; and (d) follows from
the definition of cx [n]. Next, we note that
o
n
H
E h̄ [n] x [n] xH [n] g [n]
o
n
(a)
H
= E h̄ [n] E x [n] xH [n] g [n]
n
o n
(b)
H
= E h̄ [0] R̃x Cx [n] g [n] ,
(51)
where (a) follows from the independence assumption AS4;
(b) follows from property R1; (c) follows from property R3;
(d) follows from property R4 by writing g̃∗ [n] = g̃∗ [n] · 1; (e)
follows from Lemma 2; (f ) follows from definition (14).
Plugging (39), (42), (45) and (46) into (37) yields (35).
It follows from (47) that applying the recursion (35) n times
with q = ck yields (19).
Next, we prove (18). Plugging the relationship (7) into the
definitions of e[n] and eopt [n] results in
e [n] = h̄ [n] x [n] + gH [n] x [n] + v [n] ,
o
o
n
n
2
2
E |e[n]| −E |eopt [n]|
2
H
=E
h̄ [n]+g [n] x [n]
n n
oo
H
+ 2Re E h̄ [n]+g [n] x [n] v ∗ [n]
o
n
2
− 2Re E gH [n]x [n] v ∗ [n]
− E gH [n]x [n]
n
o
2
H
(a)
2
=E
,
h̄ [n]+g [n] x [n] −E gH [n]x [n]
(47b)
where (a) follows from the independence assumption AS3,
and (b) follows from Lemma 2 and from the definition of
Cx [n]. Plugging (50) and (51) into (49) for n ≥ N0 − 1
results in
oE
D n
2
ξ[n] = E h̄ [n] c [n]
x
N0
D n
o n
E
H
E h̄ [0] R̃x Cx [n] g [n]
+2Re
N0
n
X
1
2
E h̄ [k] c [((k)) ]
x
N0
N0
k=n−N0 +1
n
!
o k
H
,
+2Re E h̄ [0] R̃x Cx ((k))N0 g ((k))N0
(a)
=
(52)
where (a) follows since cx [n], Cx [n], and g[n] are all periodic
with period N0 .
C. Proof of Theorem 2
We begin by noting some properties satisfied by the quantities defined in Subsection IV-C, stated in the following lemma:
zsxv [n],
Lemma 3. The random vectors zxv [n],
and pxv [n],
defined in (15), (20), and (13), respectively, satisfy:
P1 zxv [n] and zsxv [n] are equal ∀n ≥ η, where η is defined
in AS7.
P2 pxv [n] and zsxv [n] are periodic with period N0 .
Proof: Property P1 follows from AS6 since cv [n, l] defined
in (12) satisfies cv [n, l] = 0N02 ×1 , ∀|l| > η, where η is defined
in AS7. Therefore, it follows that zxv [n] and zsxv [n] defined
in (15) and (20), respectively, are equal ∀n ≥ η.
Lastly, from the cyclostationarity of v[n] combined with
AS7 it follows that pxv [n] and zsxv[n], defined in (13) and
(20), respectively, are periodic with period N0 This proves
P2.
We now show that lim E h̄[n] = 0M ×1 if and only if
n→∞
0 < µ < λmax2(Cx ) . Note that Cx is obtained as the average of
˜
N0 Hermitian ˜and positive semi-definite
covariance matrices.
Thus, Cx is also Hermitian positive semi-definite, and its
M
˜
eigenvalues
{λk }k=1 are all real-valued and non-negative.
From AS1 it follows that the eigenvalues of Cx are
strictly
˜ Eh̄[n] =
positive. From (11) and (17) it follows that lim
n→∞
n
0M ×1 for every h[0] if and only if lim R̃x = 0M ×M .
n→∞
M
Since R̃x = IM − µCx , its eigenvalues are {1−µ · λk }k=1 ,
where λk > 0, ∀k˜ ∈ {1, 2, . . . , M }, we conclude that
the TA-LMS algorithm is mean convergent if and only if
|1−µ · λk | < 1 for all k ∈ {1, 2, . . . , M } [20, Ch. 7.10], which
occurs if and only if 0 < µ < λmax2(Cx ) .
n
o
˜
2
Next, we show that if (24) is satisfied, then E h̄[n]
is convergent. For n ≥ N0 , repeating the recursion (35) N0
times yields
n
o
n
o
2
2
E h̄ [n] q = E h̄ [n−N0 ] FN0 q
+µ2 ·
N0
X
l−1
pH
q
xv [n−l] F
−2µ·Re
l=1
l−1
H
[n−l]
F
q
,
[n−l]−µ·z
zH
xv
xg
where pxv [n], F, zxg [n] and zxv [n] are defined in (13), (16),
(14), and (15), respectively. Define ztxv [n] , zxv [n] − zsxv [n],
where zsxv [n] is defined in (20), it follows that
n
o
n
o
2
2
E h̄ [n] q = E h̄ [n−N0 ] FN0 q
(N
)
0
X
H
H
2
s
l−1
pxv [n−l]+2 (zxv [n−l])
+µ ·Re
F q
l=1
(N
)
0
X
H l−1
H
t
zxg [n−l]−µ zxv [n−l]
F q . (53)
−2µ·Re
l=1
Define
h̄k [n] , h̄[n · N0 + k],
ak ,
X
N0
l=1
F̄ , FN0 ,
pxv ((k−l))N0
+2zsxv
and
bk [n] ,
X
N0
(54b)
((k−l))N0
H
F
l−1
H
,
(54c)
zH
xg [(n+1) N0 +k−l]
l=1
− µ ztxv [(n+1) N0 +k−l]
H
Fl−1
H
. (54d)
From AS6-AS7 it follows that ak and bk [n] are bounded ∀k ∈
s
N0 . From P2 in Lemma 3 it follows that pxv [n] and
zxv [n]
are periodic with period
N0, hence, pxv [n] = pxv ((n))N0
and zsxv [n] = zsxv ((n))N0 , hence, (53) can be written as
n
o
n
o
2
2
E h̄k [n+1] q = E h̄k [n] F̄q +µ2 ·Re aH
k q
H
− 2µ·Re bk [n] q .
(55)
The
deviation (MSD) is obtained from
o
n mean-square
2
E h̄ [n] q by setting q = vec−1 (IM ). Following [2, Ch.
24.2], we use (55) to formulate M 2 state-space recursions for
each k ∈ N0 as follows:
o
o
n
n
2
2
l
E h̄k [n+1] F̄l q = E h̄k [n] F̄l+1 q +µ2 ·Re aH
k F̄ q
l
(56)
−2µ·Re bH
k [n] F̄ q ,
M 2 −1
l ∈ {0, 1, . . . , M 2 − 1}. Let {αl }l=0
be the coefficients
of the characteristic polynomial of F̄ [20, Pg. 492]. It follows from the Cayley-Hamilton theorem [20, Pg. 532] that
2
MP
−1
2
αl F̄l . Hence, it follows from the linearF̄M = −
l=0
ity of the weighted Euclidean norm [2, Eq. (23.31)] that
2
n
n
o
o
MP
−1
2
2
E h̄k [n] F̄M 2 q = −
αl E h̄k [n] F̄l q . Therefore, by
l=0
l=1
X
N0
o
n
2
E h̄k [n]
converges to a fixed and finite value for
n → ∞, ∀k ∈ N0 . To that aim, define for n ≥ 0
(54a)
k ∈ N0 . We will show that if (24) is satisfied, then
2
defining the M
k [n], ak , and bk [n], such that
n × 1 vectors
o h̄
˜
˜
2
l ˜
h̄k [n] l , E h̄k [n] F̄l q , (ak )l , aH
k F̄ q, and (bk [n])l ,
˜
˜2
˜
l
2
2
bH
k [n] F̄ q, l ∈ {0, 1, . . . , M −1}, and the M × M matrix
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0
F̄ , .
,
..
..
˜
.
0
0
0
0
1
−α0 −α1 −α2 −α3 . . . −αM 2 −1
the state-space recursions (56) can be written as a set of N0
multivariate difference equations
h̄k [n + 1] = F̄h̄k [n] + µ2 ·Re {ak } − 2µ·Re {bk [n]} , (57)
˜
˜˜
˜
˜
k ∈ N0 , n ≥ 0. Note that ∀k ∈ N0 , (57) represents an M 2 ×
M 2 multivariate LTI system with input signal µ2 Re {ak } −
˜
2µRe {bk [n]} and output signal h̄k [n]. We denote the impulse
˜
˜
response sequence matrix of the LTI system2 corresponding
2
2
ton(57) by the
o M × M matrix function G[n]. Note that
2
E h̄k [n]
is convergent if and only if h̄k [n] is bounded
˜
and tends to a steady-state value [2, Ch. 23.4]. In order to
analyze the conditions for steady-state convergence of h̄k [n],
a
b
a
b ˜
we write h̄k [n] = h̄k [n] + h̄k [n], where h̄k [n] and h̄k [n] are
˜ of the ˜system when
˜
the outputs
the input˜ signal is ˜µ2 ak and
˜ the
−2µRe {bk [n]}, respectively. In the following we study
a
b
˜
conditions for steady-state convergence of h̄k [n] and h̄k [n]:
˜
˜
a
a
Convergence of h̄k [n]: Note that h̄k [n] is obtained from
˜
˜ equation, for which
a first-order non-homogenous
difference
a
it follows from [22, Pg. 193] that h̄k [n] remains bounded
˜ if and only if all the
and converges to a steady-state value
eigenvalues of F̄ are inside the unit circle, which in turn is
˜
equivalent to BIBO
stability of the multivariate LTI system
with impulse response sequence matrix G[n] [23, Ch. 5.2].
b
Convergence of h̄k [n]: Let ⋆ denote the convolution operator and k·k1 be ˜the l1 norm, i.e., for a vector y ∈ CN ,
NP
−1
|(y)l |, and for a matrix Y ∈ CN1 ×N2 , kYk1 =
kyk1 =
l=0
NP
2 −1
1 −1 NP
lim hbk [n] =
n→∞
˜
l1 =0 l2 =0
0M 2 ×1 when the multivariate LTI system with impulse response sequence matrix G[n] is BIBO stable and µ <
2
λmax(Cx ) . This is achieved by showing that ∀ǫ > 0, ∃n0 (ǫ)
such ˜that ∀n > n0 (ǫ), hbk [n] 1 < ǫ. Note that the multivari˜
ate LTI system with impulse
response sequence matrix G[n]
∞
P
kG [l]k1 < ∞ [23,
is BIBO stable if and only if γ ,
(Y)l1 ,l2 . We next prove that
l=−∞
Thm. 5.MD1]. From (54d) we note that for µ <
lim
n→∞
bH
k
[n] = lim
n→∞
N0
X
zH
xg [(n+1) N0 +k−l]
l=1
− µ ztxv [(n+1) N0 +k−l]
(a)
= lim
n→∞
N0
X
l=1
2
λmax(Cx )
˜
(b)
H
Fl−1
l−1
zH
= 0M 2 ×1 , (58)
xg [(n+1) N0 +k−l] F
where (a) follows since ztxv [n] = zxv [n] − zsxv [n] = 0M 2 ×1
for all n ≥ η by property P1 in Lemma 3; (b) follows since for
µ < λmax2(Cx ) , it follows from the definition of zxg [n] (14) that
˜
lim zxg [n]
= 0M 2 ×1 . It follows that lim bk [n] = 0M 2 ×1 ,
n→∞
n→∞ ˜
and thus ∀ǫ̃ > 0, ∃ñ0 (ǫ̃) such that ∀n > ñ0 (ǫ̃), kbk [n]k1 < ǫ̃.
˜
ǫ
Let ǫ̃ = 2γ
and define
(
bk [n] 0 ≤ n ≤ ñ0 (ǫ̃)
L
bk [n] , ˜
,
0M 2 ×1 n > ñ0 (ǫ̃), n < 0
˜
(
0M 2 ×1 n ≤ ñ0 (ǫ̃)
R
bk [n] ,
.
bk [n] n > ñ0 (ǫ̃)
˜
˜
2 Following [23, Ch. 5], G[n] is said to be the impulse response sequence
matrix of an M 2 × M 2 multivariate LTI system with M 2 × 1 input sequence
xG [n] and M 2 ×1 output sequence yG [n], if its the input-output relationship
∞
P
satisfies yG [n] =
G[l]xG [n − l].
l=−∞
L
R
b
Clearly, bk [n] = bL
k [n]+ bk [n] and h̄k [n] = G[n]⋆ bk [n]+
˜
˜
˜
˜
˜
R
G[n] ⋆ bk [n], thus
˜
b
R
h̄k [n] ≤ G[n] ⋆ bL
k [n] 1 + G[n] ⋆ bk [n] 1 . (59)
1
˜
˜
˜
Next, note that bL
[n] is bounded and its support is finite,
˜ k summable. If the multivariate LTI system
hence it is absolutely
with impulse response sequence matrix G[n] is BIBO stable
then G[n] is absolutely summable [23, Ch. 5.2], which implies
that G[n]⋆ bL
[n] is also absolutely summable [24, Proposition
˜k
10.3]. It follows
that lim G [n] ⋆ bL
[n] = 0M 2 ×1 , thus
n→∞
˜k
∃n1 (ǫ) such that ∀n > n1 (ǫ),
ǫ
G[n] ⋆ bL
(60)
k [n] 1 < .
2
˜
As for the term G [n] ⋆ bR
[n], observe that if the multivariate
˜ k response sequence matrix G[n] is
LTI system with impulse
BIBO stable then
∞
X
G [n] ∗ bR
G [n − l] bR
k [n] 1 =
k [l]
˜
˜
l=−∞
1
(a)
=
∞
X
l=ñ0 (ǫ̃)
(b)
≤
∞
X
l=ñ0 (ǫ̃)
(c)
≤ γ · ǫ̃ ≤
G [n − l] bk [l]
˜
1
kG [n − l]k1 kbk [l]k1
˜
ǫ
,
2
(61)
where (a) follows from the definition of bR
[n]; (b) fol˜ k ∀n > ñ (ǫ̃),
lows from [20, Eq. (5.25)]; (c) follows since
0
kbk [n]k1 < ǫ̃, n
and as a consequence
of
BIBO
stability.
Setting
o
˜
ǫ
), n1 (ǫ) , then by plugging (60) and
n0 (ǫ) = max ñ0 ( 2γ
(61) into (59) it follows that ∀n > n0 (ǫ), hbk [n] 1 < ǫ,
˜
hence lim hbk [n] = 0M 2 ×1 .
n→∞ ˜
So far we have shown that if 0 < µ < λmax2(Cx ) and all the
˜
eigenvalues of F̄ are inside the unit circle (which
implies that
˜
the multivariate LTI system with impulse response sequence
matrix G[n] is BIBO stable), then h̄[n] is mean-square stable.
We now show that the latter condition is equivalent to the
constraints on A−1 B and H in (24). Note that it follows from
[2, Pg. 346] that the eigenvalues of F̄ are the eigenvalues of F̄.
Also note that from (54b) it follows˜that all the eigenvalues of
F̄ are inside the unit circle if and only if all the eigenvalues
of F are inside the unit circle. Since Cx is Hermitian and
positive-definite by AS1, it follows from ˜[20, Pg. 598] that A,
defined in (21), is also Hermitian positive-definite. We now
show that B, defined in AS5, is a positive semi-definite matrix.
2
Note that ∀q̃ ∈ CM and Q̃ = vec−1 (q̃),
n
T
o
q̃H Bq̃ = q̃HE X [n] XH [n] ⊗ X [n] XH [n] q̃
n
o
(a) H
= q̃ vec E X [n] XH [n] Q̃X [n] XH [n]
n n
oo
(b)
= Tr E Q̃HX [n] XH [n] Q̃X [n] XH [n]
n n
oo
= Tr E XH [n] Q̃HX [n] XH [n] Q̃X [n]
, (62)
wheren (a) follows from (31) and ob follows from (32).
As E XH [n] Q̃HX [n] XH [n] Q̃X [n] is Hermitian positive
semi-definite, (62) is non-negative, thus B is positive semidefinite. Recall that F = IM 2 − µA + µ2 B, where µ > 0,
A is positive-definite, and B is positive semi-definite and
finite (by AS7). It therefore follows from [19, Appendix A]
that when H has at least one real-valued positive eigenvalue
then the eigenvalues
n of F are guaranteed oto be inside the unit
1
1
, and when H has
circle if µ < min λmax(A
−1 B) , λ
max(H)
no real-valued positive eigenvalues, the eigenvalues of F are
1
guaranteed to be inside the unit circle if µ < λmax(A
−1 B) .
2
Combining this with the condition 0 < µ < λmax(Cx ) yields
˜
(24).
D. Proof of Theorem 3
In order to derive the steady-state ETA-MSE, we first show
that it can be obtained as the following limit:
Lemma 4. The steady-state ETA-MSE can be written as
D n
oE
2
ξs = lim E h̄ [n] cx [n]
.
(63)
n→∞
N0
Proof: Note that in steady-state
oE
D n
H
lim E h̄ [n] x [n] xH [n] g [n]
n→∞
N0
o
E
D n
(a)
H
H
= lim E h̄ [n] E x [n] x [n] g [n]
n→∞
(b)
N0
= 0,
from Def. 1, as
where (a) follows from AS3, and (b) follows
the filter is in steady-state, and as E x [n] xH [n] is finite
∀n ∈ Z. The steady-state ETA-MSE, obtained by taking n →
∞ in (49), is therefore given by
2
H
.
(64)
ξs = lim E h̄ [n] x [n]
n→∞
N0
Plugging (50) into (64) yields (63).
Assuming that µ satisfies (24), by Lemma 4 the steady-state
ETA-MSE is given by
D n
oE
2
ξs = lim E h̄ [m] cx [m]
m→∞
=
(a)
=
1
N0
1
N0
NX
0 −1
N0
lim E
n
h̄ [m − k]
lim E
n
h̄k [n]
m→∞
k=0
NX
0 −1
k=0
n→∞
2
cx [m−k]
2
cx [k]
o
,
o
(65)
where (a) follows from the definition of h̄k [n]
in (54a), the
periodicity of cx [n], i.e., cx [n] = cx ((n))N0 , and from the
fact that when the limit exists then any subsequence converges
to the limit [25, Def. 3.5]. Next, recalling the definitions of ak ,
bk [n], and F̄ stated in (54), it follows from (55) that ∀k ∈ N0
in the steady-state it holds that
o
o
n
n
2
2
lim E h̄k [n] q = lim E h̄k [n] F̄q +µ2 ·Re aH
k q
n→∞
n→∞
H
− 2µ· lim Re bk [n] q
n→∞
n
o
(a)
2
= lim E h̄k [n] F̄q +µ2 ·Re aH
k q ,
n→∞
guarantees
where (a) follows
from
n
o (58). Thus,nsince (24) o
2
2
that lim E h̄k [n] q and lim E h̄k [n] F̄q both exn→∞
n→∞
ist and are finite, then using the linearity of the weighted
Euclidean norm [2, Eq. (23.31)] we have
o
n
2
(66)
lim E h̄k [n] (I 2 −F̄)q = µ2 ·Re aH
k q ,
n→∞
M
−1
cx [k] in (66) results in
∀k ∈ N0 . Setting q = IM 2 − F̄
n
o
n
o
−1
2
lim E h̄k [n] c [k] = µ2·Re aH
cx [k] . (67)
k IM 2 − F̄
n→∞
x
Plugging (67) into (65) results in
N0 −1
o
n
−1
1 X
2
−
F̄
c
[k]
Re aH
I
x
M
k
N0
k=0
(
)
N0 −1
n
−1 o
1 X
2
H
= µ Re
Tr cx [k]ak IM 2 − F̄
N0
k=0
))
!
( (
N0 −1
−1
1 X
(a) 2
H
cx [k]ak
IM 2 − F̄
= µ Re Tr
N0
k=0
n n
−1 oo
(b) 2
,
(68)
= µ Re Tr P IM 2 − FN0
ξs = µ2
(a) follows from the linearity of the trace operator [18, Ch,
1.1]; (b) follows from the definitions of ak , F̄, and P in (54c),
(54b), and (22), respectively.
R EFERENCES
[1] S. Haykin. Adaptive Filter Theory. Prentice Hall, 2003.
[2] A. H. Sayed. Adaptive Filters. IEEE Press, 2008.
[3] W. A. Gardner, A. Napolitano, and L. Paura. “Cyclostationarity: Half a
century of research”. Signal Process., vol. 86, Apr. 2006, pp. 639-697.
[4] W. A. Gardner (Editor). Cyclostationarity in Communications and Signal
Processing. IEEE Press, 1994.
[5] N. Shlezinger and R. Dabora. “On the capacity of narrowband PLC
channels”. IEEE Trans. Commun., vol. 63, no. 4, Apr. 2015, pp. 1191
- 1201.
[6] D. C. McLernon. “Analysis of LMS algorithm with inputs from
cyclostationary random processes”. Electronics Letters, vol. 27, no. 2,
Jan. 1991, pp. 136–138.
[7] N. J. Bershad, E. Eweda, and J. C. M. Bermudez. “Stochastic analysis
of the LMS and NLMS algorithms for cyclostationary white Gaussian
inputs”. IEEE Trans. Signal Process., vol. 62, no. 9, May 2014, pp.
2238-2249.
[8] W. A. Gardner. “Cyclic Wiener filtering: theory and method”. IEEE
Trans. Commun., vol. 41, no. 1, Jan. 1993, pp. 151-163.
[9] O. A. Yeste-Ojeda and J. Grajal. “Adaptive-FRESH filters for compensation of cycle-frequency errors”. IEEE Trans. Signal Process., vol. 58,
no. 1, Jan. 2010, pp. 1-10.
[10] N. Shlezinger and R. Dabora. “Frequency-shift filtering for OFDM
signal recovery in narrowband power line communications”. IEEE
Trans. Commun., vol. 62, no. 4, Apr. 2014, pp. 1283-1295.
[11] D. C. McLernon. “One-dimensional linear periodically time-varying
structures, derivations, interrelationships and properties”. IEEE Proc.
Vis. Image Signal Process., vol. 146, no. 5, Oct. 1999, pp. 245–252.
[12] F. D. Nesser and J. L. Massey. “Proper complex random processes with
applications to information theory,” IEEE Trans. Inf. Theory, vol. 39,
no. 4, pp. 1293–1302, Jul. 1993.
[13] G. B. Giannakis. “Cyclostationary signal analysis”. Digital Signal
Processing Handbook, CRC Press, 1998, pp. 17.1–17.31.
[14] N. C. Shin and A. H. Sayed. “Mean-square performance of a family of
affine projection algorithms”. IEEE Trans. Signal Process., vol. 52, no.
1, Jan. 2004, pp. 90–102.
[15] W. A. Gardner. “Learning characteristics of stochastic-gradient-descent
algorithms: A general study, analysis, and critique”. Signal Process.,
vol. 6, no. 2, Apr. 1984, pp. 113–133.
[16] F. J. Cañete, J. A. Cortés, L. Dı́ez, and J. T. Entrambasaguas. “A channel
model proposal for indoor power line communications”. IEEE Commun.
Mag., vol. 49, no. 12, Dec. 2011, pp. 166–174.
[17] IEEE Standards Association. “P1901.2/D0.09.00 draft standard for low
frequency (less than 500 kHz) narrow band power line communications
for smart grid applications”. Jun. 2013.
[18] K. B. Petersen and M. S. Pedersen. The Matrix Cookbook. Technical
University of Denmark, 2008.
[19] T. Y. Al-Naffouri and A. H. Sayed. “Transient analysis of datanormalized adaptive filters”. IEEE Trans. Signal Processing, vol. 51,
no. 3, Mar. 2003, pp. 639–652.
[20] C. D. Meyer. Matrix Analysis and Applied Linear Algebra. Society for
Industrial and Applied Mathematics, 2000.
[21] A. Papoulis. Probability, Random Variables, and Stochastic Processes.
McGraw-Hill, 1991.
[22] S. Elaydi. An Introduction to Difference Equations. Springer Science
and Business Media, 2005.
[23] C. T. Chen. Linear System Theory and Design. Oxford University Press,
1995.
[24] J. D. Hamilton. Time Series Analysis. Princeton University Press, 1994.
[25] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill New
York, 1976.