Adaptive LMS-Type Filter for Cyclostationary Signals - Full Version Nir Shlezinger, Koby Todros, and Ron Dabora Department of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel Email: nirshl@post.bgu.ac.il, {todros, ron}@ee.bgu.ac.il Abstract—Adaptive filters are employed in many signal processing and communications systems. Commonly, the design and analysis of adaptive algorithms, such as the least mean-squares (LMS) algorithm, is based on the assumptions that the signals are wide-sense stationary (WSS). However, in many cases, including, for example, interference-limited wireless communications and power line communications, the considered signals are jointly cyclostationary. In this paper we propose a new LMS-type algorithm for adaptive filtering of jointly cyclostationary signals using the time-averaged mean-squared error objective. When the considered signals are jointly WSS, the proposed algorithm specializes to the standard LMS algorithm. We characterize the performance of the algorithm without assuming specific distributions on the considered signals, and derive conditions for convergence. We then evaluate the performance of the proposed algorithm, called time-averaged LMS, in a simulation study of practical channel estimation scenarios. The results show a very good agreement between the theoretical and empirical performance measures. I. I NTRODUCTION Adaptive filters play an important role in the implementation of linear estimators. A commonly employed class of adaptation algorithms is the class of least mean-squares (LMS)-type algorithms [1, Ch. 9], [2, Ch. 10]. The majority of LMStype filters are designed for linear estimation of a widesense stationary (WSS) signal of interest (SOI), based on a statistically dependent jointly WSS (JWSS) input signal. In many practical scenarios, the considered signals are nonstationary, but rather belong to the class of wide-sense cyclostationary (WSCS) signals [3, Sec. 5-7]. For example, most digitally modulated communications signals are WSCS [4, Ch. 1]. Cyclostationarity is also observed in power line communications (PLC), especially in the narrowband (NB) frequency range (0 − 500 kHz) [5]. The work [6] studied the convergence of the LMS algorithm for linear estimation of jointly wide-sense cyclostationary (JWSCS) signals when no specific model relating the input signal and the SOI is assumed. It was shown that the filter coefficients are mean convergent when the step-size of the algorithm approaches zero. The work [7] applied the LMS algorithm to the identification of a linear time-invariant (LTI) system with WSCS Gaussian input, and an output that is corrupted by additive WSS Gaussian noise. For this scenario, the transient, tracking, and the steady-state performance of the LMS algorithm were studied, and convergence conditions were derived. However, in many practical scenarios involving WSCS signals, e.g., interference-limited communications and NB-PLC, the relationship between the considered signals cannot be modeled using a single LTI system with WSS noise. This work was supported by the Ministry of Economy of Israel through the Israeli Smart Grid Consortium. The optimal linear estimator in the minimum mean-squared error (MSE) sense of a WSCS SOI based on a JWSCS input signal is the cyclic Wiener filter (CWF) [8]. The CWF achieves the minimal MSE (MMSE) at any time instant, and is realized as a linear periodically time-varying (LPTV) filter. A common structure for realizing LPTV filters is the frequency shift (FRESH) filter [8], which implements an LPTV system by summing the outputs of multiple LTI filters, each applied to frequency-shifted versions of the input signal. Adaptive FRESH filters were considered in several works, e.g., [9], [10], in which adaptive algorithms originally designed for JWSS signals, such as the standard LMS algorithm, were applied to the adaptation of FRESH filters. Any LPTV filter can be implemented using a set of LTI filters [11], with possibly infinite impulse responses. However, in practical designs, linear estimators consist of a limited number of LTI filters. Thus, in practice, when the number of LTI filters used for implementing an LPTV system is smaller than the one required for implementing the CWF, the resulting linear estimator does not achieve the MMSE. In such cases, the time-averaged MSE (TA-MSE) criterion is used instead of the instantaneous MSE [8]–[10] in order to achieve a filter with static coefficients. This motivates the development of an LMS-type algorithm for adaptive filtering for JWSCS signals based on the minimum TA-MSE (MTA-MSE) criterion. Main Contributions: In this work we develop a new LMStype algorithm for adaptive linear estimation of discrete-time (DT) signals, where the SOI and input signal are assumed to be jointly proper-complex (JPC) [12] and JWSCS [4, Ch. 1]. Our setup accommodates various filter structures for JWSCS signals, including LTI and LPTV filters. We first observe that a linear MMSE (LMMSE) estimator with time-invariant coefficients does not necessarily exist for JWSCS signals. It follows that the TA-MSE objective is typically preferable over the MSE since it leads to linear estimators with time-invariant coefficients. We then derive an adaptive algorithm based on minimizing the TA-MSE objective via stochastic approximation of the steepest descent (SD) algorithm. The new algorithm, referred to as the time-averaged LMS (TA-LMS), specializes to the standard LMS algorithm when the considered signals are JWSS. Transient and steadystate performance analysis of the proposed algorithm is carried out without imposing a specific distribution on the considered signals. Finally, the proposed TA-LMS algorithm is applied to a practical channel estimation problem in a simulation example that show a very good agreement between the theoretical and empirical performance measures. The rest of this paper is organized as follows: In Section II the considered estimation problem is stated. In Section III the new TA-LMS algorithm is derived, and in Section IV the performance analysis is carried out. In Section V the proposed algorithm is illustrated in simulation examples. Lastly, in Section VI, concluding remarks are provided. filter [11], the setup (1) can realize any scalar LPTV filter of finite dimensions. II. P RELIMINARIES AND P ROBLEM F ORMULATION Notations: We denote column vectors with lower-case boldface letters, e.g., x; the k-th element (k ≥ 0) of a vector x is denoted with (x)k . Matrices are denoted with upper-case boldface letter, e.g., X. We denote the n × n identity matrix with In and the all-zero n × m matrix with 0n×m . Hermitian transpose, transpose, complex conjugate, and stochastic expectation are denoted by (·)H , (·)T , (·)∗ , and E{·}, respectively. Let Re {x} denote the real part of the complex number x, Tr {·} denote the trace operator, ((n))m denote the remainder of n when divided by m, ⊗ denote to Kronecker product, and h·iN denote time-averaging over NP −1 x [n − k]. The sets N > 0 samples, e.g., hx[n]iN , N1 In this section we derive an adaptive linear estimator for WSCS signals based on the TA-MSE objective. This objective is motivated by the fact that when the estimator is restricted to be in the class of linear estimators (1), an LMMSE estimator may not exist for JWSCS signals [8]. The reason is that an LMMSE estimator in the class of (1) exists if and only if there exists a time-invariant vector hM M SE that satisfies  E x [n] xH [n] hM M SE = E {x [n] d∗ [n]} , (2) k=0 III. A DAPTIVE L INEAR E STIMATION OF C YCLOSTATIONARY S IGNALS BASED ON TA-MSE ∀n ∈ {0, 1, . . . , N0 − 1} , N0 . One scenario in which a time-invariant LMMSE estimator does exist, is the scenario considered in [7], in which x[n] is the input of an LTI system, and d[n] is the output of the system corrupted by a WSS noise. MTA-MSE Linear Estimator: Define the deterministic  H quantities: C [n] , E x [n] x [n] x D n D , cx [n] , vecE(Cx [n]), oE 2 , and cd , E |d [n]| , cxd , E {x [n] d∗ [n]} N0 N0 ˜ ˜ Cx , hCx [n]iN0 . Note that cd , cxd , and Cx are time-invariant ˜ ˜ while C ˜ [n] and c [n] are ˜ since d[n] and x[n] are JWSCS, x x periodic with period N0 . We assume that the following holds: of integers and non-negative integers are denoted by Z and N, respectively. For an n × n matrix X, let λmax (X) denote the largest real eigenvalue of X, given that such exists, and x = vec (X) denote the n2 × 1 column vector obtained by stacking the columns of X one below the other. The matrix X is recovered from its vectorized representation x via X = vec−1 (x). For an n×1 vector y and an n2 ×1 vector x, AS1 C is non-singular. x 2 2 let kyk denote the squared Euclidean norm yH y, and kykx ˜ The TA-MSE associated with the linear estimator (1) is denote the weighted squared Euclidean norm yHvec−1 (x) y. D n oE b 2 Lastly, for a sequence of n×n matrices {Xk }k=a , b ≥ a are H d [n]−h x[n] J (h) , E b Q N0  Xk denote the matrix product Xb Xb−1 · · · Xa . integers, let H = cd − 2Re cH (3) k=a xd h + h Cx h. ˜ ˜ ˜ Cyclostationary Stochastic Processes: A DT properTherefore, the optimal filter coefficients vector in the sense of complex (PC) multivariate process x[n] is said to be WSCS if minimum TA-MSE, denoted by hopt , must satisfy the timeboth its mean value and autocorrelation function are periodic   averaged Wiener-Hopf equations (see, e.g., [9, Eq. (5)]): with  some period, N0 , i.e., E x[n]} = E x[n + N0 ]}, and Cx hopt = cxd , (4) E x[n+l]xH[n]} = E x[n + N0 + l]xH[n + N0 ]}, see [3, Sec. ˜ ˜ 3.5]. A pair of JPC DT processes x1 [n] and x2 [n] are said to be resulting in the linear MTA-MSE (LMTA-MSE) estimator JWSCS with period N0 if each process isWSCS with period dˆopt [n] , hH opt x[n]. The following lemma states an equivaN0 and the cross-correlation function E x1 [n + l]xH [n]} is lence between the LMMSE and LMTA-MSE estimators. 2 periodic with period N0 w.r.t. n [3, Sec. 3.6.2]. Lemma 1. If a time-invariant LMMSE estimator exists, then Problem Formulation: We study the problem of linear it is uniquely given by the LMTA-MSE estimator dˆopt [n]. estimation of a scalar zero-mean SOI d[n] based on an M × 1 Proof: Applying the time-averaging operator h·iN0 to both multivariate zero-mean input signal x[n], where x[n] and d[n] are JPC and JWSCS with period N0 . Let h denote an M × 1 sides of (2) yields (4). Hence, when the conditions of Lemma time-invariant coefficients vector. The linear estimate of d[n] 1 are satisfied, the LMMSE estimator is also TA-MSE optimal. Uniqueness follows from the non-singularity of Cx , that is given by ˜ ˆ = hH x[n], d[n] n ≥ 0. (1) implies strict convexity of the TA-MSE objective (3). The property stated in Lemma 1 further motivates the use The formulation (1) accommodates a wide range of filters, of the TA-MSE as an objective function. including scalar LTI and LPTV filters. For example, when Adaptive LMS Estimator Based on TA-MSE: The prothe input signal is a scalar signal, denoted by r[n], the vector input signal x[n] is obtained by a multivariate mapping of r[n]. posed algorithm is obtained via instantaneous approximation Then, a finite impulse response (FIR) LTI filter with M taps of the SD algorithm under the TA-MSE objective function (3). is obtained by letting (x[n])k = r[n−k], k ∈ {0, 1, . . . , M − For a fixed step-size µ, the SD update equation is [2, Ch. 8.3] 1} [2, Ch. 10.5], and a scalar FRESH filter is obtained by ∂ (a) h[n+1] = h[n]−µ ∗J (h) = h[n]+µ(cxd − Cx h[n]) , (5) setting K, L ∈ N such that M = K · L, selecting a set of ∂h ˜ ˜ K−1 h=h[n] K cyclic frequencies {αk }k=0 , and letting (x[n])u·L+v = jαu (n−v) r[n−v]e , u ∈ {0, 1, . . . , K −1}, v ∈ {0, 1, . . . , L−1} where (a) follows by applying the gradient to the [9], [10]. As any DT LPTV system can be realized as a FRESH TA-MSE (3). Define the M × N0 matrix X [n] , i x [n], x [n−1], . . . , x [n−N0 +1] , and the N0 × 1 vech iT tor d [n] , √1N d [n], d [n−1], . . . , d [n−N0 +1] . As Cx 0 ˜ and cxd represent time-averaged correlations of the JWSCS ˜ signals, their corresponding instantaneous unbiased estimators are given by X[n] XH [n] and X[n] d∗ [n], respectively. As in the derivation of the standard LMS algorithm [2, Ch. 10.2], the TA-LMS algorithm is obtained from the SD (5) by replacing the time-averaged covariances with their instantaneous estimates. The resulting update equation is
h[n+1] = h[n]+µX [n] d∗ [n]−XH [n] h [n] , (6) n ≥ 0. Note that (6) specializes to the standard LMS for WSS signals when the period is N0 = 1, i.e., the signals are JWSS. √1 N0 h IV. TA-LMS P ERFORMANCE A NALYSIS We analyze the performance of the TA-LMS algorithm (6) for linear estimation of WSCS signals. A. Data Model and Assumptions Since condition (2) is not necessarily satisfied, i.e., a timeinvariant LMMSE estimator does not necessarily exist, we let hM [n] denote the possibly time-varying coefficients of the LMMSE estimator and v[n] be the resulting estimation error. The SOI d[n] and the LMMSE estimator are related via: d[n] = hH (7) M [n]x[n] + v[n]. Note that the stationary linear data model used for the analysis of the standard LMS algorithm [2, Ch. 10.2] is a special case of (7), obtained by setting N0 = 1. Since d[n] and x[n] are JWSCS with period N0 , it follows from [13, Ch. 17.5.1] that hM[n] defines an M × 1 periodic sequence , hM[n] = hM [n+N0 ], ∀n ∈ Z. Combined with the fact that d[n] and x[n] are zero-mean JPC JWSCS, it follows from (7) that v[n] is a zero-mean PC WSCS process with period N0 . Furthermore, E{x[n]v ∗ [n]} = 0M ×1 by the orthogonality principle [2, Ch. 4.2]. Similarly to the standard approach used for analyzing the LMS algorithm for JWSS signals, e.g., [1], [2], [15], we make the following assumptions: AS2 The estimation error of the LMMSE estimator, v [n1 ], and the input signal x [n2 ] are mutually independent ∀n1 , n2 , see also [2, Ch. 15.2], [15, Sec. B.2]. This is satisfied, for H example, if the SOI d[n] is of the form d[n] = gN [n]x[n] + 0 z[n] where gN0 [n] is a deterministic periodically time-varying coefficients vector and z[n] is a PC WSCS process independent of x[n]. In this case hM[n] = gN0 [n] and v[n] = z[n]. AS3 The coefficients vector h[n] is independent of the instantaneous input vector x[n], [2, Ch. 16.4].  see [1, Pg. 392], n AS4 h[n] is independent of x [k] xH [k] k=n−N0+1 . A similar assumption was used in [14]. Note that AS3-AS4 are satisfied asymptotically, when the filter has converged, since then the effect of the inputs on the filter coefficients is negligible. In addition to AS2-AS4, we make the following assumption: n
T
o AS5 The matrix B[n] , E X[n]XH [n] ⊗ X[n]XH [n] is time-invariant, hence B[n] = B. We note that a similar assumption was made in the analysis of LMS with non-Gaussian WSS inputs in [2, Eq. (24.9)]. This assumption is satisfied when, for example, x[n1 ] and x[n2 ] are mutually independent ∀n1 6= n2 and x[n] is fourth-order cyclostationary. We emphasize that AS1-AS5 are made only to facilitate performance analysis and are not needed for the derivation of the TA-LMS algorithm in Section III, which assumes only that the signals are JPC and JWSCS. B. Performance Measures We begin by defining the error measures. Denote the coefficients error vector of the TA-LMS algorithm w.r.t. the MTA-MSE filter, hopt , obtained from (4), by h̄ [n] , hopt − h [n], the error between the SOI and its TA-LMS estimate by e[n] , d [n] − hH [n] x [n], and the error between the SOI and the LMTA-MSE estimate by eopt [n] , d [n] − hH opt x [n]. In the transient performance analysis we characterize the timeevolutions of the expected coefficients error vector, E{h̄[n]}, and of the excess TA-MSE (ETA-MSE), defined as the difference between the TA-MSE at the TA-LMS output and at the LMTA-MSE estimator output, i.e., D n o n oE 2 2 ξ[n] , E |e [n]| − E |eopt [n]| . (8) N0 Convergence of the algorithm is defined as follows: Definition 1. The TA-LMS filter is said o to be convergent if n  2 lim E h̄[n] = 0M ×1 and E h̄[n] is convergent [1, Ch. n→∞ 9.4] [2, Ch. 23.4]. In the steady-state performance analysis we characterize the asymptotic behavior of a convergent filter, via the steady-state ETA-MSE, defined as ξs , lim ξ[n]. n→∞ C. Performance Analysis Transient Performance: We first characterize the timeevolutions of expected coefficients error vector and of the ETA-MSE. To that aim, let g[n] , hM [n] − hopt denote the difference between the LMMSE filter and the LMTA-MSE filter, and define g̃[n] , x [n] xH [n] g [n] N0 . (9) Additionally, define the M × M matrices Rx [n] , IM −µX [n] XH [n] , (10) n Q n n Rx [l] for k ≤ n and Lk , IM for k > n, and Lk , l=k R̃x , E {Rx [n]} = IM − µCx . (11) ˜ Lastly, define the N0 × 1 random vector v [n] ,  T √1 v [n], v [n − 1], . . . , v [n − N0 + 1] , the N02 × 1 vector N0 
cv [n, l] , vec E v [n] vH [n − l] , (12) the M 2 × 1 vectors n o pxv[n] ,vec E g̃[n]g̃H [n] +E{X∗ [n]⊗X[n]}c∗v [n, 0], (13) oH o  n n  n T , (14) zxg [n] , E h̄ [0] R̃Tx E RTx [n]⊗g̃H [n] X  n o H n T zxv[n] , cTv [n, k]E Lnn−k+1 X[n−k] ⊗XH [n] , (15) k=1 and the M 2 × M 2 matrix  (a) (16) F[n] , E RTx [n] ⊗ Rx [n] ≡ F, where (a) follows from the definition of X[n] and of Rx [n] (10) and from AS5. The time-evolution of the expected coefficients error vector is characterized in the following lemma: where B is defined in AS5. Using these definitions, we state a sufficient condition for the TA-LMS to be convergent: Lemma 2. The TA-LMS mean coefficients error is given by  n  E h̄ [n] = (IM −µCx ) E h̄ [0] , n ≥ 0. (17) ˜ [A proof is given in Appendix A] Theorem 2. Assume that H has at least one real-valued positive eigenvalue, then the TA-LMS algorithm converges if   2 1 1 0 < µ < min . (24a) , , λmax (Cx ) λmax (A−1 B) λmax (H) ˜ If H does not have any real positive eigenvalues then the TA-LMS converges  if  1 2 . (24b) , 0 < µ < min λmax (Cx ) λmax (A−1 B) ˜ [A proof is given in Appendix C] Next, we characterize the time-evolution of the ETA-MSE:   Theorem 1. Define ck , cx ((k))N0 . For all n ≥ N0 − 1, the ETA-MSE of the TA-LMS  n filter isogiven by n n o X 1 2 H ξ[n] = E h̄ [k] c +2 · Re E h̄ [0] k N0 k=n−N0+1   k     , (18) × R̃x Cx ((k))N0 g ((k))N0 o n 2 is computed via where E h̄ [k] c k n o o n n X 2 2 l−1 ck E h̄[n] c = E h̄ [0] Fn c +µ2 pH xv [n−l] F k k l=1 )  n  X H H l−1 −2µ·Re zxg [n−l]−µ·zxv [n−l] F ck , (19) ( l=1 and R̃x , pxv , zxg , zxv , and F are defined in (11), (13), (14), (15), and (16), respectively. [A proof is given in Appendix B] Convergence and Steady-State Analysis: In order to characterize the steady-state performance and derive conditions for convergence, we introduce two additional assumptions: AS6 The temporal correlation of v[n] is bounded and spans a finite interval, i.e., E{v[n+l]v ∗[n]} is bounded ∀n, l ∈ Z and ∃Lmax > 0 s.t. E{v[n+l]v ∗[n]} = 0 for all |l| ≥ Lmax . AS7 Define η , Lmax + N0 . All the 2N -th order moments of x[n], N ∈ {1, 2, . . . , η + 1}, are bounded and periodic1 2N−1 with period N0 . Specifically, for any set of pairs {ki , ni }i=0 where N−1 ki ∈ {0, 1, . . . , M − 1} and  ni ∈ N, we assume that Q ∗ E (x [n2i ])k2i (x [n2i+1 ])k2i+1 is bounded and equal to i=0 N−1  Q ∗ E (x [n2i +N0 ])k2i (x [n2i+1 +N0 ])k2i+1 . This is satisfied i=0 when, e.g., x[n] is PC Gaussian WSCS with period N0 . Next, define the M 2 × 1 vector X η n T oH zsxv[n] , cTv [n, k]E Lnn−k+1 X[n−k] ⊗XH [n] , (20) k=1 where η is defined in AS7, the M 2 × M 2 matrices
A , CTx ⊗IM + (IM ⊗ Cx ) , (21) ˜ ˜ N0−1 X N0    1 X cx [k] pxv ((k−l))N0 P, N0 k=0 l=1  H l−1  F , (22) +2zsxv ((k−l))N0 and the 2M 2 × 2M 2 matrix   1 A −B , (23) H, 2 2IM 2 0M 2 ×M 2 1 Note that in the analysis of the LMS algorithm for non-Gaussian WSS inputs it was assumed that the fourth-order moments of the input are timeinvariant [2, Eq. (24.9)] and bounded [2, Pg. 361]. The steady-state ETA-MSE of the TA-LMS algorithm is explicitly stated in the following: Theorem 3. When (24) is satisfied, the steady-state ETA-MSE of the TA-LMS algorithm n isngiven by
−1 oo 2 , (25) ξs = µ · Re Tr P IM 2 − FN0 where F and P are defined in (16) and (22), respectively. [A proof is given in Appendix D] It can be shown that when N0 = 1, i.e., the considered signals are JWSS, and v[n] is temporally uncorrelated, then, the TA-LMS specializes to the standard LMS, and (25) specializes to the excess MSE of the standard LMS algorithm with non-Gaussian inputs stated in [2, Thm. 24.1]. V. N UMERICAL E XAMPLES In this section we evaluate the performance of the TA-LMS algorithm and demonstrate the theoretical results presented in Section IV for two scenarios: SN1, a scenario which satisfies AS1-AS7; and SN2, a practical NB-PLC channel estimation scenario. In scenario SN1 we let φ[n] be an i.i.d. process, uniformly   distributed over [0, 2π), and set r[n] = 1 + 0.5 sin 2πn ejφ[n] , with period N0 = 20. The vector N0 x[n] is constructed from r[n] by setting (x[n])k = r[n]ejαk n , 2π . The resulting k ∈ {0, 1, 2}, {α0 , α1 , α2 } = {−1, 0, 1} · N 0 coefficients vector h[n] corresponds to a FRESH filter with K = 3 branches, where each branch has L = 1 taps. Hence, the dimension of the coefficients vector is M = 3. Furthermore, we  set hM [n]  inthe data model (7) to satisfy (hM [n])k = 1+0.5 sin 2πn e−|k+1| . The process v[n] in (7) is set to N0 be a WSCS temporally uncorrelated Gaussian process,   inde pendent of x[n], with variance σv2 [n] = γv 1+0.5 sin 2πn , N0 where γv is set to achieve a signal-to-noise ratio (SNR) of 7 hE{|d[n]−v[n]|2 }iN0 . It can be dB, defined here as SNR , hE{|v[n]|2 }iN0 shown that for this setting AS1-AS7 are satisfied. In scenario SN2 we study a practical channel estimation scenario. We consider a scalar NB-PLC channel with input signal r[n] and output signal d[n], which is the SOI in this scenario. The signal r[n] is set to be a passband OFDM signal with 36 subcarriers, each modulated with a QPSK constellation, with 4 cyclic prefix samples. As shown in [10, Sec. II.C], r[n] is WSCS with period N0 = 40. The signal r[n] is transmitted over an NB-PLC channel, modeled via a real 4 40 SN1, Empirical SN1, Theoretical SN2, Empirical SN2, Theoretical −2 2 −2.5 1 −3 100 120 140 0 −1 −2 SN1, Empirical SN1, Theoretical SN2, Empirical SN2, Theoretical 35 30 Steady−state TAMSE [dB] TA−MSE [dB] 3 25 −2 20 −3 15 SN1 Stability bound 10 −4 0.2 5 0.4 0.6 0.8 SN2 Stability bound 0 −3 −5 −4 50 100 150 200 Iteration 250 300 350 −10 0 0.2 0.4 0.6 0.8 Step−size 1 1.2 1.4 1.6 Fig. 1. TA-MSE comparison for scenarios SN1 and SN2. Fig. 2. Steady-state TA-MSE comparison for scenarios SN1 and SN2. LPTV channel transfer function (CTF) g[n, l] with additive real WSCS Gaussian noise w[n], both with period N0 . Hence, ∞ P g[n, l]r[n − l] + w[n], see [5, Sec. III.C]. Here, d[n] = the signals are JWSS the proposed algorithm specializes to the standard LMS algorithm. The simulation study shows that the theoretical analysis reliably characterizes the empirical performance of the algorithm. A PPENDIX A. Proof of Lemma 2 It follows from the TA-LMS update equation (6) that
h̄[n+1] = h̄[n]−µ X [n] d∗ [n] − X [n] XH [n] h [n] . (26) l=0 the estimator (1) represents the output of a FRESH filter whose input is r[n], with K = 3 branches corresponding 2π to frequency shifts {α0 , α1 , α2 } = {−1, 0, 1} · N . At each 0 branch there is an FIR filter with L = 3 taps, and the overall dimensionality is thus M = 9. The multivariate input signal x[n] is constructed from r[n] according to the description below (1). Following [5], we generate the LPTV CTF g[n, l], n ∈ N0 , using the channel generator proposed in [16] adapted to NB-PLC by setting the parameters as detailed in [5, Sec. V]. The noise w[n] is simulated based on the model adopted by the IEEE P1901.2 standard [17], with a set of typical parameters which corresponds to low voltage site 11 (LV11) in [17, Appendix G]. The MMSE filter hM [n] in (7) is obtained from the orthogonality principle and v[n] is obtained as the estimation error of hM [n]. Here, the SNR is defined hE{|d[n]−w[n]|2 }iN0 . As NB-PLC channels are as SNR , hE{|w[n]|2 }iN0 typically characterized by low SNR, we chose SNR = 0dB. Fig. 1 depicts the theoretical TA-MSE for both scenarios and for step-size µ = 0.01. We note that in both scenarios there is an excellent agreement between the theoretical and the empirical TA-MSEs. In Fig. 2, the theoretical steadystate TA-MSE is compared to its empirical value for various step-sizes and with the theoretical stability threshold, obtained from (24). Observing Fig. 2, we note that in both scenarios the theoretical steady-state TA-MSE provides an accurate characterization of the empirical steady-state TA-MSE. It is also noted that when AS1-AS7 are satisfied, the stability of the algorithm is accurately predicted using the step-size region in (24), however, in scenario SN2 there is a gap between the theoretical stability threshold and its empirical measurement, which is due to the fact that, unlike scenario SN1, assumptions AS1-AS7 are not satisfied here. VI. C ONCLUSIONS In this work, a new adaptive algorithm for linear estimation of JWSCS signals, based on the TA-MSE objective is introduced and its performance and stability are characterized without imposing specific distribution on the signals. When Note that for n ≥ 0 (a) X [n] d∗ [n] = N0−1 1 X x [n−k] xH [n−k] (hopt +g [n−k]) N0 + (b) k=0 N 0−1 X 1 N0 x [n−k] v ∗ [n−k] k=0 = X [n] XH [n] hopt +g̃ [n]+X [n] v∗ [n] , (27) where (a) follows by plugging (7) into the definition of d [n] and using the definition of g[n], and (b) follows from the definition of g̃[n] (9). Plugging (27) into (26) yields     h̄[n+1] = IM −µX[n]XH [n] h̄[n]−µ g̃[n]+X[n]v∗ [n] . (28) Next, we show that E{g̃ [n]} = 0M ×1 , ∀n ∈ Z. Since v[n] is zero-mean and independent of x[n] by AS2, applying the Hermitian transpose to (7), followed by left multiplication of both sides by x[n], and then applying the stochastic expectation and using AS2 yields  E {x [n] d∗ [n]} = E x [n] xH [n] (hopt + g[n]) . (29) Applying the time-averaging operator to (29) and recalling the definition of g̃[n] (9) results in cxd = Cx hopt + E{g̃ [n]} = ˜ optimal ˜ filter, it follows that 0M ×1 . Since hopt is the TA-MSE E{g̃ [n]} = 0M ×1 . As also E{X [n]v∗ [n]} = 0M ×1 by AS2, applying the stochastic expectation to (28) yields the recursion   E h̄[n] = (IM −µCx ) E h̄ [n − 1] . (30) ˜ Repeating the recursion in (30) n times yields (17). B. Proof of Theorem 1 We begin by stating the following equalities and properties that will be used in the sequel (see [18, Ch. 9.2]): R1 For any matrix triplet A1 , A2 , A3 of compatible dimensions, it holds that
vec (A1 A2 A3 ) = AT3 ⊗ A1 vec (A2 ) . (31) R2 For any pair of square matrices A1 , A2 of identical dimensions, it holds that  T Tr AT1 A2 = vec (A1 ) vec (A2 ) . (32) R3 For any pair of matrices A1 , B1 , it holds that T (A1 ⊗ B1 ) = AT1 ⊗ BT1 , (33) R4 For any four matrices A1 , A2 , B1 , B2 of compatible dimensions, it holds that A1 A2 ⊗ B1 B2 = (A1 ⊗ B1 ) (A2 ⊗ B2 ) . (34) In order to prove the theorem, we first derive a recursive relationship for the mean of the weighted squared Euclidean norm of h̄ [n], which is stated in the following proposition: Proposition 1. For any M 2 × 1 vector q such that Q = vec−1 {q} is Hermitian positive semi-definite, the coefficients error vector h̄[n] satisfies the following recursion for n ≥ 0: o o n n 2 2 E h̄ [n+1] q = E h̄ [n] Fq +µ2 pH xv [n]q  
H −2µ · Re zH [n]−µ · z [n] q , (35) xg xv where F, pxv , zxg , and zxv are defined in (16), (13), (14), and (15), respectively. Proof: Let Q be the M × M Hermitian matrix obtained via Q , vec−1 (q). Note that h̄ [n+1] (a) H 2 q H , h̄ [n+1] Q h̄ [n+1] 2 = h̄ [n] Rx [n]QRx [n]h̄ [n]+µ2 ·kg̃ [n]+X [n] v∗ [n]kq n  o −2µ·Re vT [n] XH + g̃H [n] QRx [n]h̄ [n] , (36) where F is defined in (16). Therefore, o n H E h̄ [n] Rx [n]QRx [n]h̄ [n] n n oo (a) H = E E h̄ [n] Rx [n]QRx [n]h̄ [n] h̄ [n] o n (b) H = E h̄ [n]E {Rx [n]QRx [n]} h̄ [n] n o (c) 2 = E h̄ [n] Fq , (39) where (a) follows from the law of total expectation [21, Ch. 7.4]; (b) follows from the independence assumption AS4; and (c) follows from (38) and the definition of the weighted Euclidean norm. Next, note that o n n o o n 2 E kg̃ [n]kq = Tr E g̃ [n] g̃H [n] Q   n oH (a) = vec E g̃ [n] g̃H [n] q, (40) where (a) follows  n o∗  vec E g̃ [n] g̃H [n] and since (32)n oT  vec E g̃ [n] g̃H [n] . from = Similarly, o n 2 E kX [n] v∗ [n]kq n  o = Tr E v∗ [n] vT [n] XH [n] QX [n] n  o  (a) = Tr E v∗ [n] vT [n] E XH [n] QX [n]     T  (b) = vec E v [n] vH [n] vec E XH [n] QX [n]  (c) T = cv [n, 0] E XT [n] ⊗ XH [n] q  n o H (d) q, (41) = E X∗ [n] ⊗ X [n] c∗v [n, 0] where (a) follows as v[n] is independent of X[n] by AS2; (b) follows from property R2; (c) follows from property R1 and from the definition of cv [n, l] in (12); (d) follows since conjugate transposition is distributive over the Kronecker product [18, Ch. 9.2]. It follows from plugging definition (13) into (40) and (41) That o o n n 2 2 (42) E kg̃ [n]kq + E kX [n] v∗ [n]kq = pH xv [n]q. where (a) follows from the relationship (28). Since v[n] is zero-mean and independent of X[n] by AS2, applying the In order to obtain the third element in (37), we note that stochastic expectation to (36) results in repeating the recursion in (28) n times for n ≥ 1 yields n o n o 2 H E h̄ [n+1] q = E h̄ [n] Rx [n]QRx [n]h̄ [n] h̄[n] = L0n−1 h̄ [0] o o n n n   X 2 2 n−1 +µ2 ·E kg̃ [n]kq +µ2 E kX [n] v∗ [n]kq −µ X [n−k] v∗ [n − k]+g̃ [n−k] . Ln−k+1 n  o k=1 −2µ·Re E vT [n] XH [n] QRx [n]h̄ [n] n n oo From the definition of Lnk it follows that Rx [n]Lkn−1 = Lnk , H −2µ·Re E g̃ [n] QRx [n]h̄ [n] . (37) ∀k ≤ n, it follows that   Next, we explicitly compute each of the elements in the right E vT [n] XH [n] QRx [n]h̄ [n] = E vT [n] XH [n] QLn0 h̄ [0]   n hand side (RHS) of (37). First, from (31) it follows that X T H n ∗   n o −µ E v [n] X [n] QLn−k+1 X [n−k] v [n−k] vec E {Rx [n]QRx [n]} = E vec (Rx [n]QRx [n]) k=1  n X  = E RTx [n] ⊗ Rx [n] q E vT [n] XH [n] QLnn−k+1 g̃ [n−k] . (43) −µ = Fq, (38) k=1 The first and the last elements in the RHS of (43) are zero n since v[n] is zero-mean and independent of {X[l]}l=0 by AS2. As for the second element, we note that  E vT [n] XH [n] QLnn−k+1 X [n−k] v∗ [n−k]   (a) = Tr E v∗ [n−k] vT [n]   × E XH [n] QLnn−k+1 X [n−k]  T  (b) = Tr E v [n] vH [n−k]   H n × E X [n] QLn−k+1 X [n−k] (c) 
= cTv [n, k] vec E XH [n] QLnn−k+1 X [n−k] n o
T (d) T = cv [n, k] E Lnn−k+1 X [n−k] ⊗XH [n] q, (44) where (a) follows since v[n] and X[n] are mutually independent processes by AS2; (b) follows since   T  E v∗ [n−k] vT [n] = E v [n] vH [n−k] ; (c) follows from property R2; (d) follows from property R1. Plugging (44) into (43) yields  E vT [n] XH [n] QRx [n]h̄ [n] n o n T X = −µ cTv [n, k] E Lnn−k+1 X [n−k] ⊗XH [n] q k=1 (a) = −µ · zH xv [n]q, (45) where (a) follows from (15). Lastly, we write n o E g̃H [n] QRx [n]h̄ [n] o n  (a) = E g̃H [n] QRx [n]E h̄ [n] n o 
T (b) = E Rx [n]E h̄ [n] ⊗g̃H [n] q oT  n 
(c) q = E Rx [n] E h̄ [n] ⊗g̃∗ [n]   T  (d) = E {Rx [n]⊗g̃∗ [n]} E h̄ [n] q  n o   T n  (e) R̃x E h̄ [0] = E RTx [n]⊗g̃H [n] q (f ) = zH xg [n] q, H (46) (47a) and eopt [n] = gH [n] x [n] + v [n] . (48) where (a) follows since by AS2, x[n] is zero-mean and independent of v[n], and by AS3, x[n] is also independent of h̄[n]. The ETA-MSE is obtained by time-averaging (48), which results in    2 H ξ[n] = E h̄ [n] x [n] N0 D n oE  H . (49) + 2Re E h̄ [n] x [n] xH [n] g [n] N0 Note that the first summand in the RHS of (49) satisfies   oo n n 2 (a) H H = E E h̄ [n]x[n] xH [n]h̄[n] h̄[n] E h̄ [n] x [n] n o  (b) H = E h̄ [n] E x[n] xH [n] h̄[n] o n (c) H = E h̄ [n] Cx [n] h̄ [n] o n (d) 2 (50) = E h̄ [n] c [n] , x where (a) follows from the law of total expectation [21, Ch. 7.4]; (b) follows from the independence assumption AS3; (c) follows from the definition of Cx [n]; and (d) follows from the definition of cx [n]. Next, we note that o n H E h̄ [n] x [n] xH [n] g [n] o  n (a) H = E h̄ [n] E x [n] xH [n] g [n] n o  n (b) H = E h̄ [0] R̃x Cx [n] g [n] , (51) where (a) follows from the independence assumption AS4; (b) follows from property R1; (c) follows from property R3; (d) follows from property R4 by writing g̃∗ [n] = g̃∗ [n] · 1; (e) follows from Lemma 2; (f ) follows from definition (14). Plugging (39), (42), (45) and (46) into (37) yields (35). It follows from (47) that applying the recursion (35) n times with q = ck yields (19). Next, we prove (18). Plugging the relationship (7) into the definitions of e[n] and eopt [n] results in e [n] = h̄ [n] x [n] + gH [n] x [n] + v [n] , o o n n 2 2 E |e[n]| −E |eopt [n]|   2
H =E h̄ [n]+g [n] x [n] n n oo
H + 2Re E h̄ [n]+g [n] x [n] v ∗ [n] o n   2 − 2Re E gH [n]x [n] v ∗ [n] − E gH [n]x [n]   n o 2
H (a) 2 =E , h̄ [n]+g [n] x [n] −E gH [n]x [n] (47b) where (a) follows from the independence assumption AS3, and (b) follows from Lemma 2 and from the definition of Cx [n]. Plugging (50) and (51) into (49) for n ≥ N0 − 1 results in oE D n 2 ξ[n] = E h̄ [n] c [n] x N0 D n o  n E  H E h̄ [0] R̃x Cx [n] g [n] +2Re N0 n X   1 2 E h̄ [k] c [((k)) ] x N0 N0 k=n−N0 +1  n ! o k     H , +2Re E h̄ [0] R̃x Cx ((k))N0 g ((k))N0 (a) = (52) where (a) follows since cx [n], Cx [n], and g[n] are all periodic with period N0 . C. Proof of Theorem 2 We begin by noting some properties satisfied by the quantities defined in Subsection IV-C, stated in the following lemma: zsxv [n], Lemma 3. The random vectors zxv [n], and pxv [n], defined in (15), (20), and (13), respectively, satisfy: P1 zxv [n] and zsxv [n] are equal ∀n ≥ η, where η is defined in AS7. P2 pxv [n] and zsxv [n] are periodic with period N0 . Proof: Property P1 follows from AS6 since cv [n, l] defined in (12) satisfies cv [n, l] = 0N02 ×1 , ∀|l| > η, where η is defined in AS7. Therefore, it follows that zxv [n] and zsxv [n] defined in (15) and (20), respectively, are equal ∀n ≥ η. Lastly, from the cyclostationarity of v[n] combined with AS7 it follows that pxv [n] and zsxv[n], defined in (13) and (20), respectively, are periodic with period N0 This proves P2.  We now show that lim E h̄[n] = 0M ×1 if and only if n→∞ 0 < µ < λmax2(Cx ) . Note that Cx is obtained as the average of ˜ N0 Hermitian ˜and positive semi-definite covariance matrices. Thus, Cx is also Hermitian positive semi-definite, and its M ˜ eigenvalues {λk }k=1 are all real-valued and non-negative. From AS1 it follows that the eigenvalues of Cx are strictly ˜ Eh̄[n] = positive. From (11) and (17) it follows that lim  n→∞ n 0M ×1 for every h[0] if and only if lim R̃x = 0M ×M . n→∞ M Since R̃x = IM − µCx , its eigenvalues are {1−µ · λk }k=1 , where λk > 0, ∀k˜ ∈ {1, 2, . . . , M }, we conclude that the TA-LMS algorithm is mean convergent if and only if |1−µ · λk | < 1 for all k ∈ {1, 2, . . . , M } [20, Ch. 7.10], which occurs if and only if 0 < µ < λmax2(Cx ) . n o ˜ 2 Next, we show that if (24) is satisfied, then E h̄[n] is convergent. For n ≥ N0 , repeating the recursion (35) N0 times yields n o n o 2 2 E h̄ [n] q = E h̄ [n−N0 ] FN0 q +µ2 · N0 X l−1 pH q xv [n−l] F −2µ·Re l=1  l−1 H [n−l] F q , [n−l]−µ·z zH xv xg  where pxv [n], F, zxg [n] and zxv [n] are defined in (13), (16), (14), and (15), respectively. Define ztxv [n] , zxv [n] − zsxv [n], where zsxv [n] is defined in (20), it follows that n o n o 2 2 E h̄ [n] q = E h̄ [n−N0 ] FN0 q (N ) 0   X H H 2 s l−1 pxv [n−l]+2 (zxv [n−l]) +µ ·Re F q l=1 (N ) 0  X
H l−1 H t zxg [n−l]−µ zxv [n−l] F q . (53) −2µ·Re l=1 Define h̄k [n] , h̄[n · N0 + k], ak , X N0  l=1 F̄ , FN0 ,   pxv ((k−l))N0 +2zsxv and bk [n] , X N0   (54b) ((k−l))N0  H F l−1 H , (54c) zH xg [(n+1) N0 +k−l] l=1 − µ ztxv [(n+1) N0 +k−l]
H  Fl−1 H . (54d) From AS6-AS7 it follows that ak and bk [n] are bounded ∀k ∈ s N0 . From P2 in Lemma 3 it follows that pxv [n] and  zxv [n] are periodic with period N0, hence, pxv [n] = pxv ((n))N0  and zsxv [n] = zsxv ((n))N0 , hence, (53) can be written as n o n o  2 2 E h̄k [n+1] q = E h̄k [n] F̄q +µ2 ·Re aH k q  H − 2µ·Re bk [n] q . (55) The deviation (MSD) is obtained from o n mean-square 2 E h̄ [n] q by setting q = vec−1 (IM ). Following [2, Ch. 24.2], we use (55) to formulate M 2 state-space recursions for each k ∈ N0 as follows: o o n n  2 2 l E h̄k [n+1] F̄l q = E h̄k [n] F̄l+1 q +µ2 ·Re aH k F̄ q  l (56) −2µ·Re bH k [n] F̄ q , M 2 −1 l ∈ {0, 1, . . . , M 2 − 1}. Let {αl }l=0 be the coefficients of the characteristic polynomial of F̄ [20, Pg. 492]. It follows from the Cayley-Hamilton theorem [20, Pg. 532] that 2 MP −1 2 αl F̄l . Hence, it follows from the linearF̄M = − l=0 ity of the weighted Euclidean norm [2, Eq. (23.31)] that 2 n n o o MP −1 2 2 E h̄k [n] F̄M 2 q = − αl E h̄k [n] F̄l q . Therefore, by l=0 l=1 X N0  o n 2 E h̄k [n] converges to a fixed and finite value for n → ∞, ∀k ∈ N0 . To that aim, define for n ≥ 0 (54a) k ∈ N0 . We will show that if (24) is satisfied, then 2 defining the M k [n], ak , and bk [n], such that n × 1 vectors o h̄
˜ ˜ 2 l ˜ h̄k [n] l , E h̄k [n] F̄l q , (ak )l , aH k F̄ q, and (bk [n])l , ˜ ˜2 ˜ l 2 2 bH k [n] F̄ q, l ∈ {0, 1, . . . , M −1}, and the M × M matrix   0 1 0 0 0  0  0 1 0 0    0  0 0 1 0   F̄ ,  . , .. ..  ˜  .    0  0 0 0 1 −α0 −α1 −α2 −α3 . . . −αM 2 −1 the state-space recursions (56) can be written as a set of N0 multivariate difference equations h̄k [n + 1] = F̄h̄k [n] + µ2 ·Re {ak } − 2µ·Re {bk [n]} , (57) ˜ ˜˜ ˜ ˜ k ∈ N0 , n ≥ 0. Note that ∀k ∈ N0 , (57) represents an M 2 × M 2 multivariate LTI system with input signal µ2 Re {ak } − ˜ 2µRe {bk [n]} and output signal h̄k [n]. We denote the impulse ˜ ˜ response sequence matrix of the LTI system2 corresponding 2 2 ton(57) by the o M × M matrix function G[n]. Note that 2 E h̄k [n] is convergent if and only if h̄k [n] is bounded ˜ and tends to a steady-state value [2, Ch. 23.4]. In order to analyze the conditions for steady-state convergence of h̄k [n], a b a b ˜ we write h̄k [n] = h̄k [n] + h̄k [n], where h̄k [n] and h̄k [n] are ˜ of the ˜system when ˜ the outputs the input˜ signal is ˜µ2 ak and ˜ the −2µRe {bk [n]}, respectively. In the following we study a b ˜ conditions for steady-state convergence of h̄k [n] and h̄k [n]: ˜ ˜ a a Convergence of h̄k [n]: Note that h̄k [n] is obtained from ˜ ˜ equation, for which a first-order non-homogenous difference a it follows from [22, Pg. 193] that h̄k [n] remains bounded ˜ if and only if all the and converges to a steady-state value eigenvalues of F̄ are inside the unit circle, which in turn is ˜ equivalent to BIBO stability of the multivariate LTI system with impulse response sequence matrix G[n] [23, Ch. 5.2]. b Convergence of h̄k [n]: Let ⋆ denote the convolution operator and k·k1 be ˜the l1 norm, i.e., for a vector y ∈ CN , NP −1 |(y)l |, and for a matrix Y ∈ CN1 ×N2 , kYk1 = kyk1 = l=0 NP 2 −1 1 −1 NP lim hbk [n] = n→∞ ˜ l1 =0 l2 =0 0M 2 ×1 when the multivariate LTI system with impulse response sequence matrix G[n] is BIBO stable and µ < 2 λmax(Cx ) . This is achieved by showing that ∀ǫ > 0, ∃n0 (ǫ) such ˜that ∀n > n0 (ǫ), hbk [n] 1 < ǫ. Note that the multivari˜ ate LTI system with impulse response sequence matrix G[n] ∞ P kG [l]k1 < ∞ [23, is BIBO stable if and only if γ , (Y)l1 ,l2 . We next prove that l=−∞ Thm. 5.MD1]. From (54d) we note that for µ < lim n→∞ bH k [n] = lim n→∞ N0  X zH xg [(n+1) N0 +k−l] l=1 − µ ztxv [(n+1) N0 +k−l] (a) = lim n→∞ N0 X l=1 2 λmax(Cx ) ˜ (b)
H  Fl−1 l−1 zH = 0M 2 ×1 , (58) xg [(n+1) N0 +k−l] F where (a) follows since ztxv [n] = zxv [n] − zsxv [n] = 0M 2 ×1 for all n ≥ η by property P1 in Lemma 3; (b) follows since for µ < λmax2(Cx ) , it follows from the definition of zxg [n] (14) that ˜ lim zxg [n] = 0M 2 ×1 . It follows that lim bk [n] = 0M 2 ×1 , n→∞ n→∞ ˜ and thus ∀ǫ̃ > 0, ∃ñ0 (ǫ̃) such that ∀n > ñ0 (ǫ̃), kbk [n]k1 < ǫ̃. ˜ ǫ Let ǫ̃ = 2γ and define ( bk [n] 0 ≤ n ≤ ñ0 (ǫ̃) L bk [n] , ˜ , 0M 2 ×1 n > ñ0 (ǫ̃), n < 0 ˜ ( 0M 2 ×1 n ≤ ñ0 (ǫ̃) R bk [n] , . bk [n] n > ñ0 (ǫ̃) ˜ ˜ 2 Following [23, Ch. 5], G[n] is said to be the impulse response sequence matrix of an M 2 × M 2 multivariate LTI system with M 2 × 1 input sequence xG [n] and M 2 ×1 output sequence yG [n], if its the input-output relationship ∞ P satisfies yG [n] = G[l]xG [n − l]. l=−∞ L R b Clearly, bk [n] = bL k [n]+ bk [n] and h̄k [n] = G[n]⋆ bk [n]+ ˜ ˜ ˜ ˜ ˜ R G[n] ⋆ bk [n], thus ˜ b R h̄k [n] ≤ G[n] ⋆ bL k [n] 1 + G[n] ⋆ bk [n] 1 . (59) 1 ˜ ˜ ˜ Next, note that bL [n] is bounded and its support is finite, ˜ k summable. If the multivariate LTI system hence it is absolutely with impulse response sequence matrix G[n] is BIBO stable then G[n] is absolutely summable [23, Ch. 5.2], which implies that G[n]⋆ bL [n] is also absolutely summable [24, Proposition ˜k 10.3]. It follows that lim G [n] ⋆ bL [n] = 0M 2 ×1 , thus n→∞ ˜k ∃n1 (ǫ) such that ∀n > n1 (ǫ), ǫ G[n] ⋆ bL (60) k [n] 1 < . 2 ˜ As for the term G [n] ⋆ bR [n], observe that if the multivariate ˜ k response sequence matrix G[n] is LTI system with impulse BIBO stable then ∞ X G [n] ∗ bR G [n − l] bR k [n] 1 = k [l] ˜ ˜ l=−∞ 1 (a) = ∞ X l=ñ0 (ǫ̃) (b) ≤ ∞ X l=ñ0 (ǫ̃) (c) ≤ γ · ǫ̃ ≤ G [n − l] bk [l] ˜ 1 kG [n − l]k1 kbk [l]k1 ˜ ǫ , 2 (61) where (a) follows from the definition of bR [n]; (b) fol˜ k ∀n > ñ (ǫ̃), lows from [20, Eq. (5.25)]; (c) follows since 0 kbk [n]k1 < ǫ̃, n and as a consequence of BIBO stability. Setting o ˜ ǫ ), n1 (ǫ) , then by plugging (60) and n0 (ǫ) = max ñ0 ( 2γ (61) into (59) it follows that ∀n > n0 (ǫ), hbk [n] 1 < ǫ, ˜ hence lim hbk [n] = 0M 2 ×1 . n→∞ ˜ So far we have shown that if 0 < µ < λmax2(Cx ) and all the ˜ eigenvalues of F̄ are inside the unit circle (which implies that ˜ the multivariate LTI system with impulse response sequence matrix G[n] is BIBO stable), then h̄[n] is mean-square stable. We now show that the latter condition is equivalent to the constraints on A−1 B and H in (24). Note that it follows from [2, Pg. 346] that the eigenvalues of F̄ are the eigenvalues of F̄. Also note that from (54b) it follows˜that all the eigenvalues of F̄ are inside the unit circle if and only if all the eigenvalues of F are inside the unit circle. Since Cx is Hermitian and positive-definite by AS1, it follows from ˜[20, Pg. 598] that A, defined in (21), is also Hermitian positive-definite. We now show that B, defined in AS5, is a positive semi-definite matrix. 2 Note that ∀q̃ ∈ CM and Q̃ = vec−1 (q̃), n
T
o q̃H Bq̃ = q̃HE X [n] XH [n] ⊗ X [n] XH [n] q̃  n o (a) H = q̃ vec E X [n] XH [n] Q̃X [n] XH [n] n n oo (b) = Tr E Q̃HX [n] XH [n] Q̃X [n] XH [n] n n oo = Tr E XH [n] Q̃HX [n] XH [n] Q̃X [n] , (62) wheren (a) follows from (31) and ob follows from (32). As E XH [n] Q̃HX [n] XH [n] Q̃X [n] is Hermitian positive semi-definite, (62) is non-negative, thus B is positive semidefinite. Recall that F = IM 2 − µA + µ2 B, where µ > 0, A is positive-definite, and B is positive semi-definite and finite (by AS7). It therefore follows from [19, Appendix A] that when H has at least one real-valued positive eigenvalue then the eigenvalues n of F are guaranteed oto be inside the unit 1 1 , and when H has circle if µ < min λmax(A −1 B) , λ max(H) no real-valued positive eigenvalues, the eigenvalues of F are 1 guaranteed to be inside the unit circle if µ < λmax(A −1 B) . 2 Combining this with the condition 0 < µ < λmax(Cx ) yields ˜ (24). D. Proof of Theorem 3 In order to derive the steady-state ETA-MSE, we first show that it can be obtained as the following limit: Lemma 4. The steady-state ETA-MSE can be written as D n oE 2 ξs = lim E h̄ [n] cx [n] . (63) n→∞ N0 Proof: Note that in steady-state oE D n H lim E h̄ [n] x [n] xH [n] g [n] n→∞ N0 o  E D n (a) H H = lim E h̄ [n] E x [n] x [n] g [n] n→∞ (b) N0 = 0, from Def. 1, as where (a) follows from AS3, and (b) follows  the filter is in steady-state, and as E x [n] xH [n] is finite ∀n ∈ Z. The steady-state ETA-MSE, obtained by taking n → ∞ in (49), is therefore given by    2 H . (64) ξs = lim E h̄ [n] x [n] n→∞ N0 Plugging (50) into (64) yields (63). Assuming that µ satisfies (24), by Lemma 4 the steady-state ETA-MSE is given by D n oE 2 ξs = lim E h̄ [m] cx [m] m→∞ = (a) = 1 N0 1 N0 NX 0 −1 N0 lim E n h̄ [m − k] lim E n h̄k [n] m→∞ k=0 NX 0 −1 k=0 n→∞ 2 cx [m−k] 2 cx [k] o , o (65) where (a) follows from the definition of h̄k [n]  in (54a), the periodicity of cx [n], i.e., cx [n] = cx ((n))N0 , and from the fact that when the limit exists then any subsequence converges to the limit [25, Def. 3.5]. Next, recalling the definitions of ak , bk [n], and F̄ stated in (54), it follows from (55) that ∀k ∈ N0 in the steady-state it holds that o o n n  2 2 lim E h̄k [n] q = lim E h̄k [n] F̄q +µ2 ·Re aH k q n→∞ n→∞  H − 2µ· lim Re bk [n] q n→∞ n o  (a) 2 = lim E h̄k [n] F̄q +µ2 ·Re aH k q , n→∞ guarantees where (a) follows from n o (58). Thus,nsince (24) o 2 2 that lim E h̄k [n] q and lim E h̄k [n] F̄q both exn→∞ n→∞ ist and are finite, then using the linearity of the weighted Euclidean norm [2, Eq. (23.31)] we have o n  2 (66) lim E h̄k [n] (I 2 −F̄)q = µ2 ·Re aH k q , n→∞ M
−1 cx [k] in (66) results in ∀k ∈ N0 . Setting q = IM 2 − F̄ n o n o
−1 2 lim E h̄k [n] c [k] = µ2·Re aH cx [k] . (67) k IM 2 − F̄ n→∞ x Plugging (67) into (65) results in N0 −1 o n
−1 1 X 2 − F̄ c [k] Re aH I x M k N0 k=0 ( ) N0 −1 n
−1 o 1 X 2 H = µ Re Tr cx [k]ak IM 2 − F̄ N0 k=0 )) ! ( ( N0 −1
−1 1 X (a) 2 H cx [k]ak IM 2 − F̄ = µ Re Tr N0 k=0 n n
−1 oo (b) 2 , (68) = µ Re Tr P IM 2 − FN0 ξs = µ2 (a) follows from the linearity of the trace operator [18, Ch, 1.1]; (b) follows from the definitions of ak , F̄, and P in (54c), (54b), and (22), respectively. R EFERENCES [1] S. Haykin. Adaptive Filter Theory. Prentice Hall, 2003. [2] A. H. Sayed. Adaptive Filters. IEEE Press, 2008. [3] W. A. Gardner, A. Napolitano, and L. Paura. “Cyclostationarity: Half a century of research”. Signal Process., vol. 86, Apr. 2006, pp. 639-697. [4] W. A. Gardner (Editor). Cyclostationarity in Communications and Signal Processing. IEEE Press, 1994. [5] N. Shlezinger and R. Dabora. “On the capacity of narrowband PLC channels”. IEEE Trans. Commun., vol. 63, no. 4, Apr. 2015, pp. 1191 - 1201. [6] D. C. McLernon. “Analysis of LMS algorithm with inputs from cyclostationary random processes”. Electronics Letters, vol. 27, no. 2, Jan. 1991, pp. 136–138. [7] N. J. Bershad, E. Eweda, and J. C. M. Bermudez. “Stochastic analysis of the LMS and NLMS algorithms for cyclostationary white Gaussian inputs”. IEEE Trans. Signal Process., vol. 62, no. 9, May 2014, pp. 2238-2249. [8] W. A. Gardner. “Cyclic Wiener filtering: theory and method”. IEEE Trans. Commun., vol. 41, no. 1, Jan. 1993, pp. 151-163. [9] O. A. Yeste-Ojeda and J. Grajal. “Adaptive-FRESH filters for compensation of cycle-frequency errors”. IEEE Trans. Signal Process., vol. 58, no. 1, Jan. 2010, pp. 1-10. [10] N. Shlezinger and R. Dabora. “Frequency-shift filtering for OFDM signal recovery in narrowband power line communications”. IEEE Trans. Commun., vol. 62, no. 4, Apr. 2014, pp. 1283-1295. [11] D. C. McLernon. “One-dimensional linear periodically time-varying structures, derivations, interrelationships and properties”. IEEE Proc. Vis. Image Signal Process., vol. 146, no. 5, Oct. 1999, pp. 245–252. [12] F. D. Nesser and J. L. Massey. “Proper complex random processes with applications to information theory,” IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1293–1302, Jul. 1993. [13] G. B. Giannakis. “Cyclostationary signal analysis”. Digital Signal Processing Handbook, CRC Press, 1998, pp. 17.1–17.31. [14] N. C. Shin and A. H. Sayed. “Mean-square performance of a family of affine projection algorithms”. IEEE Trans. Signal Process., vol. 52, no. 1, Jan. 2004, pp. 90–102. [15] W. A. Gardner. “Learning characteristics of stochastic-gradient-descent algorithms: A general study, analysis, and critique”. Signal Process., vol. 6, no. 2, Apr. 1984, pp. 113–133. [16] F. J. Cañete, J. A. Cortés, L. Dı́ez, and J. T. Entrambasaguas. “A channel model proposal for indoor power line communications”. IEEE Commun. Mag., vol. 49, no. 12, Dec. 2011, pp. 166–174. [17] IEEE Standards Association. “P1901.2/D0.09.00 draft standard for low frequency (less than 500 kHz) narrow band power line communications for smart grid applications”. Jun. 2013. [18] K. B. Petersen and M. S. Pedersen. The Matrix Cookbook. Technical University of Denmark, 2008. [19] T. Y. Al-Naffouri and A. H. Sayed. “Transient analysis of datanormalized adaptive filters”. IEEE Trans. Signal Processing, vol. 51, no. 3, Mar. 2003, pp. 639–652. [20] C. D. Meyer. Matrix Analysis and Applied Linear Algebra. Society for Industrial and Applied Mathematics, 2000. [21] A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill, 1991. [22] S. Elaydi. An Introduction to Difference Equations. Springer Science and Business Media, 2005. [23] C. T. Chen. Linear System Theory and Design. Oxford University Press, 1995. [24] J. D. Hamilton. Time Series Analysis. Princeton University Press, 1994. [25] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill New York, 1976.