Showing 1–26 of 26 results for author: Riegler, E

Completion of Matrices with Low Description Complexity

Authors: Erwin Riegler, Günther Koliander, David Stotz, Helmut Bölcskei

Abstract: We propose a theory for matrix completion that goes beyond the low-rank structure commonly considered in the literature and applies to general matrices of low description complexity. Specifically, complexity of the sets of matrices encompassed by the theory is measured in terms of Hausdorff and upper Minkowski dimensions. Our goal is the characterization of the number of linear measurements, with… ▽ More We propose a theory for matrix completion that goes beyond the low-rank structure commonly considered in the literature and applies to general matrices of low description complexity. Specifically, complexity of the sets of matrices encompassed by the theory is measured in terms of Hausdorff and upper Minkowski dimensions. Our goal is the characterization of the number of linear measurements, with an emphasis on rank-$1$ measurements, needed for the existence of an algorithm that yields reconstruction, either perfect, with probability 1, or with arbitrarily small probability of error, depending on the setup. Concretely, we show that matrices taken from a set $\mathcal{U}$ such that $\mathcal{U}-\mathcal{U}$ has Hausdorff dimension $s$ can be recovered from $k>s$ measurements, and random matrices supported on a set $\mathcal{U}$ of Hausdorff dimension $s$ can be recovered with probability 1 from $k>s$ measurements. What is more, we establish the existence of recovery mappings that are robust against additive perturbations or noise in the measurements. Concretely, we show that there are $β$-Hölder continuous mappings recovering matrices taken from a set of upper Minkowski dimension $s$ from $k>2s/(1-β)$ measurements and, with arbitrarily small probability of error, random matrices supported on a set of upper Minkowski dimension $s$ from $k>s/(1-β)$ measurements. The numerous concrete examples we consider include low-rank matrices, sparse matrices, QR decompositions with sparse R-components, and matrices of fractal nature. △ Less

Submitted 27 November, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

Lossy Compression of General Random Variables

Authors: Erwin Riegler, Helmut Bölcskei, Günther Koliander

Abstract: This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fract… ▽ More This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our main contributions are bounds on the rate-distortion function and the quantization error. These bounds are very general and essentially only require the existence of reference measures satisfying certain regularity conditions in terms of small ball probabilities. To illustrate the wide applicability of our results, we particularize them to random variables taking values in i) manifolds, namely, hyperspheres and Grassmannians, and ii) self-similar sets characterized by iterated function systems satisfying the weak separation property. △ Less

Submitted 2 June, 2023; v1 submitted 24 November, 2021; originally announced November 2021.

Uncertainty relations and sparse signal recovery

Authors: Erwin Riegler, Helmut Bölcskei

Abstract: This chapter provides a principled introduction to uncertainty relations underlying sparse signal recovery. We start with the seminal work by Donoho and Stark, 1989, which defines uncertainty relations as upper bounds on the operator norm of the band-limitation operator followed by the time-limitation operator, generalize this theory to arbitrary pairs of operators, and then develop -- out of this… ▽ More This chapter provides a principled introduction to uncertainty relations underlying sparse signal recovery. We start with the seminal work by Donoho and Stark, 1989, which defines uncertainty relations as upper bounds on the operator norm of the band-limitation operator followed by the time-limitation operator, generalize this theory to arbitrary pairs of operators, and then develop -- out of this generalization -- the coherence-based uncertainty relations due to Elad and Bruckstein, 2002, as well as uncertainty relations in terms of concentration of $1$-norm or $2$-norm. The theory is completed with the recently discovered set-theoretic uncertainty relations which lead to best possible recovery thresholds in terms of a general measure of parsimony, namely Minkowski dimension. We also elaborate on the remarkable connection between uncertainty relations and the "large sieve", a family of inequalities developed in analytic number theory. It is finally shown how uncertainty relations allow to establish fundamental limits of practical signal recovery problems such as inpainting, declipping, super-resolution, and denoising of signals corrupted by impulse noise or narrowband interference. Detailed proofs are provided throughout the chapter. △ Less

Submitted 26 March, 2020; v1 submitted 9 November, 2018; originally announced November 2018.

Comments: Chapter in Information-theoretic Methods in Data Science, M. Rodrigues and Y. Eldar, Eds., Cambridge University Press, 2020

Rate-Distortion Theory for General Sets and Measures

Authors: Erwin Riegler, Günther Koliander, Helmut Bölcskei

Abstract: This paper is concerned with a rate-distortion theory for sequences of i.i.d. random variables with general distribution supported on general sets including manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in modeli… ▽ More This paper is concerned with a rate-distortion theory for sequences of i.i.d. random variables with general distribution supported on general sets including manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in modeling of Ethernet traffic. We derive a lower bound on the (single-letter) rate-distortion function that applies to random variables X of general distribution and for continuous X reduces to the classical Shannon lower bound. Moreover, our lower bound is explicit up to a parameter obtained by solving a convex optimization problem in a nonnegative real variable. The only requirement for the bound to apply is the existence of a sigma-finite reference measure for X satisfying a certain subregularity condition. This condition is very general and prevents the reference measure from being highly concentrated on balls of small radii. To illustrate the wide applicability of our result, we evaluate the lower bound for a random variable distributed uniformly on a manifold, namely, the unit circle, and a random variable distributed uniformly on a self-similar set, namely, the middle third Cantor set. △ Less

Submitted 24 April, 2018; originally announced April 2018.

Lossless Analog Compression

Authors: Giovanni Alberti, Helmut Bölcskei, Camillo De Lellis, Günther Koliander, Erwin Riegler

Abstract: We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers… ▽ More We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x. We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution---with respect to the s-dimensional Hausdorff measure---supported on countable unions of s-dimensional differentiable submanifolds of the m-dimensional real coordinate space. Countable unions of differentiable submanifolds include essentially all signal models used in the compressed sensing literature. Specifically, we prove that, for almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n>s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n<s linear measurements. This leads us to the introduction of the new class of s-analytic random vectors, which admit a strong converse in the sense of n greater than or equal to s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions. △ Less

Submitted 17 July, 2019; v1 submitted 19 March, 2018; originally announced March 2018.

Lossless Linear Analog Compression

Authors: Giovanni Alberti, Helmut Bölcskei, Camillo De Lellis, Günther Koliander, Erwin Riegler

Abstract: We establish the fundamental limits of lossless linear analog compression by considering the recovery of random vectors ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ from the noiseless linear measurements ${\boldsymbol{\mathsf{y}}}=\boldsymbol{A}{\boldsymbol{\mathsf{x}}}$ with measurement matrix $\boldsymbol{A}\in{\mathbb R}^{n\times m}$. Specifically, for a random vector… ▽ More We establish the fundamental limits of lossless linear analog compression by considering the recovery of random vectors ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ from the noiseless linear measurements ${\boldsymbol{\mathsf{y}}}=\boldsymbol{A}{\boldsymbol{\mathsf{x}}}$ with measurement matrix $\boldsymbol{A}\in{\mathbb R}^{n\times m}$. Specifically, for a random vector ${\boldsymbol{\mathsf{x}}}\in{\mathbb R}^m$ of arbitrary distribution we show that ${\boldsymbol{\mathsf{x}}}$ can be recovered with zero error probability from $n>\inf\underline{\operatorname{dim}}_\mathrm{MB}(U)$ linear measurements, where $\underline{\operatorname{dim}}_\mathrm{MB}(\cdot)$ denotes the lower modified Minkowski dimension and the infimum is over all sets $U\subseteq{\mathbb R}^{m}$ with $\mathbb{P}[{\boldsymbol{\mathsf{x}}}\in U]=1$. This achievability statement holds for Lebesgue almost all measurement matrices $\boldsymbol{A}$. We then show that $s$-rectifiable random vectors---a stochastic generalization of $s$-sparse vectors---can be recovered with zero error probability from $n>s$ linear measurements. From classical compressed sensing theory we would expect $n\geq s$ to be necessary for successful recovery of ${\boldsymbol{\mathsf{x}}}$. Surprisingly, certain classes of $s$-rectifiable random vectors can be recovered from fewer than $s$ measurements. Imposing an additional regularity condition on the distribution of $s$-rectifiable random vectors ${\boldsymbol{\mathsf{x}}}$, we do get the expected converse result of $s$ measurements being necessary. The resulting class of random vectors appears to be new and will be referred to as $s$-analytic random vectors. △ Less

Submitted 5 May, 2016; v1 submitted 3 May, 2016; originally announced May 2016.

Almost lossless analog signal separation and probabilistic uncertainty relations

Authors: David Stotz, Erwin Riegler, Eirikur Agustsson, Helmut Bölcskei

Abstract: We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals, modeled as general random vectors, from the noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on analog compression and encompasses, inter alia, inpainting, declipping, supe… ▽ More We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals, modeled as general random vectors, from the noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on analog compression and encompasses, inter alia, inpainting, declipping, super-resolution, the recovery of signals corrupted by impulse noise, and the separation of (e.g., audio or video) signals into two distinct components. The main results we report are general achievability bounds for the compression rate, i.e., the number of measurements relative to the dimension of the ambient space the signals live in, under either measurability or Hölder continuity imposed on the separator. Furthermore, we find a matching converse for sources of mixed discrete-continuous distribution. For measurable separators our proofs are based on a new probabilistic uncertainty relation which shows that the intersection of generic subspaces with general sets of sufficiently small Minkowski dimension is empty. Hölder continuous separators are dealt with by introducing the concept of regularized probabilistic uncertainty relations. The probabilistic uncertainty relations we develop are inspired by embedding results in dynamical systems theory due to Sauer et al. (1991) and---conceptually---parallel classical Donoho-Stark and Elad-Bruckstein uncertainty principles at the heart of compressed sensing theory. Operationally, the new uncertainty relations take the theory of sparse signal separation beyond traditional sparsity---as measured in terms of the number of non-zero entries---to the more general notion of low description complexity as quantified by Minkowski dimension. Finally, our approach also allows to significantly strengthen key results in Wu and Verdú (2010). △ Less

Submitted 13 July, 2017; v1 submitted 3 December, 2015; originally announced December 2015.

Comments: to appear in IEEE Trans. on Inf. Theory

Entropy and Source Coding for Integer-Dimensional Singular Random Variables

Authors: Günther Koliander, Georg Pichler, Erwin Riegler, Franz Hlawatsch

Abstract: Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random var… ▽ More Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random variables and the differential entropy of continuous random variables as special cases. We show that it transforms in a natural manner under Lipschitz functions, and that it is invariant under unitary transformations. We define joint entropy and conditional entropy for integer-dimensional singular random variables, and we show that the proposed entropy conveys useful expressions of the mutual information. As first applications of our entropy definition, we present a result on the minimal expected codeword length of quantized integer-dimensional singular sources and a Shannon lower bound for integer-dimensional singular sources. △ Less

Submitted 3 January, 2017; v1 submitted 13 May, 2015; originally announced May 2015.

MSC Class: 94A17; 94A29

Journal ref: IEEE Transactions on Information Theory, Volume: 62, Issue: 11, Nov. 2016, pp. 6124 - 6154

Information-Theoretic Limits of Matrix Completion

Authors: Erwin Riegler, David Stotz, Helmut Bölcskei

Abstract: We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of "low description complexity". Specifically, we consider $m\times n$ random matrices $\mathbf{X}$ of arbitrary distribution (continuous, discrete, discrete-continuous mixture, or even singular). With $\mathcal{S}$ an $\varepsilon$-support set of… ▽ More We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of "low description complexity". Specifically, we consider $m\times n$ random matrices $\mathbf{X}$ of arbitrary distribution (continuous, discrete, discrete-continuous mixture, or even singular). With $\mathcal{S}$ an $\varepsilon$-support set of $\mathbf{X}$, i.e., $\mathrm{P}[\mathbf{X}\in\mathcal{S}]\geq 1-\varepsilon$, and $\underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ denoting the lower Minkowski dimension of $\mathcal{S}$, we show that $k> \underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ trace inner product measurements with measurement matrices $A_i$, suffice to recover $\mathbf{X}$ with probability of error at most $\varepsilon$. The result holds for Lebesgue a.a. $A_i$ and does not need incoherence between the $A_i$ and the unknown matrix $\mathbf{X}$. We furthermore show that $k> \underline{\mathrm{dim}}_\mathrm{B}(\mathcal{S})$ measurements also suffice to recover the unknown matrix $\mathbf{X}$ from measurements taken with rank-one $A_i$, again this applies to a.a. rank-one $A_i$. Rank-one measurement matrices are attractive as they require less storage space than general measurement matrices and can be applied faster. Particularizing our results to the recovery of low-rank matrices, we find that $k>(m+n-r)r$ measurements are sufficient to recover matrices of rank at most $r$. Finally, we construct a class of rank-$r$ matrices that can be recovered with arbitrarily small probability of error from $k<(m+n-r)r$ measurements. △ Less

Submitted 10 August, 2016; v1 submitted 20 April, 2015; originally announced April 2015.

Almost Lossless Analog Compression without Phase Information

Authors: Erwin Riegler, Georg Tauböck

Abstract: We propose an information-theoretic framework for phase retrieval. Specifically, we consider the problem of recovering an unknown n-dimensional vector x up to an overall sign factor from m=Rn phaseless measurements with compression rate R and derive a general achievability bound for R. Surprisingly, it turns out that this bound on the compression rate is the same as the one for almost lossless ana… ▽ More We propose an information-theoretic framework for phase retrieval. Specifically, we consider the problem of recovering an unknown n-dimensional vector x up to an overall sign factor from m=Rn phaseless measurements with compression rate R and derive a general achievability bound for R. Surprisingly, it turns out that this bound on the compression rate is the same as the one for almost lossless analog compression obtained by Wu and Verdú (2010): Phaseless linear measurements are as good as linear measurements with full phase information in the sense that ignoring the sign of m measurements only leaves us with an ambiguity with respect to an overall sign factor of x. △ Less

Submitted 22 April, 2015; v1 submitted 12 April, 2015; originally announced April 2015.

Oversampling Increases the Pre-Log of Noncoherent Rayleigh Fading Channels

Authors: Meik Dörpinghaus, Günther Koliander, Giuseppe Durisi, Erwin Riegler, Heinrich Meyr

Abstract: We analyze the capacity of a continuous-time, time-selective, Rayleigh block-fading channel in the high signal-to-noise ratio (SNR) regime. The fading process is assumed stationary within each block and to change independently from block to block; furthermore, its realizations are not known a priori to the transmitter and the receiver (noncoherent setting). A common approach to analyzing the capac… ▽ More We analyze the capacity of a continuous-time, time-selective, Rayleigh block-fading channel in the high signal-to-noise ratio (SNR) regime. The fading process is assumed stationary within each block and to change independently from block to block; furthermore, its realizations are not known a priori to the transmitter and the receiver (noncoherent setting). A common approach to analyzing the capacity of this channel is to assume that the receiver performs matched filtering followed by sampling at symbol rate (symbol matched filtering). This yields a discrete-time channel in which each transmitted symbol corresponds to one output sample. Liang & Veeravalli (2004) showed that the capacity of this discrete-time channel grows logarithmically with the SNR, with a capacity pre-log equal to $1-{Q}/{N}$. Here, $N$ is the number of symbols transmitted within one fading block, and $Q$ is the rank of the covariance matrix of the discrete-time channel gains within each fading block. In this paper, we show that symbol matched filtering is not a capacity-achieving strategy for the underlying continuous-time channel. Specifically, we analyze the capacity pre-log of the discrete-time channel obtained by oversampling the continuous-time channel output, i.e., by sampling it faster than at symbol rate. We prove that by oversampling by a factor two one gets a capacity pre-log that is at least as large as $1-1/N$. Since the capacity pre-log corresponding to symbol-rate sampling is $1-Q/N$, our result implies indeed that symbol matched filtering is not capacity achieving at high SNR. △ Less

Submitted 4 August, 2014; v1 submitted 2 May, 2014; originally announced May 2014.

Comments: To appear in the IEEE Transactions on Information Theory

MSC Class: 94A15

Distributed Localization and Tracking of Mobile Networks Including Noncooperative Objects - Extended Version

Authors: Florian Meyer, Ondrej Hlinka, Henk Wymeersch, Erwin Riegler, Franz Hlawatsch

Abstract: We propose a Bayesian method for distributed sequential localization of mobile networks composed of both cooperative agents and noncooperative objects. Our method provides a consistent combination of cooperative self-localization (CS) and distributed tracking (DT). Multiple mobile agents and objects are localized and tracked using measurements between agents and objects and between agents. For a d… ▽ More We propose a Bayesian method for distributed sequential localization of mobile networks composed of both cooperative agents and noncooperative objects. Our method provides a consistent combination of cooperative self-localization (CS) and distributed tracking (DT). Multiple mobile agents and objects are localized and tracked using measurements between agents and objects and between agents. For a distributed operation and low complexity, we combine particle-based belief propagation with a consensus or gossip scheme. High localization accuracy is achieved through a probabilistic information transfer between the CS and DT parts of the underlying factor graph. Simulation results demonstrate significant improvements in both agent self-localization and object localization performance compared to separate CS and DT, and very good scaling properties with respect to the numbers of agents and objects. △ Less

Submitted 31 December, 2015; v1 submitted 7 March, 2014; originally announced March 2014.

Comments: 16 pages, 11 figures

Degrees of Freedom of Generic Block-Fading MIMO Channels without A Priori Channel State Information

Authors: Günther Koliander, Erwin Riegler, Giuseppe Durisi, Franz Hlawatsch

Abstract: We studynthe high-SNR capacity of generic MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information but both are aware of the channel statistics. In contrast to the well-established constant block-fading model, we allow the fading to vary within each block with a temporal correlation that is "generic" (in the sense… ▽ More We studynthe high-SNR capacity of generic MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information but both are aware of the channel statistics. In contrast to the well-established constant block-fading model, we allow the fading to vary within each block with a temporal correlation that is "generic" (in the sense used in the interference-alignment literature). We show that the number of degrees of freedom of a generic MIMO Rayleigh block-fading channel with $T$ transmit antennas and block length $N$ is given by $T(1-1/N)$ provided that $T<N$ and the number of receive antennas is at least $T(N-1)/(N-T)$. A comparison with the constant block-fading channel (where the fading is constant within each block) shows that, for large block lengths, generic correlation increases the number of degrees of freedom by a factor of up to four. △ Less

Submitted 5 November, 2014; v1 submitted 9 October, 2013; originally announced October 2013.

Comments: To appear in IEEE Trans. Inf. Theory

MSC Class: 94A15

Generic Correlation Increases Noncoherent MIMO Capacity

Authors: Günther Koliander, Erwin Riegler, Giuseppe Durisi, Franz Hlawatsch

Abstract: We study the high-SNR capacity of MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information. We show that when the number of receive antennas is sufficiently large and the temporal correlation within each block is "generic" (in the sense used in the interference-alignment literature), the capacity pre-log is given b… ▽ More We study the high-SNR capacity of MIMO Rayleigh block-fading channels in the noncoherent setting where neither transmitter nor receiver has a priori channel state information. We show that when the number of receive antennas is sufficiently large and the temporal correlation within each block is "generic" (in the sense used in the interference-alignment literature), the capacity pre-log is given by T(1-1/N) for T<N, where T denotes the number of transmit antennas and N denotes the block length. A comparison with the widely used constant block-fading channel (where the fading is constant within each block) shows that for a large block length, generic correlation increases the capacity pre-log by a factor of about four. △ Less

Submitted 5 June, 2013; originally announced June 2013.

Comments: To be presented at IEEE Int. Symp. Inf. Theory (ISIT) 2013, Istanbul, Turkey

MSC Class: 94A15

Almost Lossless Analog Signal Separation

Authors: David Stotz, Erwin Riegler, Helmut Bölcskei

Abstract: We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals from a noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression. The main results of the present paper are a general achievability bound for th… ▽ More We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals from a noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression. The main results of the present paper are a general achievability bound for the compression rate in the analog signal separation problem, an exact expression for the optimal compression rate in the case of signals that have mixed discrete-continuous distributions, and a new technique for showing that the intersection of generic subspaces with subsets of sufficiently small Minkowski dimension is empty. This technique can also be applied to obtain a simplified proof of a key result in Wu and Verdú (2010). △ Less

Submitted 15 May, 2013; originally announced May 2013.

Comments: To be presented at IEEE Int. Symp. Inf. Theory 2013, Istanbul, Turkey

A Lower Bound on the Noncoherent Capacity Pre-log for the MIMO Channel with Temporally Correlated Fading

Authors: Günther Koliander, Erwin Riegler, Giuseppe Durisi, Veniamin I. Morgenshtern, Franz Hlawatsch

Abstract: We derive a lower bound on the capacity pre-log of a temporally correlated Rayleigh block-fading multiple-input multiple-output (MIMO) channel with T transmit antennas and R receive antennas in the noncoherent setting (no a priori channel knowledge at the transmitter and the receiver). In this model, the fading process changes independently across blocks of length L and is temporally correlated wi… ▽ More We derive a lower bound on the capacity pre-log of a temporally correlated Rayleigh block-fading multiple-input multiple-output (MIMO) channel with T transmit antennas and R receive antennas in the noncoherent setting (no a priori channel knowledge at the transmitter and the receiver). In this model, the fading process changes independently across blocks of length L and is temporally correlated within each block for each transmit-receive antenna pair, with a given rank Q of the corresponding correlation matrix. Our result implies that for almost all choices of the coloring matrix that models the temporal correlation, the pre-log can be lower-bounded by T(1-1/L) for T <= (L-1)/Q provided that R is sufficiently large. The widely used constant block-fading model is equivalent to the temporally correlated block-fading model with Q = 1 for the special case when the temporal correlation for each transmit-receive antenna pair is the same, which is unlikely to be observed in practice. For the constant block-fading model, the capacity pre-log is given by T(1-T/L), which is smaller than our lower bound for the case Q = 1. Thus, our result suggests that the assumptions underlying the constant block- fading model lead to a pessimistic result for the capacity pre-log. △ Less

Submitted 11 February, 2013; originally announced February 2013.

Comments: 8 pages, presented at the 50th Annual Allerton Conference on Communication, Control, and Computing - 2012

MSC Class: 94A15

Fixed Points of Generalized Approximate Message Passing with Arbitrary Matrices

Authors: Sundeep Rangan, Philip Schniter, Erwin Riegler, Alyson Fletcher, Volkan Cevher

Abstract: The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these met… ▽ More The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based on Gaussian approximations of loopy belief propagation, have recently attracted considerable attention for such problems. For large random transforms, these methods exhibit fast convergence and admit precise analytic characterizations with testable conditions for optimality, even for certain non-convex problem instances. However, the behavior of AMP under general transforms is not fully understood. In this paper, we consider the generalized AMP (GAMP) algorithm and relate the method to more common optimization techniques. This analysis enables a precise characterization of the GAMP algorithm fixed-points that applies to arbitrary transforms. In particular, we show that the fixed points of the so-called max-sum GAMP algorithm for MAP estimation are critical points of a constrained maximization of the posterior density. The fixed-points of the sum-product GAMP algorithm for estimation of the posterior marginals can be interpreted as critical points of a certain free energy. △ Less

Submitted 2 May, 2016; v1 submitted 26 January, 2013; originally announced January 2013.

Simultaneous Distributed Sensor Self-Localization and Target Tracking Using Belief Propagation and Likelihood Consensus

Authors: Florian Meyer, Erwin Riegler, Ondrej Hlinka, Franz Hlawatsch

Abstract: We introduce the framework of cooperative simultaneous localization and tracking (CoSLAT), which provides a consistent combination of cooperative self-localization (CSL) and distributed target tracking (DTT) in sensor networks without a fusion center. CoSLAT extends simultaneous localization and tracking (SLAT) in that it uses also intersensor measurements. Starting from a factor graph formulation… ▽ More We introduce the framework of cooperative simultaneous localization and tracking (CoSLAT), which provides a consistent combination of cooperative self-localization (CSL) and distributed target tracking (DTT) in sensor networks without a fusion center. CoSLAT extends simultaneous localization and tracking (SLAT) in that it uses also intersensor measurements. Starting from a factor graph formulation of the CoSLAT problem, we develop a particle-based, distributed message passing algorithm for CoSLAT that combines nonparametric belief propagation with the likelihood consensus scheme. The proposed CoSLAT algorithm improves on state-of-the-art CSL and DTT algorithms by exchanging probabilistic information between CSL and DTT. Simulation results demonstrate substantial improvements in both self-localization and tracking performance. △ Less

Submitted 29 November, 2012; originally announced November 2012.

Comments: 10 pages, 5 figures

Capacity Pre-Log of Noncoherent SIMO Channels via Hironaka's Theorem

Authors: Veniamin I. Morgenshtern, Erwin Riegler, Wei Yang, Giuseppe Durisi, Shaowei Lin, Bernd Sturmfels, Helmut Bölcskei

Abstract: We find the capacity pre-log of a temporally correlated Rayleigh block-fading SIMO channel in the noncoherent setting. It is well known that for block-length L and rank of the channel covariance matrix equal to Q, the capacity pre-log in the SISO case is given by 1-Q/L. Here, Q/L can be interpreted as the pre-log penalty incurred by channel uncertainty. Our main result reveals that, by adding only… ▽ More We find the capacity pre-log of a temporally correlated Rayleigh block-fading SIMO channel in the noncoherent setting. It is well known that for block-length L and rank of the channel covariance matrix equal to Q, the capacity pre-log in the SISO case is given by 1-Q/L. Here, Q/L can be interpreted as the pre-log penalty incurred by channel uncertainty. Our main result reveals that, by adding only one receive antenna, this penalty can be reduced to 1/L and can, hence, be made to vanish in the large-L limit, even if Q/L remains constant as L goes to infinity. Intuitively, even though the SISO channels between the transmit antenna and the two receive antennas are statistically independent, the transmit signal induces enough statistical dependence between the corresponding receive signals for the second receive antenna to be able to resolve the uncertainty associated with the first receive antenna's channel and thereby make the overall system appear coherent. The proof of our main theorem is based on a deep result from algebraic geometry known as Hironaka's Theorem on the Resolution of Singularities. △ Less

Submitted 2 March, 2013; v1 submitted 12 April, 2012; originally announced April 2012.

Message-Passing Algorithms for Channel Estimation and Decoding Using Approximate Inference

Authors: Mihai-Alin Badiu, Gunvor Elisabeth Kirkelund, Carles Navarro Manchón, Erwin Riegler, Bernard Henri Fleury

Abstract: We design iterative receiver schemes for a generic wireless communication system by treating channel estimation and information decoding as an inference problem in graphical models. We introduce a recently proposed inference framework that combines belief propagation (BP) and the mean field (MF) approximation and includes these algorithms as special cases. We also show that the expectation propaga… ▽ More We design iterative receiver schemes for a generic wireless communication system by treating channel estimation and information decoding as an inference problem in graphical models. We introduce a recently proposed inference framework that combines belief propagation (BP) and the mean field (MF) approximation and includes these algorithms as special cases. We also show that the expectation propagation and expectation maximization algorithms can be embedded in the BP-MF framework with slight modifications. By applying the considered inference algorithms to our probabilistic model, we derive four different message-passing receiver schemes. Our numerical evaluation demonstrates that the receiver based on the BP-MF framework and its variant based on BP-EM yield the best compromise between performance, computational complexity and numerical stability among all candidate algorithms. △ Less

Submitted 23 July, 2012; v1 submitted 7 February, 2012; originally announced February 2012.

Comments: Accepted for publication in the Proceedings of 2012 IEEE International Symposium on Information Theory

On the Capacity of Large-MIMO Block-Fading Channels

Authors: Wei Yang, Giuseppe Durisi, Erwin Riegler

Abstract: We characterize the capacity of Rayleigh block-fading multiple-input multiple-output (MIMO) channels in the noncoherent setting where transmitter and receiver have no a priori knowledge of the realizations of the fading channel. We prove that unitary space-time modulation (USTM) is not capacity-achieving in the high signal-to-noise ratio (SNR) regime when the total number of antennas exceeds the c… ▽ More We characterize the capacity of Rayleigh block-fading multiple-input multiple-output (MIMO) channels in the noncoherent setting where transmitter and receiver have no a priori knowledge of the realizations of the fading channel. We prove that unitary space-time modulation (USTM) is not capacity-achieving in the high signal-to-noise ratio (SNR) regime when the total number of antennas exceeds the coherence time of the fading channel (expressed in multiples of the symbol duration), a situation that is relevant for MIMO systems with large antenna arrays (large-MIMO systems). This result settles a conjecture by Zheng & Tse (2002) in the affirmative. The capacity-achieving input signal, which we refer to as Beta-variate space-time modulation (BSTM), turns out to be the product of a unitary isotropically distributed random matrix, and a diagonal matrix whose nonzero entries are distributed as the square-root of the eigenvalues of a Beta-distributed random matrix of appropriate size. Numerical results illustrate that using BSTM instead of USTM in large-MIMO systems yields a rate gain as large as 13% for SNR values of practical interest. △ Less

Submitted 30 September, 2012; v1 submitted 1 February, 2012; originally announced February 2012.

Comments: To appear in IEEE Journal on Selected Areas in Communication

Merging Belief Propagation and the Mean Field Approximation: A Free Energy Approach

Authors: Erwin Riegler, Gunvor Elisabeth Kirkelund, Carles Navarro Manchón, Mihai-Alin Badiu, Bernard Henry Fleury

Abstract: We present a joint message passing approach that combines belief propagation and the mean field approximation. Our analysis is based on the region-based free energy approximation method proposed by Yedidia et al. We show that the message passing fixed-point equations obtained with this combination correspond to stationary points of a constrained region-based free energy approximation. Moreover, we… ▽ More We present a joint message passing approach that combines belief propagation and the mean field approximation. Our analysis is based on the region-based free energy approximation method proposed by Yedidia et al. We show that the message passing fixed-point equations obtained with this combination correspond to stationary points of a constrained region-based free energy approximation. Moreover, we present a convergent implementation of these message passing fixedpoint equations provided that the underlying factor graph fulfills certain technical conditions. In addition, we show how to include hard constraints in the part of the factor graph corresponding to belief propagation. Finally, we demonstrate an application of our method to iterative channel estimation and decoding in an orthogonal frequency division multiplexing (OFDM) system. △ Less

Submitted 28 June, 2012; v1 submitted 2 December, 2011; originally announced December 2011.

Receiver Architectures for MIMO-OFDM Based on a Combined VMP-SP Algorithm

Authors: Carles Navarro Manchón, Gunvor E. Kirkelund, Erwin Riegler, Lars P. B. Christensen, Bernard H. Fleury

Abstract: Iterative information processing, either based on heuristics or analytical frameworks, has been shown to be a very powerful tool for the design of efficient, yet feasible, wireless receiver architectures. Within this context, algorithms performing message-passing on a probabilistic graph, such as the sum-product (SP) and variational message passing (VMP) algorithms, have become increasingly popula… ▽ More Iterative information processing, either based on heuristics or analytical frameworks, has been shown to be a very powerful tool for the design of efficient, yet feasible, wireless receiver architectures. Within this context, algorithms performing message-passing on a probabilistic graph, such as the sum-product (SP) and variational message passing (VMP) algorithms, have become increasingly popular. In this contribution, we apply a combined VMP-SP message-passing technique to the design of receivers for MIMO-ODFM systems. The message-passing equations of the combined scheme can be obtained from the equations of the stationary points of a constrained region-based free energy approximation. When applied to a MIMO-OFDM probabilistic model, we obtain a generic receiver architecture performing iterative channel weight and noise precision estimation, equalization and data decoding. We show that this generic scheme can be particularized to a variety of different receiver structures, ranging from high-performance iterative structures to low complexity receivers. This allows for a flexible design of the signal processing specially tailored for the requirements of each specific application. The numerical assessment of our solutions, based on Monte Carlo simulations, corroborates the high performance of the proposed algorithms and their superiority to heuristic approaches. △ Less

Submitted 24 November, 2011; originally announced November 2011.

Capacity Pre-Log of SIMO Correlated Block-Fading Channels

Authors: Wei Yang, Giuseppe Durisi, Veniamin I. Morgenshtern, Erwin Riegler

Abstract: We establish an upper bound on the noncoherent capacity pre-log of temporally correlated block-fading single-input multiple-output (SIMO) channels. The upper bound matches the lower bound recently reported in Riegler et al. (2011), and, hence, yields a complete characterization of the SIMO noncoherent capacity pre-log, provided that the channel covariance matrix satisfies a mild technical conditio… ▽ More We establish an upper bound on the noncoherent capacity pre-log of temporally correlated block-fading single-input multiple-output (SIMO) channels. The upper bound matches the lower bound recently reported in Riegler et al. (2011), and, hence, yields a complete characterization of the SIMO noncoherent capacity pre-log, provided that the channel covariance matrix satisfies a mild technical condition. This result allows one to determine the optimal number of receive antennas to be used to maximize the capacity pre-log for a given block-length and a given rank of the channel covariance matrix. △ Less

Submitted 13 September, 2011; originally announced September 2011.

Comments: Accepted by ISWCS11

Noncoherent SIMO Pre-Log via Resolution of Singularities

Authors: Erwin Riegler, Veniamin I. Morgenshtern, Giuseppe Durisi, Shaowei Lin, Bernd Sturmfels, Helmut Bölcskei

Abstract: We establish a lower bound on the noncoherent capacity pre-log of a temporally correlated Rayleigh block-fading single-input multiple-output (SIMO) channel. Our result holds for arbitrary rank Q of the channel correlation matrix, arbitrary block-length L > Q, and arbitrary number of receive antennas R, and includes the result in Morgenshtern et al. (2010) as a special case. It is well known that t… ▽ More We establish a lower bound on the noncoherent capacity pre-log of a temporally correlated Rayleigh block-fading single-input multiple-output (SIMO) channel. Our result holds for arbitrary rank Q of the channel correlation matrix, arbitrary block-length L > Q, and arbitrary number of receive antennas R, and includes the result in Morgenshtern et al. (2010) as a special case. It is well known that the capacity pre-log for this channel in the single-input single-output (SISO) case is given by 1-Q/L, where Q/L is the penalty incurred by channel uncertainty. Our result reveals that this penalty can be reduced to 1/L by adding only one receive antenna, provided that L \geq 2Q - 1 and the channel correlation matrix satisfies mild technical conditions. The main technical tool used to prove our result is Hironaka's celebrated theorem on resolution of singularities in algebraic geometry. △ Less

Submitted 30 May, 2011; originally announced May 2011.

Comments: IEEE International Symposium on Information Theory 2011 (ISIT 2011), Saint Petersburg, Russia, to appear

On the Achievability of Interference Alignment in the K-User Constant MIMO Interference Channel

Authors: Roland Tresch, Maxime Guillaud, Erwin Riegler

Abstract: Interference alignment in the K-user MIMO interference channel with constant channel coefficients is considered. A novel constructive method for finding the interference alignment solution is proposed for the case where the number of transmit antennas equals the number of receive antennas (NT = NR = N), the number of transmitter-receiver pairs equals K = N + 1, and all interference alignment mul… ▽ More Interference alignment in the K-user MIMO interference channel with constant channel coefficients is considered. A novel constructive method for finding the interference alignment solution is proposed for the case where the number of transmit antennas equals the number of receive antennas (NT = NR = N), the number of transmitter-receiver pairs equals K = N + 1, and all interference alignment multiplexing gains are one. The core of the method consists of solving an eigenvalue problem that incorporates the channel matrices of all interfering links. This procedure provides insight into the feasibility of signal vector spaces alignment schemes in finite dimensional MIMO interference channels. △ Less

Submitted 28 April, 2009; originally announced April 2009.

Comments: submitted to IEEE Workshop on Statistical Signal Processing (SSP09)