Showing 1–30 of 30 results for author: Nadakuditi, R R

Detection thresholds in very sparse matrix completion

Authors: Charles Bordenave, Simon Coste, Raj Rao Nadakuditi

Abstract: Let $A$ be a rectangular matrix of size $m\times n$ and $A_1$ be the random matrix where each entry of $A$ is multiplied by an independent $\{0,1\}$-Bernoulli random variable with parameter $1/2$. This paper is about when, how and why the non-Hermitian eigen-spectra of the randomly induced asymmetric matrices $A_1 (A - A_1)^*$ and $(A-A_1)^*A_1$ captures more of the relevant information about the… ▽ More Let $A$ be a rectangular matrix of size $m\times n$ and $A_1$ be the random matrix where each entry of $A$ is multiplied by an independent $\{0,1\}$-Bernoulli random variable with parameter $1/2$. This paper is about when, how and why the non-Hermitian eigen-spectra of the randomly induced asymmetric matrices $A_1 (A - A_1)^*$ and $(A-A_1)^*A_1$ captures more of the relevant information about the principal component structure of $A$ than via its SVD or the eigen-spectra of $A A^*$ and $A^* A$, respectively. Hint: the asymmetry inducing randomness breaks the echo-chamber effect that cripples the SVD. We illustrate the application of this striking phenomenon on the low-rank matrix completion problem for the setting where each entry is observed with probability $d/n$, including the very sparse regime where $d$ is of order $1$, where matrix completion via the SVD of $A$ fails or produces unreliable recovery. We determine an asymptotically exact, matrix-dependent, non-universal detection threshold above which reliable, statistically optimal matrix recovery using a new, universal data-driven matrix-completion algorithm is possible. Averaging the left and right eigenvectors provably improves the recovered matrix but not the detection threshold. We define another variant of this asymmetric procedure that bypasses the randomization step and has a detection threshold that is smaller by a constant factor but with a computational cost that is larger by a polynomial factor of the number of observed entries. Both detection thresholds shatter the seeming barrier due to the well-known information theoretical limit $d \asymp \log n$ for matrix completion found in the literature. △ Less

Submitted 4 August, 2020; v1 submitted 12 May, 2020; originally announced May 2020.

Comments: 84 pages, 10 pictures. Submitted

Sparse Equisigned PCA: Algorithms and Performance Bounds in the Noisy Rank-1 Setting

Authors: Arvind Prasadan, Raj Rao Nadakuditi, Debashis Paul

Abstract: Singular value decomposition (SVD) based principal component analysis (PCA) breaks down in the high-dimensional and limited sample size regime below a certain critical eigen-SNR that depends on the dimensionality of the system and the number of samples. Below this critical eigen-SNR, the estimates returned by the SVD are asymptotically uncorrelated with the latent principal components. We consider… ▽ More Singular value decomposition (SVD) based principal component analysis (PCA) breaks down in the high-dimensional and limited sample size regime below a certain critical eigen-SNR that depends on the dimensionality of the system and the number of samples. Below this critical eigen-SNR, the estimates returned by the SVD are asymptotically uncorrelated with the latent principal components. We consider a setting where the left singular vector of the underlying rank one signal matrix is assumed to be sparse and the right singular vector is assumed to be equisigned, that is, having either only nonnegative or only nonpositive entries. We consider six different algorithms for estimating the sparse principal component based on different statistical criteria and prove that by exploiting sparsity, we recover consistent estimates in the low eigen-SNR regime where the SVD fails. Our analysis reveals conditions under which a coordinate selection scheme based on a \textit{sum-type decision statistic} outperforms schemes that utilize the $\ell_1$ and $\ell_2$ norm-based statistics. We derive lower bounds on the size of detectable coordinates of the principal left singular vector and utilize these lower bounds to derive lower bounds on the worst-case risk. Finally, we verify our findings with numerical simulations and illustrate the performance with a video data example, where the interest is in identifying objects. △ Less

Submitted 16 December, 2019; v1 submitted 22 May, 2019; originally announced May 2019.

Comments: To appear, Electronic Journal of Statistics, 2020

Free Component Analysis: Theory, Algorithms & Applications

Authors: Hao Wu, Raj Rao Nadakuditi

Abstract: We describe a method for unmixing mixtures of freely independent random variables in a manner analogous to the independent component analysis (ICA) based method for unmixing independent random variables from their additive mixtures. Random matrices play the role of free random variables in this context so the method we develop, which we call Free component analysis (FCA), unmixes matrices from add… ▽ More We describe a method for unmixing mixtures of freely independent random variables in a manner analogous to the independent component analysis (ICA) based method for unmixing independent random variables from their additive mixtures. Random matrices play the role of free random variables in this context so the method we develop, which we call Free component analysis (FCA), unmixes matrices from additive mixtures of matrices. Thus, while the mixing model is standard, the novelty and difference in unmixing performance comes from the introduction of a new statistical criteria, derived from free probability theory, that quantify freeness analogous to how kurtosis and entropy quantify independence. We describe the theory, the various algorithms, and compare FCA to vanilla ICA which does not account for spatial or temporal structure. We highlight why the statistical criteria make FCA also vanilla despite its matricial underpinnings and show that FCA performs comparably to, and sometimes better than, (vanilla) ICA in every application, such as image and speech unmixing, where ICA has been known to succeed. Our computational experiments suggest that not-so-random matrices, such as images and short time fourier transform matrix of waveforms are (closer to being) freer "in the wild" than we might have theoretically expected. △ Less

Submitted 6 February, 2022; v1 submitted 5 May, 2019; originally announced May 2019.

Comments: 72 pages, 16 figures

MSC Class: 60B20; 46L54; 68U10

Time Series Source Separation using Dynamic Mode Decomposition

Authors: Arvind Prasadan, Raj Rao Nadakuditi

Abstract: The Dynamic Mode Decomposition (DMD) extracted dynamic modes are the non-orthogonal eigenvectors of the matrix that best approximates the one-step temporal evolution of the multivariate samples. In the context of dynamical system analysis, the extracted dynamic modes are a generalization of global stability modes. We apply DMD to a data matrix whose rows are linearly independent, additive mixtures… ▽ More The Dynamic Mode Decomposition (DMD) extracted dynamic modes are the non-orthogonal eigenvectors of the matrix that best approximates the one-step temporal evolution of the multivariate samples. In the context of dynamical system analysis, the extracted dynamic modes are a generalization of global stability modes. We apply DMD to a data matrix whose rows are linearly independent, additive mixtures of latent time series. We show that when the latent time series are uncorrelated at a lag of one time-step then, in the large sample limit, the recovered dynamic modes will approximate, up to a column-wise normalization, the columns of the mixing matrix. Thus, DMD is a time series blind source separation algorithm in disguise, but is different from closely related second order algorithms such as the Second-Order Blind Identification (SOBI) method and the Algorithm for Multiple Unknown Signals Extraction (AMUSE). All can unmix mixed stationary, ergodic Gaussian time series in a way that kurtosis-based Independent Components Analysis (ICA) fundamentally cannot. We use our insights on single lag DMD to develop a higher-lag extension, analyze the finite sample performance with and without randomly missing data, and identify settings where the higher lag variant can outperform the conventional single lag variant. We validate our results with numerical simulations, and highlight how DMD can be used in change point detection. △ Less

Submitted 5 March, 2020; v1 submitted 4 March, 2019; originally announced March 2019.

Comments: Accepted in SIADS (SIAM's Journal of Applied Dynamical Systems)

MSC Class: 37M10; 62M10; 47A55; 47A75; 60G35; 37N30

Spectra of random networks with arbitrary degrees

Authors: M. E. J. Newman, Xiao Zhang, Raj Rao Nadakuditi

Abstract: We derive a message passing method for computing the spectra of locally tree-like networks and an approximation to it that allows us to compute closed-form expressions or fast numerical approximates for the spectral density of random graphs with arbitrary node degrees -- the so-called configuration model. We find the latter approximation to work well for all but the sparsest of networks. We also d… ▽ More We derive a message passing method for computing the spectra of locally tree-like networks and an approximation to it that allows us to compute closed-form expressions or fast numerical approximates for the spectral density of random graphs with arbitrary node degrees -- the so-called configuration model. We find the latter approximation to work well for all but the sparsest of networks. We also derive bounds on the position of the band edges of the spectrum, which are important for identifying structural phase transitions in networks. △ Less

Submitted 7 January, 2019; originally announced January 2019.

Comments: 10 pages, 4 figures

Journal ref: Phys. Rev. E 99, 042309 (2019)

Online Adaptive Image Reconstruction (OnAIR) Using Dictionary Models

Authors: Brian E. Moore, Saiprasad Ravishankar, Raj Rao Nadakuditi, Jeffrey A. Fessler

Abstract: Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential) adaptive reconstruction of dynamic image sequences from linear (typically under… ▽ More Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential) adaptive reconstruction of dynamic image sequences from linear (typically undersampled) measurements. We model the spatiotemporal patches of the underlying dynamic image sequence as sparse in a dictionary, and we simultaneously estimate the dictionary and the images sequentially from streaming measurements. Multiple constraints on the adapted dictionary are also considered such as a unitary matrix, or low-rank dictionary atoms that provide additional efficiency or robustness. The proposed online algorithms are memory efficient and involve simple updates of the dictionary atoms, sparse coefficients, and images. Numerical experiments demonstrate the usefulness of the proposed methods in inverse problems such as video reconstruction or inpainting from noisy, subsampled pixels, and dynamic magnetic resonance image reconstruction from very limited measurements. △ Less

Submitted 21 July, 2019; v1 submitted 6 September, 2018; originally announced September 2018.

Comments: To appear in IEEE Transactions on Computational Imaging

Panoramic Robust PCA for Foreground-Background Separation on Noisy, Free-Motion Camera Video

Authors: Brian E. Moore, Chen Gao, Raj Rao Nadakuditi

Abstract: This work presents a new robust PCA method for foreground-background separation on freely moving camera video with possible dense and sparse corruptions. Our proposed method registers the frames of the corrupted video and then encodes the varying perspective arising from camera motion as missing data in a global model. This formulation allows our algorithm to produce a panoramic background compone… ▽ More This work presents a new robust PCA method for foreground-background separation on freely moving camera video with possible dense and sparse corruptions. Our proposed method registers the frames of the corrupted video and then encodes the varying perspective arising from camera motion as missing data in a global model. This formulation allows our algorithm to produce a panoramic background component that automatically stitches together corrupted data from partially overlapping frames to reconstruct the full field of view. We model the registered video as the sum of a low-rank component that captures the background, a smooth component that captures the dynamic foreground of the scene, and a sparse component that isolates possible outliers and other sparse corruptions in the video. The low-rank portion of our model is based on a recent low-rank matrix estimator (OptShrink) that has been shown to yield superior low-rank subspace estimates in practice. To estimate the smooth foreground component of our model, we use a weighted total variation framework that enables our method to reliably decouple the true foreground of the video from sparse corruptions. We perform extensive numerical experiments on both static and moving camera video subject to a variety of dense and sparse corruptions. Our experiments demonstrate the state-of-the-art performance of our proposed method compared to existing methods both in terms of foreground and background estimation accuracy. △ Less

Submitted 3 January, 2019; v1 submitted 17 December, 2017; originally announced December 2017.

Comments: IEEE TCI. Project webpage: https://gaochen315.github.io/pRPCA/ Code: https://github.com/gaochen315/panoramicRPCA

Robust Photometric Stereo via Dictionary Learning

Authors: Andrew J. Wagenmaker, Brian E. Moore, Raj Rao Nadakuditi

Abstract: Photometric stereo is a method that seeks to reconstruct the normal vectors of an object from a set of images of the object illuminated under different light sources. While effective in some situations, classical photometric stereo relies on a diffuse surface model that cannot handle objects with complex reflectance patterns, and it is sensitive to non-idealities in the images. In this work, we pr… ▽ More Photometric stereo is a method that seeks to reconstruct the normal vectors of an object from a set of images of the object illuminated under different light sources. While effective in some situations, classical photometric stereo relies on a diffuse surface model that cannot handle objects with complex reflectance patterns, and it is sensitive to non-idealities in the images. In this work, we propose a novel approach to photometric stereo that relies on dictionary learning to produce robust normal vector reconstructions. Specifically, we develop two formulations for applying dictionary learning to photometric stereo. We propose a model that applies dictionary learning to regularize and reconstruct the normal vectors from the images under the classic Lambertian reflectance model. We then generalize this model to explicitly model non-Lambertian objects. We investigate both approaches through extensive experimentation on synthetic and real benchmark datasets and observe state-of-the-art performance compared to existing robust photometric stereo methods. △ Less

Submitted 7 August, 2018; v1 submitted 24 October, 2017; originally announced October 2017.

Comments: To appear in IEEE Transactions on Computational Imaging

Robust Surface Reconstruction from Gradients via Adaptive Dictionary Regularization

Authors: Andrew J. Wagenmaker, Brian E. Moore, Raj Rao Nadakuditi

Abstract: This paper introduces a novel approach to robust surface reconstruction from photometric stereo normal vector maps that is particularly well-suited for reconstructing surfaces from noisy gradients. Specifically, we propose an adaptive dictionary learning based approach that attempts to simultaneously integrate the gradient fields while sparsely representing the spatial patches of the reconstructed… ▽ More This paper introduces a novel approach to robust surface reconstruction from photometric stereo normal vector maps that is particularly well-suited for reconstructing surfaces from noisy gradients. Specifically, we propose an adaptive dictionary learning based approach that attempts to simultaneously integrate the gradient fields while sparsely representing the spatial patches of the reconstructed surface in an adaptive dictionary domain. We show that our formulation learns the underlying structure of the surface, effectively acting as an adaptive regularizer that enforces a smoothness constraint on the reconstructed surface. Our method is general and may be coupled with many existing approaches in the literature to improve the integrity of the reconstructed surfaces. We demonstrate the performance of our method on synthetic data as well as real photometric stereo data and evaluate its robustness to noise. △ Less

Submitted 30 September, 2017; originally announced October 2017.

Comments: ICIP 2017

Robust Photometric Stereo Using Learned Image and Gradient Dictionaries

Authors: Andrew J. Wagenmaker, Brian E. Moore, Raj Rao Nadakuditi

Abstract: Photometric stereo is a method for estimating the normal vectors of an object from images of the object under varying lighting conditions. Motivated by several recent works that extend photometric stereo to more general objects and lighting conditions, we study a new robust approach to photometric stereo that utilizes dictionary learning. Specifically, we propose and analyze two approaches to adap… ▽ More Photometric stereo is a method for estimating the normal vectors of an object from images of the object under varying lighting conditions. Motivated by several recent works that extend photometric stereo to more general objects and lighting conditions, we study a new robust approach to photometric stereo that utilizes dictionary learning. Specifically, we propose and analyze two approaches to adaptive dictionary regularization for the photometric stereo problem. First, we propose an image preprocessing step that utilizes an adaptive dictionary learning model to remove noise and other non-idealities from the image dataset before estimating the normal vectors. We also propose an alternative model where we directly apply the adaptive dictionary regularization to the normal vectors themselves during estimation. We study the practical performance of both methods through extensive simulations, which demonstrate the state-of-the-art performance of both methods in the presence of noise. △ Less

Submitted 30 September, 2017; originally announced October 2017.

Comments: ICIP 2017

Augmented Robust PCA For Foreground-Background Separation on Noisy, Moving Camera Video

Authors: Chen Gao, Brian E. Moore, Raj Rao Nadakuditi

Abstract: This work presents a novel approach for robust PCA with total variation regularization for foreground-background separation and denoising on noisy, moving camera video. Our proposed algorithm registers the raw (possibly corrupted) frames of a video and then jointly processes the registered frames to produce a decomposition of the scene into a low-rank background component that captures the static… ▽ More This work presents a novel approach for robust PCA with total variation regularization for foreground-background separation and denoising on noisy, moving camera video. Our proposed algorithm registers the raw (possibly corrupted) frames of a video and then jointly processes the registered frames to produce a decomposition of the scene into a low-rank background component that captures the static components of the scene, a smooth foreground component that captures the dynamic components of the scene, and a sparse component that can isolate corruptions and other non-idealities. Unlike existing methods, our proposed algorithm produces a panoramic low-rank component that spans the entire field of view, automatically stitching together corrupted data from partially overlapping scenes. The low-rank portion of our robust PCA model is based on a recently discovered optimal low-rank matrix estimator (OptShrink) that requires no parameter tuning. We demonstrate the performance of our algorithm on both static and moving camera videos corrupted by noise and outliers. △ Less

Submitted 27 September, 2017; originally announced September 2017.

Mode Control in a Multimode Fiber Through Acquiring its Transmission Matrix from a Reference-less Optical System

Authors: Moussa N'Gom, Theodore B. Norris, Eric Michielssen, Raj Rao Nadakuditi

Abstract: A simple imaging system together with complex semidefinite programming is used to generate the transmission matrix of a multimode fiber. Once the transmission matrix is acquired, we can modulate the phase of the input signal to induce strong mode interference at the fiber output. The optical design does not contain a reference arm and no interferometric measurements are required. We use a phase-on… ▽ More A simple imaging system together with complex semidefinite programming is used to generate the transmission matrix of a multimode fiber. Once the transmission matrix is acquired, we can modulate the phase of the input signal to induce strong mode interference at the fiber output. The optical design does not contain a reference arm and no interferometric measurements are required. We use a phase-only spatial light modulator to shape the profile of the propagating modes and the output intensity at an individual pixel is monitored. The semidefinite program uses a convex optimization algorithm to generate the transmission matrix of the optical system using intensity only measurements. This simple yet powerful method can be used to compensate for modal dispersion in multimode fiber communication systems. It also yields great promises for the next generation biomedical imaging, quantum communication, and cryptography. △ Less

Submitted 23 October, 2017; v1 submitted 9 July, 2017; originally announced July 2017.

Comments: This manuscript is 9 pages long and contains 7 figures. It has been submitted to Optics Letters for publication

Controlling Light Transmission Through Highly Scattering Media Using Semi-Definite Programming as a Phase Retrieval Computation Method

Authors: Moussa N'Gom, Miao-Bin Lien, Nooshin M. Estakhri, Theodore B. Norris, Eric Michielssen, Raj Rao Nadakuditi

Abstract: Complex Semi-Definite Programming (SDP) is introduced as a novel approach to phase retrieval enabled control of monochromatic light transmission through highly scattering media. In a simple optical setup, a spatial light modulator is used to generate a random sequence of phase-modulated wavefronts, and the resulting intensity speckle patterns in the transmitted light are acquired on a camera. The… ▽ More Complex Semi-Definite Programming (SDP) is introduced as a novel approach to phase retrieval enabled control of monochromatic light transmission through highly scattering media. In a simple optical setup, a spatial light modulator is used to generate a random sequence of phase-modulated wavefronts, and the resulting intensity speckle patterns in the transmitted light are acquired on a camera. The SDP algorithm allows computation of the complex transmission matrix of the system from this sequence of intensity-only measurements, without need for a reference beam. Once the transmission matrix is determined, optimal wavefronts are computed that focus the incident beam to any position or sequence of positions on the far side of the scattering medium, without the need for any subsequent measurements or wavefront shaping iterations. The number of measurements required and the degree of enhancement of the intensity at focus is determined by the number of pixels controlled by the spatial light modulator. △ Less

Submitted 1 May, 2017; v1 submitted 24 December, 2016; originally announced December 2016.

Comments: 10 pages, 7 figures, this article is accepted for publication in Nature Scientific Report

Low-rank and Adaptive Sparse Signal (LASSI) Models for Highly Accelerated Dynamic Imaging

Authors: Saiprasad Ravishankar, Brian E. Moore, Raj Rao Nadakuditi, Jeffrey A. Fessler

Abstract: Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has demonstrated the promise of sparsity and low-rank techniques. For example, the pat… ▽ More Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery from undersampled measurements. In the context of inverse problems in dynamic imaging, recent research has demonstrated the promise of sparsity and low-rank techniques. For example, the patches of the underlying data are modeled as sparse in an adaptive dictionary domain, and the resulting image and dictionary estimation from undersampled measurements is called dictionary-blind compressed sensing, or the dynamic image sequence is modeled as a sum of low-rank and sparse (in some transform domain) components (L+S model) that are estimated from limited measurements. In this work, we investigate a data-adaptive extension of the L+S model, dubbed LASSI, where the temporal image sequence is decomposed into a low-rank component and a component whose spatiotemporal (3D) patches are sparse in some adaptive dictionary domain. We investigate various formulations and efficient methods for jointly estimating the underlying dynamic signal components and the spatiotemporal dictionary from limited measurements. We also obtain efficient sparsity penalized dictionary-blind compressed sensing methods as special cases of our LASSI approaches. Our numerical experiments demonstrate the promising performance of LASSI schemes for dynamic magnetic resonance image reconstruction from limited k-t space data compared to recent methods such as k-t SLR and L+S, and compared to the proposed dictionary-blind compressed sensing method. △ Less

Submitted 9 January, 2017; v1 submitted 12 November, 2016; originally announced November 2016.

Journal ref: IEEE Tr. Med. Imaging 36(5):1116-28 May 2017

Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) - The $\ell_0$ Method

Authors: Saiprasad Ravishankar, Raj Rao Nadakuditi, Jeffrey A. Fessler

Abstract: The sparsity of natural signals and images in a transform domain or dictionary has been extensively exploited in several applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise in many applications compared to fixed or analytical dictionary models. However, dictionary learning problems are typically non-con… ▽ More The sparsity of natural signals and images in a transform domain or dictionary has been extensively exploited in several applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise in many applications compared to fixed or analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. In this work, we investigate an efficient method for $\ell_{0}$ "norm"-based dictionary learning by first approximating the training data set with a sum of sparse rank-one matrices and then using a block coordinate descent approach to estimate the unknowns. The proposed block coordinate descent algorithm involves efficient closed-form solutions. In particular, the sparse coding step involves a simple form of thresholding. We provide a convergence analysis for the proposed block coordinate descent approach. Our numerical experiments show the promising performance and significant speed-ups provided by our method over the classical K-SVD scheme in sparse signal representation and image denoising. △ Less

Submitted 20 April, 2017; v1 submitted 27 November, 2015; originally announced November 2015.

Comments: This work is cited by the IEEE Transactions on Computational Imaging Paper arXiv:1511.06333 (DOI: 10.1109/TCI.2017.2697206)

Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) and Its Application to Inverse Problems

Authors: Saiprasad Ravishankar, Raj Rao Nadakuditi, Jeffrey A. Fessler

Abstract: The sparsity of signals in a transform domain or dictionary has been exploited in applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise compared to analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches fo… ▽ More The sparsity of signals in a transform domain or dictionary has been exploited in applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise compared to analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. This paper exploits the ideas that drive algorithms such as K-SVD, and investigates in detail efficient methods for aggregate sparsity penalized dictionary learning by first approximating the data with a sum of sparse rank-one matrices (outer products) and then using a block coordinate descent approach to estimate the unknowns. The resulting block coordinate descent algorithms involve efficient closed-form solutions. Furthermore, we consider the problem of dictionary-blind image reconstruction, and propose novel and efficient algorithms for adaptive image reconstruction using block coordinate descent and sum of outer products methodologies. We provide a convergence study of the algorithms for dictionary learning and dictionary-blind image reconstruction. Our numerical experiments show the promising performance and speed-ups provided by the proposed methods over previous schemes in sparse data representation and compressed sensing-based image reconstruction. △ Less

Submitted 20 April, 2017; v1 submitted 19 November, 2015; originally announced November 2015.

Comments: Accepted to IEEE Transactions on Computational Imaging. This paper also cites experimental results reported in arXiv:1511.08842

Journal ref: IEEE Transactions on Computational Imaging, 3(4):694-709 Dec 2017

The transmission coefficient distribution of highly scattering sparse random media

Authors: Curtis Jin, Raj Rao Nadakuditi, Eric Michielssen

Abstract: We consider the distribution of the transmission coefficients, i.e. the singular values of the modal transmission matrix, for 2D random media with periodic boundary conditions composed of a large number of point-like non-absorbing scatterers. The scatterers are placed at random locations in the medium and have random refractive indices that are drawn from an arbitrary, known distribution. We const… ▽ More We consider the distribution of the transmission coefficients, i.e. the singular values of the modal transmission matrix, for 2D random media with periodic boundary conditions composed of a large number of point-like non-absorbing scatterers. The scatterers are placed at random locations in the medium and have random refractive indices that are drawn from an arbitrary, known distribution. We construct a randomized model for the scattering matrix that retains scatterer dependent properties essential to reproduce the transmission coefficient distribution and analytically characterize the distribution of this matrix as a function of the refractive index distribution, the number of modes, and the number of scatterers. We show that the derived distribution agrees remarkably well with results obtained using a numerically rigorous spectrally accurate simulation. Analysis of the derived distribution provides the strongest principled justification yet of why we should expect perfect transmission in such random media regardless of the refractive index distribution of the constituent scatterers. The analysis suggests a sparsity condition under which random media will exhibit a perfect transmission-supporting universal transmission coefficient distribution in the deep medium limit. △ Less

Submitted 14 April, 2015; v1 submitted 11 March, 2015; originally announced March 2015.

Comments: PDF contains animation that can be viewed with Adobe Reader. v2 fixes missing bibliography in v1

Sampling unitary invariant ensembles

Authors: Sheehan Olver, Raj Rao Nadakuditi, Thomas Trogdon

Abstract: We develop an algorithm for sampling from the unitary invariant random matrix ensembles. The algorithm is based on the representation of their eigenvalues as a determinantal point process whose kernel is given in terms of orthogonal polynomials. Using this algorithm, statistics beyond those known through analysis are calculable through Monte Carlo simulation. Unexpected phenomena are observed in t… ▽ More We develop an algorithm for sampling from the unitary invariant random matrix ensembles. The algorithm is based on the representation of their eigenvalues as a determinantal point process whose kernel is given in terms of orthogonal polynomials. Using this algorithm, statistics beyond those known through analysis are calculable through Monte Carlo simulation. Unexpected phenomena are observed in the simulations. △ Less

Submitted 31 March, 2014; originally announced April 2014.

MSC Class: 60B20

Batch latency analysis and phase transitions for a tandem of queues with exponentially distributed service times

Authors: Jinho Baik, Raj Rao Nadakuditi

Abstract: We analyze the latency or sojourn time L(m,n) for the last customer in a batch of n customers to exit from the m-th queue in a tandem of m queues in the setting where the queues are in equilibrium before the batch of customers arrives at the first queue. We first characterize the distribution of L(m,n) exactly for every m and n, under the assumption that the queues have unlimited buffers and that… ▽ More We analyze the latency or sojourn time L(m,n) for the last customer in a batch of n customers to exit from the m-th queue in a tandem of m queues in the setting where the queues are in equilibrium before the batch of customers arrives at the first queue. We first characterize the distribution of L(m,n) exactly for every m and n, under the assumption that the queues have unlimited buffers and that each server has customer independent, exponentially distributed service times with an arbitrary, known rate. We then evaluate the first two leading order terms of the distributions in the large m and n limit and bring into sharp focus the existence of phase transitions in the system behavior. The phase transition occurs due to the presence of either slow bottleneck servers or a high external arrival rate. We determine the critical thresholds for the service rate and the arrival rate, respectively, about which this phase transition occurs; it turns out that they are the same. This critical threshold depends, in a manner we make explicit, on the individual service rates, the number of customers and the number of queues but not on the external arrival rate. △ Less

Submitted 10 March, 2014; originally announced March 2014.

Comments: To be publised in Queuing Systems: Theory and Applications

Backscatter analysis based algorithms for increasing transmission through highly-scattering random media using phase-only modulated wavefronts

Authors: Curtis Jin, Raj Rao Nadakuditi, Eric Michielssen, Stephen Rand

Abstract: Recent theoretical and experimental advances have shed light on the existence of so-called `perfectly transmitting' wavefronts with transmission coefficients close to 1 in strongly backscattering random media. These perfectly transmitting eigen-wavefronts can be synthesized by spatial amplitude and phase modulation. Here, we consider the problem of transmission enhancement using phase-only modulat… ▽ More Recent theoretical and experimental advances have shed light on the existence of so-called `perfectly transmitting' wavefronts with transmission coefficients close to 1 in strongly backscattering random media. These perfectly transmitting eigen-wavefronts can be synthesized by spatial amplitude and phase modulation. Here, we consider the problem of transmission enhancement using phase-only modulated wavefronts. We develop physically realizable iterative and non-iterative algorithms for increasing the transmission through such random media using backscatter analysis. We theoretically show that, despite the phase-only modulation constraint, the non-iterative algorithms will achieve at least about 25$π$% or about 78.5% transmission assuming there is at least one perfectly transmitting eigen-wavefront and that the singular vectors of the transmission matrix obey a maximum entropy principle so that they are isotropically random. We numerically analyze the limits of phase-only modulated transmission in 2-D with fully spectrally accurate simulators and provide rigorous numerical evidence confirming our theoretical prediction in random media with periodic boundary conditions that is composed of hundreds of thousands of non-absorbing scatterers. We show via numerical simulations that the iterative algorithms we have developed converge rapidly, yielding highly transmitting wavefronts using relatively few measurements of the backscatter field. Specifically, the best performing iterative algorithm yields approx 70% transmission using just 15-20 measurements in the regime where the non-iterative algorithms yield approximately 78.5% transmission but require measuring the entire modal reflection matrix. △ Less

Submitted 20 June, 2014; v1 submitted 12 February, 2014; originally announced February 2014.

Comments: Revised version contains results of additional simulations

Local Spectrum of Truncations of Kronecker Products of Haar Distributed Unitary Matrices

Authors: Brendan Farrell, Raj Rao Nadakuditi

Abstract: We address the local spectral behavior of the random matrix $Π_1 U^{\otimes k} Π_2 U^{\otimes k *} Π_1$, where $U$ is a Haar distributed unitary matrix of size $n\times n$, the factor $k$ is at most $c_0\log n$ for a small constant $c_0>0$, and $Π_1,Π_2$ are arbitrary projections on $\ell_2^{n^k}$ of ranks proportional to $n^k$. We prove that in this setting the $k$-fold Kronecker product behaves… ▽ More We address the local spectral behavior of the random matrix $Π_1 U^{\otimes k} Π_2 U^{\otimes k *} Π_1$, where $U$ is a Haar distributed unitary matrix of size $n\times n$, the factor $k$ is at most $c_0\log n$ for a small constant $c_0>0$, and $Π_1,Π_2$ are arbitrary projections on $\ell_2^{n^k}$ of ranks proportional to $n^k$. We prove that in this setting the $k$-fold Kronecker product behaves similarly to the well-studied case when $k=1$. △ Less

Submitted 26 November, 2013; originally announced November 2013.

MSC Class: 15B52

Spectra of random graphs with community structure and arbitrary degrees

Authors: Xiao Zhang, Raj Rao Nadakuditi, M. E. J. Newman

Abstract: Using methods from random matrix theory researchers have recently calculated the full spectra of random networks with arbitrary degrees and with community structure. Both reveal interesting spectral features, including deviations from the Wigner semicircle distribution and phase transitions in the spectra of community structured networks. In this paper we generalize both calculations, giving a pre… ▽ More Using methods from random matrix theory researchers have recently calculated the full spectra of random networks with arbitrary degrees and with community structure. Both reveal interesting spectral features, including deviations from the Wigner semicircle distribution and phase transitions in the spectra of community structured networks. In this paper we generalize both calculations, giving a prescription for calculating the spectrum of a network with both community structure and an arbitrary degree distribution. In general the spectrum has two parts, a continuous spectral band, which can depart strongly from the classic semicircle form, and a set of outlying eigenvalues that indicate the presence of communities. △ Less

Submitted 30 September, 2013; originally announced October 2013.

Comments: 9 pages, 3 figures

Journal ref: Phys. Rev. E 89, 042816 (2014)

OptShrink: An algorithm for improved low-rank signal matrix denoising by optimal, data-driven singular value shrinkage

Authors: Raj Rao Nadakuditi

Abstract: The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the_representation_ problem of how to best approximate a noisy measurement matrix using a low-rank matrix. Here, we consider the (unobservable)_denoising_ problem of how to best approximate a low-rank signal matrix buried in noise by optimal (re)weighting of the singular vectors of the measurement… ▽ More The truncated singular value decomposition (SVD) of the measurement matrix is the optimal solution to the_representation_ problem of how to best approximate a noisy measurement matrix using a low-rank matrix. Here, we consider the (unobservable)_denoising_ problem of how to best approximate a low-rank signal matrix buried in noise by optimal (re)weighting of the singular vectors of the measurement matrix. We exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the i.i.d. Gaussian noise case. Our analysis brings into sharp focus the shrinkage-and-thresholding form of the optimal weights, the non-convex nature of the associated shrinkage function (on the singular values) and explains why matrix regularization via singular value thresholding with convex penalty functions (such as the nuclear norm) will always be suboptimal. We validate our theoretical predictions with numerical simulations, develop an implementable algorithm (OptShrink) that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries. △ Less

Submitted 18 April, 2014; v1 submitted 25 June, 2013; originally announced June 2013.

Comments: Published version. The algorithm can be downloaded from http://www.eecs.umich.edu/~rajnrao/optshrink

Journal ref: IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 1-17, May 2014

Iterative, backscatter-analysis algorithms for increasing transmission and focusing light through highly-scattering random media

Authors: Curtis Jin, Raj Rao Nadakuditi, Eric Michielssen, Stephen Rand

Abstract: Scattering hinders the passage of light through random media and consequently limits the usefulness of optical techniques for sensing and imaging. Thus, methods for increasing the transmission of light through such random media are of interest. Against this backdrop, recent theoretical and experimental advances have suggested the existence of a few highly transmitting eigen-wavefronts with transmi… ▽ More Scattering hinders the passage of light through random media and consequently limits the usefulness of optical techniques for sensing and imaging. Thus, methods for increasing the transmission of light through such random media are of interest. Against this backdrop, recent theoretical and experimental advances have suggested the existence of a few highly transmitting eigen-wavefronts with transmission coefficients close to one in strongly backscattering random media. Here, we numerically analyze this phenomenon in 2-D with fully spectrally accurate simulators and provide rigorous numerical evidence confirming the existence of these highly transmitting eigen-wavefronts in random media with periodic boundary conditions that is composed of hundreds of thousands of non-absorbing scatterers. Motivated by bio-imaging applications where it is not possible to measure the transmitted fields, we develop physically realizable algorithms for increasing the transmission through such random media using backscatter analysis. We show via numerical simulations that the algorithms converge rapidly, yielding a near-optimum wavefront in just a few iterations. We also develop an algorithm that combines the knowledge of these highly transmitting eigen-wavefronts obtained from backscatter analysis, with intensity measurements at a point to produce a near-optimal focus with significantly fewer measurements than a method that does not utilize this information. △ Less

Submitted 20 June, 2014; v1 submitted 14 March, 2013; originally announced March 2013.

Comments: Published version

Journal ref: JOSA A, Vol. 30, Issue 8, pp. 1592-1602 (2013)

When are the most informative components for inference also the principal components?

Authors: Raj Rao Nadakuditi

Abstract: Which components of the singular value decomposition of a signal-plus-noise data matrix are most informative for the inferential task of detecting or estimating an embedded low-rank signal matrix? Principal component analysis ascribes greater importance to the components that capture the greatest variation, i.e., the singular vectors associated with the largest singular values. This choice is ofte… ▽ More Which components of the singular value decomposition of a signal-plus-noise data matrix are most informative for the inferential task of detecting or estimating an embedded low-rank signal matrix? Principal component analysis ascribes greater importance to the components that capture the greatest variation, i.e., the singular vectors associated with the largest singular values. This choice is often justified by invoking the Eckart-Young theorem even though that work addresses the problem of how to best represent a signal-plus-noise matrix using a low-rank approximation and not how to best_infer_ the underlying low-rank signal component. Here we take a first-principles approach in which we start with a signal-plus-noise data matrix and show how the spectrum of the noise-only component governs whether the principal or the middle components of the singular value decomposition of the data matrix will be the informative components for inference. Simply put, if the noise spectrum is supported on a connected interval, in a sense we make precise, then the use of the principal components is justified. When the noise spectrum is supported on multiple intervals, then the middle components might be more informative than the principal components. The end result is a proper justification of the use of principal components in the setting where the noise matrix is i.i.d. Gaussian and the identification of scenarios, generically involving heterogeneous noise models such as mixtures of Gaussians, where the middle components might be more informative than the principal components so that they may be exploited to extract additional processing gain. Our results show how the blind use of principal components can lead to suboptimal or even faulty inference because of phase transitions that separate a regime where the principal components are informative from a regime where they are uninformative. △ Less

Submitted 5 February, 2013; originally announced February 2013.

Comments: Submitted to IEEE Transactions on Information Theory

Spectra of random graphs with arbitrary expected degrees

Authors: Raj Rao Nadakuditi, M. E. J. Newman

Abstract: We study random graphs with arbitrary distributions of expected degree and derive expressions for the spectra of their adjacency and modularity matrices. We give a complete prescription for calculating the spectra that is exact in the limit of large network size and large vertex degrees. We also study the effect on the spectra of hubs in the network, vertices of unusually high degree, and show tha… ▽ More We study random graphs with arbitrary distributions of expected degree and derive expressions for the spectra of their adjacency and modularity matrices. We give a complete prescription for calculating the spectra that is exact in the limit of large network size and large vertex degrees. We also study the effect on the spectra of hubs in the network, vertices of unusually high degree, and show that these produce isolated eigenvalues outside the main spectral band, akin to impurity states in condensed matter systems, with accompanying eigenvectors that are strongly localized around the hubs. We also give numerical results that confirm our analytic expressions. △ Less

Submitted 6 August, 2012; originally announced August 2012.

Comments: 14 pages, 5 figures

Journal ref: Phys. Rev. E 87, 012803 (2013)

Graph spectra and the detectability of community structure in networks

Authors: Raj Rao Nadakuditi, M. E. J. Newman

Abstract: We study networks that display community structure -- groups of nodes within which connections are unusually dense. Using methods from random matrix theory, we calculate the spectra of such networks in the limit of large size, and hence demonstrate the presence of a phase transition in matrix methods for community detection, such as the popular modularity maximization method. The transition separa… ▽ More We study networks that display community structure -- groups of nodes within which connections are unusually dense. Using methods from random matrix theory, we calculate the spectra of such networks in the limit of large size, and hence demonstrate the presence of a phase transition in matrix methods for community detection, such as the popular modularity maximization method. The transition separates a regime in which such methods successfully detect the community structure from one in which the structure is present but is not detected. By comparing these results with recent analyses of maximum-likelihood methods we are able to show that spectral modularity maximization is an optimal detection method in the sense that no other method will succeed in the regime where the modularity method fails. △ Less

Submitted 8 May, 2012; originally announced May 2012.

Comments: 5 pages, 2 figures

Journal ref: Phys. Rev. Lett. 108, 188701 (2012)

Numerical computation of convolutions in free probability theory

Authors: Sheehan Olver, Raj Rao Nadakuditi

Abstract: We develop a numerical approach for computing the additive, multiplicative and compressive convolution operations from free probability theory. We utilize the regularity properties of free convolution to identify (pairs of) `admissible' measures whose convolution results in a so-called `invertible measure' which is either a smoothly-decaying measure supported on the entire real line (such as the G… ▽ More We develop a numerical approach for computing the additive, multiplicative and compressive convolution operations from free probability theory. We utilize the regularity properties of free convolution to identify (pairs of) `admissible' measures whose convolution results in a so-called `invertible measure' which is either a smoothly-decaying measure supported on the entire real line (such as the Gaussian) or square-root decaying measure supported on a compact interval (such as the semi-circle). This class of measures is important because these measures along with their Cauchy transforms can be accurately represented via a Fourier or Chebyshev series expansion, respectively. Thus, knowledge of the functional inverse of their Cauchy transform suffices for numerically recovering the invertible measure via a non-standard yet well-behaved Vandermonde system of equations. We describe explicit algorithms for computing the inverse Cauchy transform alluded to and recovering the associated measure with spectral accuracy. Convergence is guaranteed under broad assumptions on the input measures. △ Less

Submitted 19 July, 2013; v1 submitted 8 March, 2012; originally announced March 2012.

MSC Class: 15A52; 46L54; 60F99

The singular values and vectors of low rank perturbations of large rectangular random matrices

Authors: Florent Benaych-Georges, Raj Rao Nadakuditi

Abstract: In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the perturbed matrix. As in the prequel, where we considered the eigenvalue aspect of the problem, the non-random… ▽ More In this paper, we consider the singular values and singular vectors of finite, low rank perturbations of large rectangular random matrices. Specifically, we prove almost sure convergence of the extreme singular values and appropriate projections of the corresponding singular vectors of the perturbed matrix. As in the prequel, where we considered the eigenvalue aspect of the problem, the non-random limiting value is shown to depend explicitly on the limiting singular value distribution of the unperturbed matrix via an integral transforms that linearizes rectangular additive convolution in free probability theory. The large matrix limit of the extreme singular values of the perturbed matrix differs from that of the original matrix if and only if the singular values of the perturbing matrix are above a certain critical threshold which depends on this same aforementioned integral transform. We examine the consequence of this singular value phase transition on the associated left and right singular eigenvectors and discuss the finite $n$ fluctuations above these non-random limits. △ Less

Submitted 26 January, 2012; v1 submitted 11 March, 2011; originally announced March 2011.

Comments: 22 pages, presentation of the main results and of the hypotheses slightly modified

The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices

Authors: Florent Benaych-Georges, Raj Rao Nadakuditi

Abstract: We consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting non-random value is shown to depend explicitly on the limiting eigenvalue d… ▽ More We consider the eigenvalues and eigenvectors of finite, low rank perturbations of random matrices. Specifically, we prove almost sure convergence of the extreme eigenvalues and appropriate projections of the corresponding eigenvectors of the perturbed matrix for additive and multiplicative perturbation models. The limiting non-random value is shown to depend explicitly on the limiting eigenvalue distribution of the unperturbed random matrix and the assumed perturbation model via integral transforms that correspond to very well known objects in free probability theory that linearize non-commutative free additive and multiplicative convolution. Furthermore, we uncover a phase transition phenomenon whereby the large matrix limit of the extreme eigenvalues of the perturbed matrix differs from that of the original matrix if and only if the eigenvalues of the perturbing matrix are above a certain critical threshold. Square root decay of the eigenvalue density at the edge is sufficient to ensure that this threshold is finite. This critical threshold is intimately related to the same aforementioned integral transforms and our proof techniques bring this connection and the origin of the phase transition into focus. Consequently, our results extend the class of `spiked' random matrix models about which such predictions (called the BBP phase transition) can be made well beyond the Wigner, Wishart and Jacobi random ensembles found in the literature. We examine the impact of this eigenvalue phase transition on the associated eigenvectors and observe an analogous phase transition in the eigenvectors. Various extensions of our results to the problem of non-extreme eigenvalues are discussed. △ Less

Submitted 27 December, 2010; v1 submitted 12 October, 2009; originally announced October 2009.

Comments: 27 pages, 1 figure. The paragraph devoted to rectangular matrices has been suppressed in this version (it will appear independently in a forthcoming paper)