Showing 1–42 of 42 results for author: Sengupta, A M

Fast Scrambling at the Boundary

Authors: Ancel Larzul, Anirvan M. Sengupta, Antoine Georges, Marco Schirò

Abstract: Many-body systems which saturate the quantum bound on chaos are attracting interest across a wide range of fields. Notable examples include the Sachdev-Ye-Kitaev model and its variations, all characterised by some form or randomness and all to all couplings. Here we study many-body quantum chaos in a quantum impurity model showing Non-Fermi-Liquid physics, the overscreened multichannel $SU(N)$ Kon… ▽ More Many-body systems which saturate the quantum bound on chaos are attracting interest across a wide range of fields. Notable examples include the Sachdev-Ye-Kitaev model and its variations, all characterised by some form or randomness and all to all couplings. Here we study many-body quantum chaos in a quantum impurity model showing Non-Fermi-Liquid physics, the overscreened multichannel $SU(N)$ Kondo model. We compute exactly the low-temperature behavior of the out-of time order correlator in the limit of large $N$ and large number of channels $K$, at fixed ratio $γ=K/N$. Due to strong correlations at the impurity site the spin fractionalizes in auxiliary fermions and bosons. We show that all the degrees of freedom of our theory acquire a Lyapunov exponent which is linear in temperature as $T\rightarrow 0$, with a prefactor that depends on $γ$. Remarkably, for $N=K$ the impurity spin displays maximal chaos, while bosons and fermions only get up to half of the maximal Lyapunov exponent. Our results highlights two new features: a non-disordered model which is maximally chaotic due to strong correlations at its boundary and a fractionalization of quantum chaos. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: 16 pages, 7 figures

Machine learning-based compression of quantum many body physics: PCA and autoencoder representation of the vertex function

Authors: Jiawei Zang, Matija Medvidović, Dominik Kiese, Domenico Di Sante, Anirvan M. Sengupta, Andrew J. Millis

Abstract: Characterizing complex many-body phases of matter has been a central question in quantum physics for decades. Numerical methods built around approximations of the renormalization group (RG) flow equations have offered reliable and systematically improvable answers to the initial question -- what simple physics drives quantum order and disorder? The flow equations are a very high dimensional set of… ▽ More Characterizing complex many-body phases of matter has been a central question in quantum physics for decades. Numerical methods built around approximations of the renormalization group (RG) flow equations have offered reliable and systematically improvable answers to the initial question -- what simple physics drives quantum order and disorder? The flow equations are a very high dimensional set of coupled nonlinear equations whose solution is the two particle vertex function, a function of three continuous momenta that describes particle-particle scattering and encodes much of the low energy physics including whether the system exhibits various forms of long ranged order. In this work, we take a simple and interpretable data-driven approach to the open question of compressing the two-particle vertex. We use PCA and an autoencoder neural network to derive compact, low-dimensional representations of underlying physics for the case of interacting fermions on a lattice. We quantify errors in the representations by multiple metrics and show that a simple linear PCA offers more physical insight and better out-of-distribution (zero-shot) generalization than the nominally more expressive nonlinear models. Even with a modest number of principal components (10 - 20), we find excellent reconstruction of vertex functions across the phase diagram. This result suggests that many other many-body functions may be similarly compressible, potentially allowing for efficient computation of observables. Finally, we identify principal component subspaces that are shared between known phases, offering new physical insight. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Comments: 4 figures

Compressing the two-particle Green's function using wavelets: Theory and application to the Hubbard atom

Authors: Emin Moghadas, Nikolaus Dräger, Alessandro Toschi, Jiawei Zang, Matija Medvidović, Dominik Kiese, Andrew J. Millis, Anirvan M. Sengupta, Sabine Andergassen, Domenico Di Sante

Abstract: Precise algorithms capable of providing controlled solutions in the presence of strong interactions are transforming the landscape of quantum many-body physics. Particularly exciting breakthroughs are enabling the computation of non-zero temperature correlation functions. However, computational challenges arise due to constraints in resources and memory limitations, especially in scenarios involvi… ▽ More Precise algorithms capable of providing controlled solutions in the presence of strong interactions are transforming the landscape of quantum many-body physics. Particularly exciting breakthroughs are enabling the computation of non-zero temperature correlation functions. However, computational challenges arise due to constraints in resources and memory limitations, especially in scenarios involving complex Green's functions and lattice effects. Leveraging the principles of signal processing and data compression, this paper explores the wavelet decomposition as a versatile and efficient method for obtaining compact and resource-efficient representations of the many-body theory of interacting systems. The effectiveness of the wavelet decomposition is illustrated through its application to the representation of generalized susceptibilities and self-energies in a prototypical interacting fermionic system, namely the Hubbard model at half-filling in its atomic limit. These results are the first proof-of-principle application of the wavelet compression within the realm of many-body physics and demonstrate the potential of this wavelet-based compression scheme for understanding the physics of correlated electron systems. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Comments: 25 pages, 16 figures, 2 tables

Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning

Authors: Yanis Bahroun, Dmitri B. Chklovskii, Anirvan M. Sengupta

Abstract: A normative approach called Similarity Matching was recently introduced for deriving and understanding the algorithmic basis of neural computation focused on unsupervised problems. It involves deriving algorithms from computational objectives and evaluating their compatibility with anatomical and physiological observations. In particular, it introduces neural architectures by considering dual alte… ▽ More A normative approach called Similarity Matching was recently introduced for deriving and understanding the algorithmic basis of neural computation focused on unsupervised problems. It involves deriving algorithms from computational objectives and evaluating their compatibility with anatomical and physiological observations. In particular, it introduces neural architectures by considering dual alternatives instead of primal formulations of popular models such as PCA. However, its connection to the Representer theorem remains unexplored. In this work, we propose to use teachings from this approach to explore supervised learning algorithms and clarify the notion of Hebbian learning. We examine regularized supervised learning and elucidate the emergence of neural architecture and additive versus multiplicative update rules. In this work, we focus not on developing new algorithms but on showing that the Representer theorem offers the perfect lens to study biologically plausible learning algorithms. We argue that many past and current advancements in the field rely on some form of dual formulation to introduce biological plausibility. In short, as long as a dual formulation exists, it is possible to derive biologically plausible algorithms. Our work sheds light on the pivotal role of the Representer theorem in advancing our comprehension of neural computation. △ Less

Submitted 2 August, 2023; originally announced September 2023.

Learning the eigenstructure of quantum dynamics using classical shadows

Authors: Atithi Acharya, Siddhartha Saha, Shagesh Sridharan, Yanis Bahroun, Anirvan M. Sengupta

Abstract: Learning dynamics from repeated observation of the time evolution of an open quantum system, namely, the problem of quantum process tomography is an important task. This task is difficult in general, but, with some additional constraints could be tractable. This motivates us to look at the problem of Lindblad operator discovery from observations. We point out that for moderate size Hilbert spaces,… ▽ More Learning dynamics from repeated observation of the time evolution of an open quantum system, namely, the problem of quantum process tomography is an important task. This task is difficult in general, but, with some additional constraints could be tractable. This motivates us to look at the problem of Lindblad operator discovery from observations. We point out that for moderate size Hilbert spaces, low Kraus rank of the channel, and short time steps, the eigenvalues of the Choi matrix corresponding to the channel have a special structure. We use the least-square method for the estimation of a channel where, for fixed inputs, we estimate the outputs by classical shadows. The resultant noisy estimate of the channel can then be denoised by diagonalizing the nominal Choi matrix, truncating some eigenvalues, and altering it to a genuine Choi matrix. This processed Choi matrix is then compared to the original one. We see that as the number of samples increases, our reconstruction becomes more accurate. We also use tools from random matrix theory to understand the effect of estimation noise in the eigenspectrum of the estimated Choi matrix. △ Less

Submitted 22 September, 2023; originally announced September 2023.

Unlocking the Potential of Similarity Matching: Scalability, Supervision and Pre-training

Authors: Yanis Bahroun, Shagesh Sridharan, Atithi Acharya, Dmitri B. Chklovskii, Anirvan M. Sengupta

Abstract: While effective, the backpropagation (BP) algorithm exhibits limitations in terms of biological plausibility, computational cost, and suitability for online learning. As a result, there has been a growing interest in developing alternative biologically plausible learning approaches that rely on local learning rules. This study focuses on the primarily unsupervised similarity matching (SM) framewor… ▽ More While effective, the backpropagation (BP) algorithm exhibits limitations in terms of biological plausibility, computational cost, and suitability for online learning. As a result, there has been a growing interest in developing alternative biologically plausible learning approaches that rely on local learning rules. This study focuses on the primarily unsupervised similarity matching (SM) framework, which aligns with observed mechanisms in biological systems and offers online, localized, and biologically plausible algorithms. i) To scale SM to large datasets, we propose an implementation of Convolutional Nonnegative SM using PyTorch. ii) We introduce a localized supervised SM objective reminiscent of canonical correlation analysis, facilitating stacking SM layers. iii) We leverage the PyTorch implementation for pre-training architectures such as LeNet and compare the evaluation of features against BP-trained models. This work combines biologically plausible algorithms with computational efficiency opening multiple avenues for further explorations. △ Less

Submitted 2 August, 2023; originally announced August 2023.

Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity

Authors: David Lipshutz, Yanis Bahroun, Siavash Golkar, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: An established normative approach for understanding the algorithmic basis of neural computation is to derive online algorithms from principled computational objectives and evaluate their compatibility with anatomical and physiological observations. Similarity matching objectives have served as successful starting points for deriving online algorithms that map onto neural networks (NNs) with point… ▽ More An established normative approach for understanding the algorithmic basis of neural computation is to derive online algorithms from principled computational objectives and evaluate their compatibility with anatomical and physiological observations. Similarity matching objectives have served as successful starting points for deriving online algorithms that map onto neural networks (NNs) with point neurons and Hebbian/anti-Hebbian plasticity. These NN models account for many anatomical and physiological observations; however, the objectives have limited computational power and the derived NNs do not explain multi-compartmental neuronal structures and non-Hebbian forms of plasticity that are prevalent throughout the brain. In this article, we unify and generalize recent extensions of the similarity matching approach to address more complex objectives, including a large class of unsupervised and self-supervised learning tasks that can be formulated as symmetric generalized eigenvalue problems or nonnegative matrix factorization problems. Interestingly, the online algorithms derived from these objectives naturally map onto NNs with multi-compartmental neurons and local, non-Hebbian learning rules. Therefore, this unified extension of the similarity matching approach provides a normative framework that facilitates understanding multi-compartmental neuronal structures and non-Hebbian plasticity found throughout the brain. △ Less

Submitted 3 August, 2023; v1 submitted 20 February, 2023; originally announced February 2023.

Comments: Added: Figure 1, sections 2, 3

Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchy

Authors: Siavash Golkar, Tiberiu Tesileanu, Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: Predictive coding has emerged as an influential normative model of neural computation, with numerous extensions and applications. As such, much effort has been put into mapping PC faithfully onto the cortex, but there are issues that remain unresolved or controversial. In particular, current implementations often involve separate value and error neurons and require symmetric forward and backward w… ▽ More Predictive coding has emerged as an influential normative model of neural computation, with numerous extensions and applications. As such, much effort has been put into mapping PC faithfully onto the cortex, but there are issues that remain unresolved or controversial. In particular, current implementations often involve separate value and error neurons and require symmetric forward and backward weights across different brain regions. These features have not been experimentally confirmed. In this work, we show that the PC framework in the linear regime can be modified to map faithfully onto the cortical hierarchy in a manner compatible with empirical observations. By employing a disentangling-inspired constraint on hidden-layer neural activities, we derive an upper bound for the PC objective. Optimization of this upper bound leads to an algorithm that shows the same performance as the original objective and maps onto a biologically plausible network. The units of this network can be interpreted as multi-compartmental neurons with non-Hebbian learning rules, with a remarkable resemblance to recent experimental findings. There exist prior models which also capture these features, but they are phenomenological, while our work is a normative derivation. The network we derive does not involve one-to-one connectivity or signal multiplexing, which the phenomenological models required, indicating that these features are not necessary for learning in the cortex. The normative nature of our algorithm in the simplified linear case allows us to prove interesting properties of the framework and analytically understand the computational role of our network's components. The parameters of our network have natural interpretations as physiological quantities in a multi-compartmental model of pyramidal neurons, providing a concrete link between PC and experimental measurements carried out in the cortex. △ Less

Submitted 4 March, 2023; v1 submitted 27 October, 2022; originally announced October 2022.

Comments: 24 pages, 9 figures; accepted in NeurIPS 2022

Deep Learning the Functional Renormalization Group

Authors: Domenico Di Sante, Matija Medvidović, Alessandro Toschi, Giorgio Sangiovanni, Cesare Franchini, Anirvan M. Sengupta, Andrew J. Millis

Abstract: We perform a data-driven dimensionality reduction of the scale-dependent 4-point vertex function characterizing the functional Renormalization Group (fRG) flow for the widely studied two-dimensional $t - t'$ Hubbard model on the square lattice. We demonstrate that a deep learning architecture based on a Neural Ordinary Differential Equation solver in a low-dimensional latent space efficiently lear… ▽ More We perform a data-driven dimensionality reduction of the scale-dependent 4-point vertex function characterizing the functional Renormalization Group (fRG) flow for the widely studied two-dimensional $t - t'$ Hubbard model on the square lattice. We demonstrate that a deep learning architecture based on a Neural Ordinary Differential Equation solver in a low-dimensional latent space efficiently learns the fRG dynamics that delineates the various magnetic and $d$-wave superconducting regimes of the Hubbard model. We further present a Dynamic Mode Decomposition analysis that confirms that a small number of modes are indeed sufficient to capture the fRG dynamics. Our work demonstrates the possibility of using artificial intelligence to extract compact representations of the 4-point vertex functions for correlated electrons, a goal of utmost importance for the success of cutting-edge quantum field theoretical methods for tackling the many-electron problem. △ Less

Submitted 28 March, 2023; v1 submitted 26 February, 2022; originally announced February 2022.

Comments: 6 pages, 5 figures

Journal ref: Phys. Rev. Lett. 129, 136402 (2022)

A Normative and Biologically Plausible Algorithm for Independent Component Analysis

Authors: Yanis Bahroun, Dmitri B Chklovskii, Anirvan M Sengupta

Abstract: The brain effortlessly solves blind source separation (BSS) problems, but the algorithm it uses remains elusive. In signal processing, linear BSS problems are often solved by Independent Component Analysis (ICA). To serve as a model of a biological circuit, the ICA neural network (NN) must satisfy at least the following requirements: 1. The algorithm must operate in the online setting where data s… ▽ More The brain effortlessly solves blind source separation (BSS) problems, but the algorithm it uses remains elusive. In signal processing, linear BSS problems are often solved by Independent Component Analysis (ICA). To serve as a model of a biological circuit, the ICA neural network (NN) must satisfy at least the following requirements: 1. The algorithm must operate in the online setting where data samples are streamed one at a time, and the NN computes the sources on the fly without storing any significant fraction of the data in memory. 2. The synaptic weight update is local, i.e., it depends only on the biophysical variables present in the vicinity of a synapse. Here, we propose a novel objective function for ICA from which we derive a biologically plausible NN, including both the neural architecture and the synaptic learning rules. Interestingly, our algorithm relies on modulating synaptic plasticity by the total activity of the output neurons. In the brain, this could be accomplished by neuromodulators, extracellular calcium, local field potential, or nitric oxide. △ Less

Submitted 16 November, 2021; originally announced November 2021.

Neural optimal feedback control with local learning rules

Authors: Johannes Friedrich, Siavash Golkar, Shiva Farashahi, Alexander Genkin, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the K… ▽ More A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. A prominent framework for addressing such control problems is Optimal Feedback Control (OFC). OFC generates control actions that optimize behaviorally relevant criteria by integrating noisy sensory stimuli and the predictions of an internal model using the Kalman filter or its extensions. However, a satisfactory neural model of Kalman filtering and control is lacking because existing proposals have the following limitations: not considering the delay of sensory feedback, training in alternating phases, and requiring knowledge of the noise covariance matrices, as well as that of systems dynamics. Moreover, the majority of these studies considered Kalman filtering in isolation, and not jointly with control. To address these shortcomings, we introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach (i.e., policy gradient algorithm). We implement this algorithm in a biologically plausible neural network with local synaptic plasticity rules. This network performs system identification and Kalman filtering, without the need for multiple phases with distinct update rules or the knowledge of the noise covariances. It can perform state estimation with delayed sensory feedback, with the help of an internal model. It learns the control policy without requiring any knowledge of the dynamics, thus avoiding the need for weight transport. In this way, our implementation of OFC solves the credit assignment problem needed to produce the appropriate sensory-motor control in the presence of stimulus delay. △ Less

Submitted 12 November, 2021; originally announced November 2021.

Comments: Manuscript and supplementary material of NeurIPS 2021 paper

Informationally complete POVM-based shadow tomography

Authors: Atithi Acharya, Siddhartha Saha, Anirvan M. Sengupta

Abstract: Recently introduced shadow tomography protocols use classical shadows of quantum states to predict many target functions of an unknown quantum state. Unlike full quantum state tomography, shadow tomography does not insist on accurate recovery of the density matrix for high rank mixed states. Yet, such a protocol makes multiple accurate predictions with high confidence, based on a moderate number o… ▽ More Recently introduced shadow tomography protocols use classical shadows of quantum states to predict many target functions of an unknown quantum state. Unlike full quantum state tomography, shadow tomography does not insist on accurate recovery of the density matrix for high rank mixed states. Yet, such a protocol makes multiple accurate predictions with high confidence, based on a moderate number of quantum measurements. One particular influential algorithm, proposed by Huang, Kueng, and Preskill arXiv:2002.08953, requires additional circuits for performing certain random unitary transformations. In this paper, we avoid these transformations but employ an arbitrary informationally complete POVM and show that such a procedure can compute k-bit correlation functions for quantum states reliably. We also show that, for this application, we do not need the median of means procedure of Huang et al. Finally, we discuss the contrast between the computation of correlation functions and fidelity of reconstruction of low rank density matrices. △ Less

Submitted 26 May, 2021; v1 submitted 12 May, 2021; originally announced May 2021.

Comments: Added more references, and a description of numerical computations

Neural circuits for dynamics-based segmentation of time series

Authors: Tiberiu Tesileanu, Siavash Golkar, Samaneh Nasiri, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmen… ▽ More The brain must extract behaviorally relevant latent variables from the signals streamed by the sensory organs. Such latent variables are often encoded in the dynamics that generated the signal rather than in the specific realization of the waveform. Therefore, one problem faced by the brain is to segment time series based on underlying dynamics. We present two algorithms for performing this segmentation task that are biologically plausible, which we define as acting in a streaming setting and all learning rules being local. One algorithm is model-based and can be derived from an optimization problem involving a mixture of autoregressive processes. This algorithm relies on feedback in the form of a prediction error, and can also be used for forecasting future samples. In some brain regions, such as the retina, the feedback connections necessary to use the prediction error for learning are absent. For this case, we propose a second, model-free algorithm that uses a running estimate of the autocorrelation structure of the signal to perform the segmentation. We show that both algorithms do well when tasked with segmenting signals drawn from autoregressive models with piecewise-constant parameters. In particular, the segmentation accuracy is similar to that obtained from oracle-like methods in which the ground-truth parameters of the autoregressive models are known. We also test our methods on datasets generated by alternating snippets of voice recordings. We provide implementations of our algorithms at https://github.com/ttesileanu/bio-time-series. △ Less

Submitted 5 October, 2021; v1 submitted 23 April, 2021; originally announced April 2021.

Comments: v2.1; 34 pages, 14 figures

A Similarity-preserving Neural Network Trained on Transformed Images Recapitulates Salient Features of the Fly Motion Detection Circuit

Authors: Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: Learning to detect content-independent transformations from data is one of the central problems in biological and artificial intelligence. An example of such problem is unsupervised learning of a visual motion detector from pairs of consecutive video frames. Rao and Ruderman formulated this problem in terms of learning infinitesimal transformation operators (Lie group generators) via minimizing im… ▽ More Learning to detect content-independent transformations from data is one of the central problems in biological and artificial intelligence. An example of such problem is unsupervised learning of a visual motion detector from pairs of consecutive video frames. Rao and Ruderman formulated this problem in terms of learning infinitesimal transformation operators (Lie group generators) via minimizing image reconstruction error. Unfortunately, it is difficult to map their model onto a biologically plausible neural network (NN) with local learning rules. Here we propose a biologically plausible model of motion detection. We also adopt the transformation-operator approach but, instead of reconstruction-error minimization, start with a similarity-preserving objective function. An online algorithm that optimizes such an objective function naturally maps onto an NN with biologically plausible learning rules. The trained NN recapitulates major features of the well-studied motion detector in the fly. In particular, it is consistent with the experimental observation that local motion detectors combine information from at least three adjacent pixels, something that contradicts the celebrated Hassenstein-Reichardt model. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Comments: Body and supplementary materials of NeurIPS 2019 paper

A biologically plausible neural network for local supervision in cortical microcircuits

Authors: Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: The backpropagation algorithm is an invaluable tool for training artificial neural networks; however, because of a weight sharing requirement, it does not provide a plausible model of brain function. Here, in the context of a two-layer network, we derive an algorithm for training a neural network which avoids this problem by not requiring explicit error computation and backpropagation. Furthermore… ▽ More The backpropagation algorithm is an invaluable tool for training artificial neural networks; however, because of a weight sharing requirement, it does not provide a plausible model of brain function. Here, in the context of a two-layer network, we derive an algorithm for training a neural network which avoids this problem by not requiring explicit error computation and backpropagation. Furthermore, our algorithm maps onto a neural network that bears a remarkable resemblance to the connectivity structure and learning rules of the cortex. We find that our algorithm empirically performs comparably to backprop on a number of datasets. △ Less

Submitted 30 November, 2020; originally announced November 2020.

Comments: Abstract presented at the NeurIPS 2020 workshop "Beyond Backpropagation". arXiv admin note: text overlap with arXiv:2010.12660

A simple normative network approximates local non-Hebbian learning in the cortex

Authors: Siavash Golkar, David Lipshutz, Yanis Bahroun, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: To guide behavior, the brain extracts relevant features from high-dimensional data streamed by sensory organs. Neuroscience experiments demonstrate that the processing of sensory inputs by cortical neurons is modulated by instructive signals which provide context and task-relevant information. Here, adopting a normative approach, we model these instructive signals as supervisory inputs guiding the… ▽ More To guide behavior, the brain extracts relevant features from high-dimensional data streamed by sensory organs. Neuroscience experiments demonstrate that the processing of sensory inputs by cortical neurons is modulated by instructive signals which provide context and task-relevant information. Here, adopting a normative approach, we model these instructive signals as supervisory inputs guiding the projection of the feedforward data. Mathematically, we start with a family of Reduced-Rank Regression (RRR) objective functions which include Reduced Rank (minimum) Mean Square Error (RRMSE) and Canonical Correlation Analysis (CCA), and derive novel offline and online optimization algorithms, which we call Bio-RRR. The online algorithms can be implemented by neural networks whose synaptic learning rules resemble calcium plateau potential dependent plasticity observed in the cortex. We detail how, in our model, the calcium plateau potential can be interpreted as a backpropagating error signal. We demonstrate that, despite relying exclusively on biologically plausible local learning rules, our algorithms perform competitively with existing implementations of RRMSE and CCA. △ Less

Submitted 23 October, 2020; originally announced October 2020.

Comments: Body and supplementary materials of NeurIPS 2020 paper. 19 pages, 7 figures

A biologically plausible neural network for multi-channel Canonical Correlation Analysis

Authors: David Lipshutz, Yanis Bahroun, Siavash Golkar, Anirvan M. Sengupta, Dmitri B. Chklovskii

Abstract: Cortical pyramidal neurons receive inputs from multiple distinct neural populations and integrate these inputs in separate dendritic compartments. We explore the possibility that cortical microcircuits implement Canonical Correlation Analysis (CCA), an unsupervised learning method that projects the inputs onto a common subspace so as to maximize the correlations between the projections. To this en… ▽ More Cortical pyramidal neurons receive inputs from multiple distinct neural populations and integrate these inputs in separate dendritic compartments. We explore the possibility that cortical microcircuits implement Canonical Correlation Analysis (CCA), an unsupervised learning method that projects the inputs onto a common subspace so as to maximize the correlations between the projections. To this end, we seek a multi-channel CCA algorithm that can be implemented in a biologically plausible neural network. For biological plausibility, we require that the network operates in the online setting and its synaptic update rules are local. Starting from a novel CCA objective function, we derive an online optimization algorithm whose optimization steps can be implemented in a single-layer neural network with multi-compartmental neurons and local non-Hebbian learning rules. We also derive an extension of our online CCA algorithm with adaptive output rank and output whitening. Interestingly, the extension maps onto a neural network whose neural architecture and synaptic updates resemble neural circuitry and synaptic plasticity observed experimentally in cortical pyramidal neurons. △ Less

Submitted 26 March, 2021; v1 submitted 1 October, 2020; originally announced October 2020.

Comments: 46 pages, 14 figures

A Neural Network for Semi-Supervised Learning on Manifolds

Authors: Alexander Genkin, Anirvan M. Sengupta, Dmitri Chklovskii

Abstract: Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that… ▽ More Semi-supervised learning algorithms typically construct a weighted graph of data points to represent a manifold. However, an explicit graph representation is problematic for neural networks operating in the online setting. Here, we propose a feed-forward neural network capable of semi-supervised learning on manifolds without using an explicit graph representation. Our algorithm uses channels that represent localities on the manifold such that correlations between channels represent manifold structure. The proposed neural network has two layers. The first layer learns to build a representation of low-dimensional manifolds in the input data as proposed recently in [8]. The second learns to classify data using both occasional supervision and similarity of the manifold representation of the data. The channel carrying label information for the second layer is assumed to be "silent" most of the time. Learning in both layers is Hebbian, making our network design biologically plausible. We experimentally demonstrate the effect of semi-supervised learning on non-trivial manifolds. △ Less

Submitted 21 August, 2019; originally announced August 2019.

Comments: 12 pages, 4 figures, accepted in ICANN 2019

Journal ref: Artificial Neural Networks and Machine Learning - ICANN 2019 (pp. 375-386). Springer International Publishing

Clustering is semidefinitely not that hard: Nonnegative SDP for manifold disentangling

Authors: Mariano Tepper, Anirvan M. Sengupta, Dmitri Chklovskii

Abstract: In solving hard computational problems, semidefinite program (SDP) relaxations often play an important role because they come with a guarantee of optimality. Here, we focus on a popular semidefinite relaxation of K-means clustering which yields the same solution as the non-convex original formulation for well segregated datasets. We report an unexpected finding: when data contains (greater than ze… ▽ More In solving hard computational problems, semidefinite program (SDP) relaxations often play an important role because they come with a guarantee of optimality. Here, we focus on a popular semidefinite relaxation of K-means clustering which yields the same solution as the non-convex original formulation for well segregated datasets. We report an unexpected finding: when data contains (greater than zero-dimensional) manifolds, the SDP solution captures such geometrical structures. Unlike traditional manifold embedding techniques, our approach does not rely on manually defining a kernel but rather enforces locality via a nonnegativity constraint. We thus call our approach NOnnegative MAnifold Disentangling, or NOMAD. To build an intuitive understanding of its manifold learning capabilities, we develop a theoretical analysis of NOMAD on idealized datasets. While NOMAD is convex and the globally optimal solution can be found by generic SDP solvers with polynomial time complexity, they are too slow for modern datasets. To address this problem, we analyze a non-convex heuristic and present a new, convex and yet efficient, algorithm, based on the conditional gradient method. Our results render NOMAD a versatile, understandable, and powerful tool for manifold learning. △ Less

Submitted 5 September, 2018; v1 submitted 19 June, 2017; originally announced June 2017.

Critical Behavior and Universality Classes for an Algorithmic Phase Transition in Sparse Reconstruction

Authors: Mohammad Ramezanali, Partha P. Mitra, Anirvan M. Sengupta

Abstract: Recovery of an $N$-dimensional, $K$-sparse solution $\mathbf{x}$ from an $M$-dimensional vector of measurements $\mathbf{y}$ for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost $||\mathbf{y}-\mathbf{H} \mathbf{x}||_2^2+λV(\mathbf{x})$. Here $\mathbf{H}$ is a known matrix and $V(\mathbf{x})$ is an algorithm-dependent sparsity-inducing pen… ▽ More Recovery of an $N$-dimensional, $K$-sparse solution $\mathbf{x}$ from an $M$-dimensional vector of measurements $\mathbf{y}$ for multivariate linear regression can be accomplished by minimizing a suitably penalized least-mean-square cost $||\mathbf{y}-\mathbf{H} \mathbf{x}||_2^2+λV(\mathbf{x})$. Here $\mathbf{H}$ is a known matrix and $V(\mathbf{x})$ is an algorithm-dependent sparsity-inducing penalty. For `random' $\mathbf{H}$, in the limit $λ\rightarrow 0$ and $M,N,K\rightarrow \infty$, keeping $ρ=K/N$ and $α=M/N$ fixed, exact recovery is possible for $α$ past a critical value $α_c = α(ρ)$. Assuming $\mathbf{x}$ has iid entries, the critical curve exhibits some universality, in that its shape does not depend on the distribution of $\mathbf{x}$. However, the algorithmic phase transition occurring at $α=α_c$ and associated universality classes remain ill-understood from a statistical physics perspective, i.e. in terms of scaling exponents near the critical curve. In this article, we analyze the mean-field equations for two algorithms, Basis Pursuit ($V(\mathbf{x})=||\mathbf{x}||_{1} $) and Elastic Net ($V(\mathbf{x})= ||\mathbf{x}||_{1} + \tfrac{g}{2} ||\mathbf{x}||_{2}^2$) and show that they belong to different universality classes in the sense of scaling exponents, with Mean Squared Error (MSE) of the recovered vector scaling as $λ^\frac{4}{3}$ and $λ$ respectively, for small $λ$ on the critical line. In the presence of additive noise, we find that, when $α>α_c$, MSE is minimized at a non-zero value for $λ$, whereas at $α=α_c$, MSE always increases with $λ$. △ Less

Submitted 28 October, 2019; v1 submitted 29 September, 2015; originally announced September 2015.

Comments: 18 pages, 8 figures, 3 tables

Journal ref: J Stat Phys (2019) 175: 764

The cavity method for analysis of large-scale penalized regression

Authors: Mohammad Ramezanali, Partha P. Mitra, Anirvan M. Sengupta

Abstract: Penalized regression methods aim to retrieve reliable predictors among a large set of putative ones from a limited amount of measurements. In particular, penalized regression with singular penalty functions is important for sparse reconstruction algorithms. For large-scale problems, these algorithms exhibit sharp phase transition boundaries where sparse retrieval breaks down. Large optimization pr… ▽ More Penalized regression methods aim to retrieve reliable predictors among a large set of putative ones from a limited amount of measurements. In particular, penalized regression with singular penalty functions is important for sparse reconstruction algorithms. For large-scale problems, these algorithms exhibit sharp phase transition boundaries where sparse retrieval breaks down. Large optimization problems associated with sparse reconstruction have been analyzed in the literature by setting up corresponding statistical mechanical models at a finite temperature. Using replica method for mean field approximation, and subsequently taking a zero temperature limit, this approach reproduces the algorithmic phase transition boundaries. Unfortunately, the replica trick and the non-trivial zero temperature limit obscure the underlying reasons for the failure of a sparse reconstruction algorithm, and of penalized regression methods, in general. In this paper, we employ the ``cavity method'' to give an alternative derivation of the mean field equations, working directly in the zero-temperature limit. This derivation provides insight into the origin of the different terms in the self-consistency conditions. The cavity method naturally involves a quantity, the average local susceptibility, whose behavior distinguishes different phases in this system. This susceptibility can be generalized for analysis of a broader class of sparse reconstruction algorithms. △ Less

Submitted 25 November, 2015; v1 submitted 13 January, 2015; originally announced January 2015.

Comments: 15 pages, 7 figures, new version focusing only on cavity method, title change

Titration and hysteresis in epigenetic chromatin silencing

Authors: Adel Dayarian, Anirvan M. Sengupta

Abstract: Epigenetic mechanisms of silencing via heritable chromatin modifications play a major role in gene regulation and cell fate specification. We consider a model of epigenetic chromatin silencing in budding yeast and study the bifurcation diagram and characterize the bistable and the monostable regimes. The main focus of this paper is to examine how the perturbations altering the activity of histone… ▽ More Epigenetic mechanisms of silencing via heritable chromatin modifications play a major role in gene regulation and cell fate specification. We consider a model of epigenetic chromatin silencing in budding yeast and study the bifurcation diagram and characterize the bistable and the monostable regimes. The main focus of this paper is to examine how the perturbations altering the activity of histone modifying enzymes affect the epigenetic states. We analyze the implications of having the total number of silencing proteins given by the sum of proteins bound to the nucleosomes and the ones available in the ambient to be constant. This constraint couples different regions of chromatin through the shared reservoir of ambient silencing proteins. We show that the response of the system to perturbations depends dramatically on the titration effect caused by the above constraint. In particular, for a certain range of overall abundance of silencing proteins, the hysteresis loop changes qualitatively with certain jump replaced by continuous merger of different states. In addition, we find a nonmonotonic dependence of gene expression on the rate of histone deacetylation activity of Sir2. We discuss how these qualitative predictions of our model could be compared with experimental studies of the yeast system under anti-silencing drugs. △ Less

Submitted 14 February, 2013; v1 submitted 17 September, 2012; originally announced September 2012.

Report number: NSF-KITP-12-172

Journal ref: 2013 Phys. Biol. 10 036005

What does the Allen Gene Expression Atlas tell us about mouse brain evolution?

Authors: Swagatam Mukhopadhyay, Pascal Grange, Anirvan M. Sengupta, Partha P. Mitra

Abstract: We use the Allen Gene Expression Atlas (AGEA) and the OMA ortholog dataset to investigate the evolution of mouse-brain neuroanatomy from the standpoint of the molecular evolution of brain-specific genes. For each such gene, using the phylogenetic tree for all fully sequenced species and the presence of orthologs of the gene in these species, we construct and assign a discrete measure of evolutiona… ▽ More We use the Allen Gene Expression Atlas (AGEA) and the OMA ortholog dataset to investigate the evolution of mouse-brain neuroanatomy from the standpoint of the molecular evolution of brain-specific genes. For each such gene, using the phylogenetic tree for all fully sequenced species and the presence of orthologs of the gene in these species, we construct and assign a discrete measure of evolutionary age. The gene expression profile of all gene of similar age, relative to the average gene expression profile, distinguish regions of the brain that are over-represented in the corresponding evolutionary timescale. We argue that the conclusions one can draw on evolution of twelve major brain regions from such a molecular level analysis supplements existing knowledge of mouse brain evolution and introduces new quantitative tools, especially for comparative studies, when AGEA-like data sets for other species become available. Using the functional role of the genes representational of a certain evolutionary timescale and brain region we compare and contrast, wherever possible, our observations with existing knowledge in evolutionary neuroanatomy. △ Less

Submitted 1 June, 2012; originally announced June 2012.

Comments: 14 pages

The role of multiple marks in epigenetic silencing and the emergence of a stable bivalent chromatin state

Authors: Swagatam Mukhopadhyay, Anirvan M. Sengupta

Abstract: We introduce and analyze a minimal model of epigenetic silencing in budding yeast, built upon known biomolecular interactions in the system. Doing so, we identify the epigenetic marks essential for the bistability of epigenetic states. The model explicitly incorporates two key chromatin marks, namely H4K16 acetylation and H3K79 methylation, and explores whether the presence of multiple marks lead… ▽ More We introduce and analyze a minimal model of epigenetic silencing in budding yeast, built upon known biomolecular interactions in the system. Doing so, we identify the epigenetic marks essential for the bistability of epigenetic states. The model explicitly incorporates two key chromatin marks, namely H4K16 acetylation and H3K79 methylation, and explores whether the presence of multiple marks lead to a qualitatively different systems behavior. We find that having both modifications is important for the robustness of epigenetic silencing. Besides the silenced and transcriptionally active fate of chromatin, our model leads to a novel state with bivalent (i.e., both active and silencing) marks under certain perturbations (knock-out mutations, inhibition or enhancement of enzymatic activity). The bivalent state appears under several perturbations and is shown to result in patchy silencing. We also show that the titration effect, owing to a limited supply of silencing proteins, can result in counter-intuitive responses. The design principles of the silencing system is systematically investigated and disparate experimental observations are assessed within a single theoretical framework. Specifically, we discuss the behavior of Sir protein recruitment, spreading and stability of silenced regions in commonly-studied mutants (e.g., sas2, dot1) illuminating the controversial role of Dot1 in the systems biology of yeast silencing. △ Less

Submitted 5 October, 2013; v1 submitted 11 May, 2012; originally announced May 2012.

Comments: Supplementary Material, 14 pages

Journal ref: PLoS Comput Biol 9(7): e1003121

SLIQ: Simple Linear Inequalities for Efficient Contig Scaffolding

Authors: Rajat S. Roy, Kevin C. Chen, Anirvan M. Sengupta, Alexander Schliep

Abstract: Scaffolding is an important subproblem in "de novo" genome assembly in which mate pair data are used to construct a linear sequence of contigs separated by gaps. Here we present SLIQ, a set of simple linear inequalities derived from the geometry of contigs on the line that can be used to predict the relative positions and orientations of contigs from individual mate pair reads and thus produce a c… ▽ More Scaffolding is an important subproblem in "de novo" genome assembly in which mate pair data are used to construct a linear sequence of contigs separated by gaps. Here we present SLIQ, a set of simple linear inequalities derived from the geometry of contigs on the line that can be used to predict the relative positions and orientations of contigs from individual mate pair reads and thus produce a contig digraph. The SLIQ inequalities can also filter out unreliable mate pairs and can be used as a preprocessing step for any scaffolding algorithm. We tested the SLIQ inequalities on five real data sets ranging in complexity from simple bacterial genomes to complex mammalian genomes and compared the results to the majority voting procedure used by many other scaffolding algorithms. SLIQ predicted the relative positions and orientations of the contigs with high accuracy in all cases and gave more accurate position predictions than majority voting for complex genomes, in particular the human genome. Finally, we present a simple scaffolding algorithm that produces linear scaffolds given a contig digraph. We show that our algorithm is very efficient compared to other scaffolding algorithms while maintaining high accuracy in predicting both contig positions and orientations for real data sets. △ Less

Submitted 9 November, 2011; v1 submitted 6 November, 2011; originally announced November 2011.

Comments: 16 pages, 6 figures, 7 tables

Theoretical analysis of the role of chromatin interactions in long-range action of enhancers and insulators

Authors: Swagatam Mukhopadhyay, Paul D. Schedl, Vasily M. Studitsky, Anirvan M. Sengupta

Abstract: Long-distance regulatory interactions between enhancers and their target genes are commonplace in higher eukaryotes. Interposed boundaries or insulators are able to block these long distance regulatory interactions. The mechanistic basis for insulator activity and how it relates to enhancer action-at-a-distance remains unclear. Here we explore the idea that topological loops could simultaneously a… ▽ More Long-distance regulatory interactions between enhancers and their target genes are commonplace in higher eukaryotes. Interposed boundaries or insulators are able to block these long distance regulatory interactions. The mechanistic basis for insulator activity and how it relates to enhancer action-at-a-distance remains unclear. Here we explore the idea that topological loops could simultaneously account for regulatory interactions of distal enhancers and the insulating activity of boundary elements. We show that while loop formation is not in itself sufficient to explain action at a distance, incorporating transient non-specific and moderate attractive interactions between the chromatin fibers strongly enhances long-distance regulatory interactions and is sufficient to generate a euchromatin-like state. Under these same conditions, the subdivision of the loop into two topologically independent loops by insulators inhibits inter-domain interactions. The underlying cause of this effect is a suppression of crossings in the contact map at intermediate distances. Thus our model simultaneously accounts for regulatory interactions at a distance and the insulator activity of boundary elements. This unified model of the regulatory roles of chromatin loops makes several testable predictions that could be confronted with \emph{in vitro} experiments, as well as genomic chromatin conformation capture and fluorescent microscopic approaches. △ Less

Submitted 14 March, 2011; originally announced March 2011.

Comments: 10 pages, originally submitted to an (undisclosed) journal in May 2010

Statistical mechanics of transcription-factor binding site discovery using Hidden Markov Models

Authors: Pankaj Mehta, David Schwab, Anirvan M. Sengupta

Abstract: Hidden Markov Models (HMMs) are a commonly used tool for inference of transcription factor (TF) binding sites from DNA sequence data. We exploit the mathematical equivalence between HMMs for TF binding and the "inverse" statistical mechanics of hard rods in a one-dimensional disordered potential to investigate learning in HMMs. We derive analytic expressions for the Fisher information, a commonly… ▽ More Hidden Markov Models (HMMs) are a commonly used tool for inference of transcription factor (TF) binding sites from DNA sequence data. We exploit the mathematical equivalence between HMMs for TF binding and the "inverse" statistical mechanics of hard rods in a one-dimensional disordered potential to investigate learning in HMMs. We derive analytic expressions for the Fisher information, a commonly employed measure of confidence in learned parameters, in the biologically relevant limit where the density of binding sites is low. We then use techniques from statistical mechanics to derive a scaling principle relating the specificity (binding energy) of a TF to the minimum amount of training data necessary to learn it. △ Less

Submitted 27 October, 2010; v1 submitted 18 August, 2010; originally announced August 2010.

Comments: 25 pages, 2 figures, 1 table V2 - typos fixed and new references added

Shape, size and robustness: feasible regions in the parameter space of biochemical networks

Authors: Madalena Chave, Eduardo D. Sontag, Anirvan M. Sengupta

Abstract: The concept of robustness of regulatory networks has been closely related to the nature of the interactions among genes, and the capability of pattern maintenance or reproducibility. Defining this robustness property is a challenging task, but mathematical models have often associated it to the volume of the space of admissible parameters. Not only the volume of the space but also its topology a… ▽ More The concept of robustness of regulatory networks has been closely related to the nature of the interactions among genes, and the capability of pattern maintenance or reproducibility. Defining this robustness property is a challenging task, but mathematical models have often associated it to the volume of the space of admissible parameters. Not only the volume of the space but also its topology and geometry contain information on essential aspects of the network, including feasible pathways, switching between two parallel pathways or distinct/disconnected active regions of parameters. A general method is presented here to characterize the space of admissible parameters, by writing it as a semi-algebraic set, and then theoretically analyzing its topology and geometry, as well as volume. This method provides a more objective and complete measure of the robustness of a developmental module. As an illustration, the segment polarity gene network is analyzed. △ Less

Submitted 23 October, 2007; originally announced October 2007.

Comments: 38 pages, 6 figure

Epigenetic Chromatin Silencing: Bistability and Front Propagation

Authors: Mohammad Sedighi, Anirvan M. Sengupta

Abstract: The role of post-translational modification of histones in eukaryotic gene regulation is well recognized. Epigenetic silencing of genes via heritable chromatin modifications plays a major role in cell fate specification in higher organisms. We formulate a coarse-grained model of chromatin silencing in yeast and study the conditions under which the system becomes bistable, allowing for different… ▽ More The role of post-translational modification of histones in eukaryotic gene regulation is well recognized. Epigenetic silencing of genes via heritable chromatin modifications plays a major role in cell fate specification in higher organisms. We formulate a coarse-grained model of chromatin silencing in yeast and study the conditions under which the system becomes bistable, allowing for different epigenetic states. We also study the dynamics of the boundary between the two locally stable states of chromatin: silenced and unsilenced. The model could be of use in guiding the discussion on chromatin silencing in general. In the context of silencing in budding yeast, it helps us understand the phenotype of various mutants, some of which may be non-trivial to see without the help of a mathematical model. One such example is a mutation that reduces the rate of background acetylation of particular histone side-chains that competes with the deacetylation by Sir2p. The resulting negative feedback due to a Sir protein depletion effect gives rise to interesting counter-intuitive consequences. Our mathematical analysis brings forth the different dynamical behaviors possible within the same molecular model and guides the formulation of more refined hypotheses that could be addressed experimentally. △ Less

Submitted 22 October, 2007; originally announced October 2007.

Comments: 19 pages, 5 figures

Quantitative modeling and data analysis of SELEX experiments

Authors: Marko Djordjevic, Anirvan M. Sengupta

Abstract: SELEX (Systematic Evolution of Ligands by Exponential Enrichment) is an experimental procedure that allows extracting, from an initially random pool of DNA, those oligomers with high affinity for a given DNA-binding protein. We address what is a suitable experimental and computational procedure to infer parameters of transcription factor-DNA interaction from SELEX experiments. To answer this, we… ▽ More SELEX (Systematic Evolution of Ligands by Exponential Enrichment) is an experimental procedure that allows extracting, from an initially random pool of DNA, those oligomers with high affinity for a given DNA-binding protein. We address what is a suitable experimental and computational procedure to infer parameters of transcription factor-DNA interaction from SELEX experiments. To answer this, we use a biophysical model of transcription factor-DNA interactions to quantitatively model SELEX. We show that a standard procedure is unsuitable for obtaining accurate interaction parameters. However, we theoretically show that a modified experiment in which chemical potential is fixed through different rounds of the experiment allows robust generation of an appropriate data set. Based on our quantitative model, we propose a novel bioinformatic method of data analysis for such modified experiment and apply it to extract the interaction parameters for a mammalian transcription factor CTF/NFI. From a practical point of view, our method results in a significantly improved false positive/false negative trade-off, as compared to both the standard information theory based method and a widely used empirically formulated procedure. △ Less

Submitted 30 November, 2005; originally announced December 2005.

Comments: 29 pages, 8 figures, to appear in Physical Biology

Optimal Path to Epigenetic Switching

Authors: David Marin Roma, Ruadhan A. O'Flanagan, Andrei E. Ruckenstein, Anirvan M. Sengupta, Ranjan Mukhopadhyay

Abstract: We use large deviation methods to calculate rates of noise-induced transitions between states in multistable genetic networks. We analyze a synthetic biochemical circuit, the toggle switch, and compare the results to those obtained from a numerical solution of the master equation. We use large deviation methods to calculate rates of noise-induced transitions between states in multistable genetic networks. We analyze a synthetic biochemical circuit, the toggle switch, and compare the results to those obtained from a numerical solution of the master equation. △ Less

Submitted 22 November, 2004; v1 submitted 2 June, 2004; originally announced June 2004.

Comments: 5 pages. 2 figures, uses revtex 4. PR-E reviewed for publication

Capacity of multivariate channels with multiplicative noise: I.Random matrix techniques and large-N expansions for full transfer matrices

Authors: Anirvan Mayukh Sengupta, Partha Pratim Mitra

Abstract: We study memoryless, discrete time, matrix channels with additive white Gaussian noise and input power constraints of the form $Y_i = \sum_j H_{ij} X_j + Z_i$, where $Y_i$ ,$X_j$ and $Z_i$ are complex, $i=1..m$, $j=1..n$, and $H$ is a complex $m\times n$ matrix with some degree of randomness in its entries. The additive Gaussian noise vector is assumed to have uncorrelated entries. Let $H$ be a… ▽ More We study memoryless, discrete time, matrix channels with additive white Gaussian noise and input power constraints of the form $Y_i = \sum_j H_{ij} X_j + Z_i$, where $Y_i$ ,$X_j$ and $Z_i$ are complex, $i=1..m$, $j=1..n$, and $H$ is a complex $m\times n$ matrix with some degree of randomness in its entries. The additive Gaussian noise vector is assumed to have uncorrelated entries. Let $H$ be a full matrix (non-sparse) with pairwise correlations between matrix entries of the form $ E[H_{ik} H^*_{jl}] = {1\over n}C_{ij} D_{kl} $, where $C$,$D$ are positive definite Hermitian matrices. Simplicities arise in the limit of large matrix sizes (the so called large-N limit) which allow us to obtain several exact expressions relating to the channel capacity. We study the probability distribution of the quantity $ f(H) = \log \det (1+P H^{\dagger}S H) $. $S$ is non-negative definite and hermitian, with $Tr S=n$. Note that the expectation $E[f(H)]$, maximised over $S$, gives the capacity of the above channel with an input power constraint in the case $H$ is known at the receiver but not at the transmitter. For arbitrary $C$,$D$ exact expressions are obtained for the expectation and variance of $f(H)$ in the large matrix size limit. For $C=D=I$, where $I$ is the identity matrix, expressions are in addition obtained for the full moment generating function for arbitrary (finite) matrix size in the large signal to noise limit. Finally, we obtain the channel capacity where the channel matrix is partly known and partly unknown and of the form $αI+ βH$, $α,β$ being known constants and entries of $H$ i.i.d. Gaussian with variance $1/n$. Channels of the form described above are of interest for wireless transmission with multiple antennae and receivers. △ Less

Submitted 31 October, 2000; originally announced October 2000.

Comments: 15 pages, 1 figure in postscript format

Communication through a Diffusive Medium: Coherence and Capacity

Authors: Aris L. Moustakas, Harold U. Baranger, Leon Balents, Anirvan M. Sengupta, Steven H. Simon

Abstract: Coherent wave propagation in disordered media gives rise to many fascinating phenomena as diverse as universal conductance fluctuations in mesoscopic metals and speckle patterns in light scattering. Here, the theory of electromagnetic wave propagation in diffusive media is combined with information theory to show how interference affects the information transmission rate between antenna arrays.… ▽ More Coherent wave propagation in disordered media gives rise to many fascinating phenomena as diverse as universal conductance fluctuations in mesoscopic metals and speckle patterns in light scattering. Here, the theory of electromagnetic wave propagation in diffusive media is combined with information theory to show how interference affects the information transmission rate between antenna arrays. Nontrivial dependencies of the information capacity on the nature of the antenna arrays are found, such as the dimensionality of the arrays and their direction with respect to the local scattering medium. This approach provides a physical picture for understanding the importance of scattering in the transfer of information through wireless communications. △ Less

Submitted 6 September, 2000; originally announced September 2000.

Comments: 5 pages, 3 figures

Journal ref: Science vol 287, p. 287 (2000)

Hall effect in the perovskite manganites

Authors: Pinaki Majumdar, Steven H. Simon, Anirvan M. Sengupta

Abstract: We compute the zero temperature phase diagram and the Hall response of the doped perovskite manganites within the model of double exchange and Jahn-Teller coupling, employing the large dimensionality approximation proposed by Millis et al. We find that in this scenario a ``hole - like'' $R_H$ for the hole doped manganites, as observed at low temperature by Matl et al., can be obtained only when… ▽ More We compute the zero temperature phase diagram and the Hall response of the doped perovskite manganites within the model of double exchange and Jahn-Teller coupling, employing the large dimensionality approximation proposed by Millis et al. We find that in this scenario a ``hole - like'' $R_H$ for the hole doped manganites, as observed at low temperature by Matl et al., can be obtained only when lattice distortions persist in the metallic state at zero temperature. The calculated temperature dependence of $R_H$ seems to be consistent with the measured ``normal'' Hall coefficient in thin films. △ Less

Submitted 1 July, 1998; originally announced July 1998.

Comments: 5 pages, latex, + 3 eps figures

Distributions of Singular Values for Some Random Matrices

Authors: A. M. Sengupta, P. P. Mitra

Abstract: The Singular Value Decomposition is a matrix decomposition technique widely used in the analysis of multivariate data, such as complex space-time images obtained in both physical and biological systems. In this paper, we examine the distribution of Singular Values of low rank matrices corrupted by additive noise. Past studies have been limited to uniform uncorrelated noise. Using diagrammatic an… ▽ More The Singular Value Decomposition is a matrix decomposition technique widely used in the analysis of multivariate data, such as complex space-time images obtained in both physical and biological systems. In this paper, we examine the distribution of Singular Values of low rank matrices corrupted by additive noise. Past studies have been limited to uniform uncorrelated noise. Using diagrammatic and saddle point integration techniques, we extend these results to heterogeneous and correlated noise sources. We also provide perturbative estimates of error bars on the reconstructed low rank matrix obtained by truncating a Singular Value Decomposition. △ Less

Submitted 25 September, 1997; originally announced September 1997.

Comments: 8 pages, Latex, uses psfig

Spin in a Fluctuating Field: The Bose (+Fermi) Kondo models

Authors: Anirvan M. Sengupta

Abstract: I consider models with an impurity spin coupled to a fluctuating gaussian field with or without additional Kondo coupling of the conventional sort. In the case of isotropic fluctuations, the renormalisation group flows for these models have controlled fixed points when the autocorrelation of the gaussian field $h(t)$, $<T h(t) h(0)> \sim {1\over t^{2-ε}}$ with small positive $ε$. In absence of a… ▽ More I consider models with an impurity spin coupled to a fluctuating gaussian field with or without additional Kondo coupling of the conventional sort. In the case of isotropic fluctuations, the renormalisation group flows for these models have controlled fixed points when the autocorrelation of the gaussian field $h(t)$, $<T h(t) h(0)> \sim {1\over t^{2-ε}}$ with small positive $ε$. In absence of any additional Kondo coupling, I get powerlaw decay of spin correlators, $<T S(t) S(0)> \sim {1\over t^ε}$ . For negative $ε$, the spin autocorrelation is constant in long time limit. The results agree with calculations in Schwinger Boson mean field theory. In presence of a Kondo coupling to itinerant electrons, the model shows a phase transition from a Kondo phase to a field fluctuation dominated phase. These models are good starting points for understanding behaviour of impurities in a system near a zero-temperature magnetic transition. They are also useful for understanding the dynamical local mean field theory of Kondo lattice with Heisenberg (spin-glass type) magnetic interactions. △ Less

Submitted 30 July, 1997; originally announced July 1997.

Comments: 4 pages, 2 eps figures (uses epsfig)

Overscreened Single Channel Kondo Problem

Authors: Anirvan M. Sengupta, Yong Baek Kim

Abstract: We consider the single channel Kondo problem with the Kondo coupling between a spin $S$ impurity and conduction electrons with spin $j$. These problems arise as multicritical points in the parameter spaces of two- and higher-level tunneling systems, and some impurity models of heavy fermion compounds. In contrast to the previous Bethe-anstaz conjectures, it turns out that the dynamics of the spi… ▽ More We consider the single channel Kondo problem with the Kondo coupling between a spin $S$ impurity and conduction electrons with spin $j$. These problems arise as multicritical points in the parameter spaces of two- and higher-level tunneling systems, and some impurity models of heavy fermion compounds. In contrast to the previous Bethe-anstaz conjectures, it turns out that the dynamics of the spin sector is the same as that of a spin $S$ impurity coupled to $k(j)$ channels of spin $1/2$ electrons with $k(j) = 2j(j+1)(2j+1)/3$. As a result, for $2S < k(j)$, the system shows non-Fermi liquid behavior with the same exponents for the thermodynamic quantities as those of $k(j)$ channel Kondo problem. However, both the finite-size spectrum and the operator content are different due to the presence of the other sectors and can be obtained by conformal field theory techniques. △ Less

Submitted 12 June, 1996; v1 submitted 19 February, 1996; originally announced February 1996.

Comments: 4 pages, revtex, no figures. Revised Version

Non-equilibrium Dynamics Following a Quench to the Critical Point in a Semi-infinite System

Authors: Satya N. Majumdar, Anirvan M. Sengupta

Abstract: We study the non-equilibrium dynamics (purely dissipative and relaxational) in a semi-infinite system following a quench from the high temperature disordered phase to its critical temperature. We show that the local autocorrelation near the surface of a semi-infinite system decays algebraically in time with a new exponent which is different from the bulk. We calculate this new non-equilibrium su… ▽ More We study the non-equilibrium dynamics (purely dissipative and relaxational) in a semi-infinite system following a quench from the high temperature disordered phase to its critical temperature. We show that the local autocorrelation near the surface of a semi-infinite system decays algebraically in time with a new exponent which is different from the bulk. We calculate this new non-equilibrium surface exponent in several cases, both analytically and numerically. △ Less

Submitted 12 June, 1995; originally announced June 1995.

Comments: revtex, 9 pages, 2 figures available from the authors

Non Fermi Liquid Behaviour near a $T=0$ spin-glass transition

Authors: Anirvan M. Sengupta, Antoine Georges

Abstract: In this paper we study the competition between the Kondo effect and RKKY interactions near the zero-temperature quantum critical point of an Ising-like metallic spin-glass. We consider the mean-field behaviour of various physical quantities. In the `quantum- critical regime' non-analytic corrections to the Fermi liquid behaviour are found for the specific heat and uniform static susceptibility,… ▽ More In this paper we study the competition between the Kondo effect and RKKY interactions near the zero-temperature quantum critical point of an Ising-like metallic spin-glass. We consider the mean-field behaviour of various physical quantities. In the `quantum- critical regime' non-analytic corrections to the Fermi liquid behaviour are found for the specific heat and uniform static susceptibility, while the resistivity and NMR relaxation rate have a non-Fermi liquid dependence on temperature. △ Less

Submitted 28 April, 1995; originally announced April 1995.

Comments: 15 pages, RevTex 3.0, 1 uuencoded ps. figure at the end

On the Emery-Kivelson Solution of the two channel Kondo problem

Authors: Anirvan M. Sengupta, Antoine Georges

Abstract: We consider the two channel Kondo model in the Emery-Kivelson approach, and calculate the total susceptibility enhancement due to the impurity $χ_{imp}=χ-χ_{bulk}$. We find that $χ_{imp}$ exactly vanishes at the solvable point, in a completely analogous way to the singular part of the specific heat $C_{imp}$. A perturbative calculation around the solvable point yields the generic behaviour… ▽ More We consider the two channel Kondo model in the Emery-Kivelson approach, and calculate the total susceptibility enhancement due to the impurity $χ_{imp}=χ-χ_{bulk}$. We find that $χ_{imp}$ exactly vanishes at the solvable point, in a completely analogous way to the singular part of the specific heat $C_{imp}$. A perturbative calculation around the solvable point yields the generic behaviour $χ_{imp} \sim \log {1 \over T}$, $C_{imp} \sim T\log T $ and the known universal value of the Wilson ratio $R_W={8 \over 3}$. From this calculation, the Kondo temperature can be identified and is found to behave as the inverse-square of the perturbation parameter. The small field, zero-temperature behaviour $χ_{imp}\sim log {1 \over h}$ is also recovered. △ Less

Submitted 3 December, 1993; originally announced December 1993.

Comments: 7 pages, REVTEX, 1 figure available on request, LPTENS preprint 93/46

Target Space Interpretation of New Moduli in 2D String Theory

Authors: Swapna Mahapatra, Sudipta Mukherji, Anirvan M. Sengupta

Abstract: We analyze the new states that have recently been discovered in 2D string theory by E. Witten and B. Zwiebach. Since the Liouville direction is uncompactified, we show that the deformations by the new ghost number two states generate equivalent classical solutions of the string fields. We argue that the new ghost number one states are responsible for generating transformations which relate such… ▽ More We analyze the new states that have recently been discovered in 2D string theory by E. Witten and B. Zwiebach. Since the Liouville direction is uncompactified, we show that the deformations by the new ghost number two states generate equivalent classical solutions of the string fields. We argue that the new ghost number one states are responsible for generating transformations which relate such equivalent solutions. We also discuss the possible interpretation of higher ghost number states of those kinds. △ Less

Submitted 30 June, 1992; originally announced June 1992.

Comments: 13 pages, TIFR/TH/92-30, June 1992

Journal ref: Mod.Phys.Lett. A7 (1992) 3119-3128

Instabilities in the gravitational background and string theory

Authors: Anirvan M. Sengupta

Abstract: We indicate the tentative source of instability in the two-dimensional black hole background. There are relevant operators among the tachyon and the higher level vertex operators in the conformal field theory. Connection of this instability with Hawking radiation is not obvious. The situation is somewhat analogous to fields in the background of a negative mass Euclidean Schwarzschild solution (i… ▽ More We indicate the tentative source of instability in the two-dimensional black hole background. There are relevant operators among the tachyon and the higher level vertex operators in the conformal field theory. Connection of this instability with Hawking radiation is not obvious. The situation is somewhat analogous to fields in the background of a negative mass Euclidean Schwarzschild solution (in four dimensions). Speculation is made about decay of the Minkowski black hole into finite temperature flat space. △ Less

Submitted 21 November, 1991; originally announced November 1991.

Comments: 9pages