-
Nonlinear classification of neural manifolds with contextual information
Authors:
Francesca Mignacco,
Chi-Ning Chou,
SueYeon Chung
Abstract:
Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and geometrical attributes of neural representations as population-level mechanistic descriptors of task implementation. In particular, manifold capacity has emerged as a prom…
▽ More
Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and geometrical attributes of neural representations as population-level mechanistic descriptors of task implementation. In particular, manifold capacity has emerged as a promising framework linking population geometry to the separability of neural manifolds. However, this metric has been limited to linear readouts. Here, we propose a theoretical framework that overcomes this limitation by leveraging contextual input information. We derive an exact formula for the context-dependent capacity that depends on manifold geometry and context correlations, and validate it on synthetic and real data. Our framework's increased expressivity captures representation untanglement in deep networks at early stages of the layer hierarchy, previously inaccessible to analysis. As context-dependent nonlinearity is ubiquitous in neural systems, our data-driven and theoretically grounded approach promises to elucidate context-dependent computation across scales, datasets, and models.
△ Less
Submitted 10 May, 2024;
originally announced May 2024.
-
Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds
Authors:
Michael Kuoch,
Chi-Ning Chou,
Nikhil Parthasarathy,
Joel Dapello,
James J. DiCarlo,
Haim Sompolinsky,
SueYeon Chung
Abstract:
Recently, growth in our understanding of the computations performed in both biological and artificial neural networks has largely been driven by either low-level mechanistic studies or global normative approaches. However, concrete methodologies for bridging the gap between these levels of abstraction remain elusive. In this work, we investigate the internal mechanisms of neural networks through t…
▽ More
Recently, growth in our understanding of the computations performed in both biological and artificial neural networks has largely been driven by either low-level mechanistic studies or global normative approaches. However, concrete methodologies for bridging the gap between these levels of abstraction remain elusive. In this work, we investigate the internal mechanisms of neural networks through the lens of neural population geometry, aiming to provide understanding at an intermediate level of abstraction, as a way to bridge that gap. Utilizing manifold capacity theory (MCT) from statistical physics and manifold alignment analysis (MAA) from high-dimensional statistics, we probe the underlying organization of task-dependent manifolds in deep neural networks and macaque neural recordings. Specifically, we quantitatively characterize how different learning objectives lead to differences in the organizational strategies of these models and demonstrate how these geometric analyses are connected to the decodability of task-relevant information. These analyses present a strong direction for bridging mechanistic and normative theories in neural networks through neural population geometry, potentially opening up many future research avenues in both machine learning and neuroscience.
△ Less
Submitted 21 December, 2023;
originally announced December 2023.
-
The Computational Lens: from Quantum Physics to Neuroscience
Authors:
Chi-Ning Chou
Abstract:
Two transformative waves of computing have redefined the way we approach science. The first wave came with the birth of the digital computer, which enabled scientists to numerically simulate their models and analyze massive datasets. This technological breakthrough led to the emergence of many sub-disciplines bearing the prefix "computational" in their names. Currently, we are in the midst of the…
▽ More
Two transformative waves of computing have redefined the way we approach science. The first wave came with the birth of the digital computer, which enabled scientists to numerically simulate their models and analyze massive datasets. This technological breakthrough led to the emergence of many sub-disciplines bearing the prefix "computational" in their names. Currently, we are in the midst of the second wave, marked by the remarkable advancements in artificial intelligence. From predicting protein structures to classifying galaxies, the scope of its applications is vast, and there can only be more awaiting us on the horizon.
While these two waves influence scientific methodology at the instrumental level, in this dissertation, I will present the computational lens in science, aiming at the conceptual level. Specifically, the central thesis posits that computation serves as a convenient and mechanistic language for understanding and analyzing information processing systems, offering the advantages of composability and modularity.
This dissertation begins with an illustration of the blueprint of the computational lens, supported by a review of relevant previous work. Subsequently, I will present my own works in quantum physics and neuroscience as concrete examples. In the concluding chapter, I will contemplate the potential of applying the computational lens across various scientific fields, in a way that can provide significant domain insights, and discuss potential future directions.
△ Less
Submitted 31 October, 2023;
originally announced October 2023.
-
MolMiner: You only look once for chemical structure recognition
Authors:
Youjun Xu,
Jinchuan Xiao,
Chia-Han Chou,
Jianhang Zhang,
Jintao Zhu,
Qiwan Hu,
Hemin Li,
Ningsheng Han,
Bingyu Liu,
Shuaipeng Zhang,
Jinyu Han,
Zhen Zhang,
Shuhao Zhang,
Weilin Zhang,
Luhua Lai,
Jianfeng Pei
Abstract:
Molecular structures are always depicted as 2D printed form in scientific documents like journal papers and patents. However, these 2D depictions are not machine-readable. Due to a backlog of decades and an increasing amount of these printed literature, there is a high demand for the translation of printed depictions into machine-readable formats, which is known as Optical Chemical Structure Recog…
▽ More
Molecular structures are always depicted as 2D printed form in scientific documents like journal papers and patents. However, these 2D depictions are not machine-readable. Due to a backlog of decades and an increasing amount of these printed literature, there is a high demand for the translation of printed depictions into machine-readable formats, which is known as Optical Chemical Structure Recognition (OCSR). Most OCSR systems developed over the last three decades follow a rule-based approach where the key step of vectorization of the depiction is based on the interpretation of vectors and nodes as bonds and atoms. Here, we present a practical software MolMiner, which is primarily built up using deep neural networks originally developed for semantic segmentation and object detection to recognize atom and bond elements from documents. These recognized elements can be easily connected as a molecular graph with distance-based construction algorithm. We carefully evaluate our software on four benchmark datasets with the state-of-the-art performance. Various real application scenarios are also tested, yielding satisfactory outcomes. The free download links of Mac and Windows versions are available: Mac: https://molminer-cdn.iipharma.cn/pharma-mind/artifact/latest/mac/PharmaMind-mac-latest-setup.dmg and Windows: https://molminer-cdn.iipharma.cn/pharma-mind/artifact/latest/win/PharmaMind-win-latest-setup.exe
△ Less
Submitted 22 May, 2022;
originally announced May 2022.
-
Using transcription-based detectors to emulate the behaviour of sequential probability ratio-based concentration detectors
Authors:
Chun Tung Chou
Abstract:
The sequential probability ratio test (SPRT) from statistics is known to have the least mean decision time compared to other sequential or fixed-time tests for given error rates. In some circumstances, cells need to make decisions accurately and quickly, therefore it has been suggested the SPRT may be used to understand the speed-accuracy tradeoff in cellular decision making. It is generally thoug…
▽ More
The sequential probability ratio test (SPRT) from statistics is known to have the least mean decision time compared to other sequential or fixed-time tests for given error rates. In some circumstances, cells need to make decisions accurately and quickly, therefore it has been suggested the SPRT may be used to understand the speed-accuracy tradeoff in cellular decision making. It is generally thought that in order for cells to make use of the SPRT, it is necessary to find biochemical circuits that can compute the log-likelihood ratio needed for the SPRT. However, this paper takes a different approach. We recognise that the high-level behaviour of the SPRT is defined by its positive detection or hit rate, and the computation of the log-likelihood ratio is just one way to realise this behaviour. In this paper, we will present a method which uses a transcription-based detector to emulate the hit rate of the SPRT without computing the exact log-likelihood ratio. We consider the problem of using a promoter with multiple binding sites to accurately and quickly detect whether the concentration of a transcription factor is above a target level. We show that it is possible to find binding and unbinding rates of the transcription factor to the promoter's binding sites so that the probability that the amount of mRNA produced will be higher than a threshold is approximately equal to the hit rate of the SPRT detector. Moreover, we show that the average time that this transcription-based detector needs to make a positive detection is less than or equal to that of the SPRT for a wide range of concentrations. We remark that the last statement does not contradict Wald's optimality result because our transcription-based detector uses an open-ended test.
△ Less
Submitted 4 October, 2022; v1 submitted 15 February, 2022;
originally announced February 2022.
-
ODE-Inspired Analysis for the Biological Version of Oja's Rule in Solving Streaming PCA
Authors:
Chi-Ning Chou,
Mien Brabeeba Wang
Abstract:
Oja's rule [Oja, Journal of mathematical biology 1982] is a well-known biologically-plausible algorithm using a Hebbian-type synaptic update rule to solve streaming principal component analysis (PCA). Computational neuroscientists have known that this biological version of Oja's rule converges to the top eigenvector of the covariance matrix of the input in the limit. However, prior to this work, i…
▽ More
Oja's rule [Oja, Journal of mathematical biology 1982] is a well-known biologically-plausible algorithm using a Hebbian-type synaptic update rule to solve streaming principal component analysis (PCA). Computational neuroscientists have known that this biological version of Oja's rule converges to the top eigenvector of the covariance matrix of the input in the limit. However, prior to this work, it was open to prove any convergence rate guarantee.
In this work, we give the first convergence rate analysis for the biological version of Oja's rule in solving streaming PCA. Moreover, our convergence rate matches the information theoretical lower bound up to logarithmic factors and outperforms the state-of-the-art upper bound for streaming PCA. Furthermore, we develop a novel framework inspired by ordinary differential equations (ODE) to analyze general stochastic dynamics. The framework abandons the traditional step-by-step analysis and instead analyzes a stochastic dynamic in one-shot by giving a closed-form solution to the entire dynamic. The one-shot framework allows us to apply stopping time and martingale techniques to have a flexible and precise control on the dynamic. We believe that this general framework is powerful and should lead to effective yet simple analysis for a large class of problems with stochastic dynamics.
△ Less
Submitted 17 June, 2020; v1 submitted 4 November, 2019;
originally announced November 2019.
-
Using biochemical circuits to approximately compute log-likelihood ratio for detecting persistent signals
Authors:
Chun Tung Chou
Abstract:
Given that biochemical circuits can process information by using analog computation, a question is: What can biochemical circuits compute? This paper considers the problem of using biochemical circuits to distinguish persistent signals from transient ones. We define a statistical detection problem over a reaction pathway consisting of three species: an inducer, a transcription factor (TF) and a ge…
▽ More
Given that biochemical circuits can process information by using analog computation, a question is: What can biochemical circuits compute? This paper considers the problem of using biochemical circuits to distinguish persistent signals from transient ones. We define a statistical detection problem over a reaction pathway consisting of three species: an inducer, a transcription factor (TF) and a gene promoter, where the inducer can activate the TF and an active TF can bind to the gene promoter. We model the pathway using the chemical master equation so the counts of bound promoters over time is a stochastic signal. We consider the problem of using the continuous-time stochastic signal of the counts of bound promoters to infer whether the inducer signal is persistent or not. We use statistical detection theory to derive the solution to this detection problem, which is to compute the log-likelihood ratio of observing a persistent signal to a transient one. We then show, using time-scale separation and other assumptions, that this log-likelihood ratio can be approximately computed by using the continuous-time signals of the number of active TF molecules and the number of bound promoters when the input is persistent. Finally, we show that the coherent feedforward gene circuits can be used to approximately compute this log-likelihood ratio when the inducer signal is persistent.
△ Less
Submitted 15 September, 2021; v1 submitted 23 July, 2019;
originally announced July 2019.
-
A Network-based Multimodal Data Fusion Approach for Characterizing Dynamic Multimodal Physiological Patterns
Authors:
Miaolin Fan,
Chun-An Chou,
Sheng-Che Yen,
Yingzi Lin
Abstract:
Characterizing the dynamic interactive patterns of complex systems helps gain in-depth understanding of how components interrelate with each other while performing certain functions as a whole. In this study, we present a novel multimodal data fusion approach to construct a complex network, which models the interactions of biological subsystems in the human body under emotional states through phys…
▽ More
Characterizing the dynamic interactive patterns of complex systems helps gain in-depth understanding of how components interrelate with each other while performing certain functions as a whole. In this study, we present a novel multimodal data fusion approach to construct a complex network, which models the interactions of biological subsystems in the human body under emotional states through physiological responses. Joint recurrence plot and temporal network metrics are employed to integrate the multimodal information at the signal level. A benchmark public dataset of is used for evaluating our model.
△ Less
Submitted 3 January, 2019;
originally announced January 2019.
-
Detection of persistent signals and its relation to coherent feedforward loops
Authors:
Chun Tung Chou
Abstract:
Many studies have shown that cells use temporal dynamics of signalling molecules to encode information. One particular class of temporal dynamics is persistent and transient signals, i.e. signals of long and short durations respectively. It has been shown that the coherent type-1 feedforward loop with an AND logic at the output (or C1-FFL for short) can be used to discriminate a persistent input s…
▽ More
Many studies have shown that cells use temporal dynamics of signalling molecules to encode information. One particular class of temporal dynamics is persistent and transient signals, i.e. signals of long and short durations respectively. It has been shown that the coherent type-1 feedforward loop with an AND logic at the output (or C1-FFL for short) can be used to discriminate a persistent input signal from a transient one. This has been done by modelling the C1-FFL, and then use the model to show that persistent and transient input signals give, respectively, a non-zero and zero output. Instead of assuming the structure of C1-FFL, this paper shows that it is possible to deduce the C1-FFL model from the requirement of discriminating a persistent signal. We do this by first formulating a statistical detection problem of distinguishing persistent signals from transient ones. The solution of the detection problem is to compute the log-likelihood ratio of observing a persistent signal to a transient signal. We show that, if this log-likelihood ratio is positive, which happens when the signal is likely to be persistent, then it can be approximately computed by a C1-FFL. Although the capability of C1-FFL to discriminate persistent signals is known, this paper adds an information processing interpretation on how a C1-FFL works as a detector of persistent signals.
△ Less
Submitted 11 October, 2018; v1 submitted 6 February, 2018;
originally announced February 2018.
-
A Markovian Approach to the Optimal Demodulation of Diffusion-based Molecular Communication Networks
Authors:
Chun Tung Chou
Abstract:
In a diffusion-based molecular communication network, transmitters and receivers communicate by using signalling molecules (or ligands) in a fluid medium. This paper assumes that the transmitter uses different chemical reactions to generate different emission patterns of signalling molecules to represent different transmission symbols, and the receiver consists of receptors. When the signalling mo…
▽ More
In a diffusion-based molecular communication network, transmitters and receivers communicate by using signalling molecules (or ligands) in a fluid medium. This paper assumes that the transmitter uses different chemical reactions to generate different emission patterns of signalling molecules to represent different transmission symbols, and the receiver consists of receptors. When the signalling molecules arrive at the receiver, they may react with the receptors to form ligand-receptor complexes. Our goal is to study the demodulation in this setup assuming that the transmitter and receiver are synchronised. We derive an optimal demodulator using the continuous history of the number of complexes at the receiver as the input to the demodulator. We do that by first deriving a communication model which includes the chemical reactions in the transmitter, diffusion in the transmission medium and the ligand-receptor process in the receiver. This model, which takes the form of a continuous-time Markov process, captures the noise in the receiver signal due to the stochastic nature of chemical reactions and diffusion. We then adopt a maximum a posterior framework and use Bayesian filtering to derive the optimal demodulator. We use numerical examples to illustrate the properties of this optimal demodulator.
△ Less
Submitted 11 August, 2015; v1 submitted 3 March, 2015;
originally announced March 2015.
-
Molecular communication networks with general molecular circuit receivers
Authors:
Chun Tung Chou
Abstract:
In a molecular communication network, transmitters may encode information in concentration or frequency of signalling molecules. When the signalling molecules reach the receivers, they react, via a set of chemical reactions or a molecular circuit, to produce output molecules. The counts of output molecules over time is the output signal of the receiver. The aim of this paper is to investigate the…
▽ More
In a molecular communication network, transmitters may encode information in concentration or frequency of signalling molecules. When the signalling molecules reach the receivers, they react, via a set of chemical reactions or a molecular circuit, to produce output molecules. The counts of output molecules over time is the output signal of the receiver. The aim of this paper is to investigate the impact of different reaction types on the information transmission capacity of molecular communication networks. We realise this aim by using a general molecular circuit model. We derive general expressions of mean receiver output, and signal and noise spectra. We use these expressions to investigate the information transmission capacities of a number of molecular circuits.
△ Less
Submitted 19 December, 2013;
originally announced December 2013.
-
Impact of receiver reaction mechanisms on the performance of molecular communication networks
Authors:
Chun Tung Chou
Abstract:
In a molecular communication network, transmitters and receivers communicate by using signalling molecules. At the receivers, the signalling molecules react, via a chain of chemical reactions, to produce output molecules. The counts of output molecules over time is considered to be the output signal of the receiver. This output signal is used to detect the presence of signalling molecules at the r…
▽ More
In a molecular communication network, transmitters and receivers communicate by using signalling molecules. At the receivers, the signalling molecules react, via a chain of chemical reactions, to produce output molecules. The counts of output molecules over time is considered to be the output signal of the receiver. This output signal is used to detect the presence of signalling molecules at the receiver. The output signal is noisy due to the stochastic nature of diffusion and chemical reactions. The aim of this paper is to characterise the properties of the output signals for two types of receivers, which are based on two different types of reaction mechanisms. We derive analytical expressions for the mean, variance and frequency properties of these two types of receivers. These expressions allow us to study the properties of these two types of receivers. In addition, our model allows us to study the effect of the diffusibility of the receiver membrane on the performance of the receivers.
△ Less
Submitted 4 December, 2013;
originally announced December 2013.
-
Extended master equation models for molecular communication networks
Authors:
Chun Tung Chou
Abstract:
We consider molecular communication networks consisting of transmitters and receivers distributed in a fluidic medium. In such networks, a transmitter sends one or more signalling molecules, which are diffused over the medium, to the receiver to realise the communication. In order to be able to engineer synthetic molecular communication networks, mathematical models for these networks are required…
▽ More
We consider molecular communication networks consisting of transmitters and receivers distributed in a fluidic medium. In such networks, a transmitter sends one or more signalling molecules, which are diffused over the medium, to the receiver to realise the communication. In order to be able to engineer synthetic molecular communication networks, mathematical models for these networks are required. This paper proposes a new stochastic model for molecular communication networks called reaction-diffusion master equation with exogenous input (RDMEX). The key idea behind RDMEX is to model the transmitters as time series of signalling molecule counts, while diffusion in the medium and chemical reactions at the receivers are modelled as Markov processes using master equation. An advantage of RDMEX is that it can readily be used to model molecular communication networks with multiple transmitters and receivers. For the case where the reaction kinetics at the receivers is linear, we show how RDMEX can be used to determine the mean and covariance of the receiver output signals, and derive closed-form expressions for the mean receiver output signal of the RDMEX model. These closed-form expressions reveal that the output signal of a receiver can be affected by the presence of other receivers. Numerical examples are provided to demonstrate the properties of the model.
△ Less
Submitted 2 November, 2013; v1 submitted 19 April, 2012;
originally announced April 2012.
