Showing 1–50 of 52 results for author: Tong, A

ImageFlowNet: Forecasting Multiscale Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images

Authors: Chen Liu, Ke Xu, Liangbo L. Shen, Guillaume Huguet, Zilong Wang, Alexander Tong, Danilo Bzdok, Jay Stewart, Jay C. Wang, Lucian V. Del Priore, Smita Krishnaswamy

Abstract: The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spat… ▽ More The forecasting of disease progression from images is a holy grail for clinical decision making. However, this task is complicated by the inherent high dimensionality, temporal sparsity and sampling irregularity in longitudinal image acquisitions. Existing methods often rely on extracting hand-crafted features and performing time-series analysis in this vector space, leading to a loss of rich spatial information within the images. To overcome these challenges, we introduce ImageFlowNet, a novel framework that learns latent-space flow fields that evolve multiscale representations in joint embedding spaces using neural ODEs and SDEs to model disease progression in the image domain. Notably, ImageFlowNet learns multiscale joint representation spaces by combining cohorts of patients together so that information can be transferred between the patient samples. The dynamics then provide plausible trajectories of progression, with the SDE providing alternative trajectories from the same starting point. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We then demonstrate ImageFlowNet's effectiveness through empirical evaluations on three longitudinal medical image datasets depicting progression in retinal geographic atrophy, multiple sclerosis, and glioblastoma. △ Less

Submitted 2 July, 2024; v1 submitted 20 June, 2024; originally announced June 2024.

Comments: Included reference to codebase. Added acknowledgements

Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation

Authors: Guillaume Huguet, James Vuckovic, Kilian Fatras, Eric Thibodeau-Laufer, Pablo Lemos, Riashat Islam, Cheng-Hao Liu, Jarrid Rector-Brooks, Tara Akhound-Sadegh, Michael Bronstein, Alexander Tong, Avishek Joey Bose

Abstract: Proteins are essential for almost all biological processes and derive their diverse functions from complex 3D structures, which are in turn determined by their amino acid sequences. In this paper, we exploit the rich biological inductive bias of amino acid sequences and introduce FoldFlow-2, a novel sequence-conditioned SE(3)-equivariant flow matching model for protein structure generation. FoldFl… ▽ More Proteins are essential for almost all biological processes and derive their diverse functions from complex 3D structures, which are in turn determined by their amino acid sequences. In this paper, we exploit the rich biological inductive bias of amino acid sequences and introduce FoldFlow-2, a novel sequence-conditioned SE(3)-equivariant flow matching model for protein structure generation. FoldFlow-2 presents substantial new architectural features over the previous FoldFlow family of models including a protein large language model to encode sequence, a new multi-modal fusion trunk that combines structure and sequence representations, and a geometric transformer based decoder. To increase diversity and novelty of generated samples -- crucial for de-novo drug design -- we train FoldFlow-2 at scale on a new dataset that is an order of magnitude larger than PDB datasets of prior works, containing both known proteins in PDB and high-quality synthetic structures achieved through filtering. We further demonstrate the ability to align FoldFlow-2 to arbitrary rewards, e.g. increasing secondary structures diversity, by introducing a Reinforced Finetuning (ReFT) objective. We empirically observe that FoldFlow-2 outperforms previous state-of-the-art protein structure-based generative models, improving over RFDiffusion in terms of unconditional generation across all metrics including designability, diversity, and novelty across all protein lengths, as well as exhibiting generalization on the task of equilibrium conformation sampling. Finally, we demonstrate that a fine-tuned FoldFlow-2 makes progress on challenging conditional design tasks such as designing scaffolds for the VHH nanobody. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: preprint

Metric Flow Matching for Smooth Interpolations on the Data Manifold

Authors: Kacper Kapusniak, Peter Potaptchik, Teodora Reu, Leo Zhang, Alexander Tong, Michael Bronstein, Avishek Joey Bose, Francesco Di Giovanni

Abstract: Matching objectives underpin the success of modern generative models and rely on constructing conditional paths that transform a source distribution into a target distribution. Despite being a fundamental building block, conditional paths have been designed principally under the assumption of Euclidean geometry, resulting in straight interpolations. However, this can be particularly restrictive fo… ▽ More Matching objectives underpin the success of modern generative models and rely on constructing conditional paths that transform a source distribution into a target distribution. Despite being a fundamental building block, conditional paths have been designed principally under the assumption of Euclidean geometry, resulting in straight interpolations. However, this can be particularly restrictive for tasks such as trajectory inference, where straight paths might lie outside the data manifold, thus failing to capture the underlying dynamics giving rise to the observed marginals. In this paper, we propose Metric Flow Matching (MFM), a novel simulation-free framework for conditional flow matching where interpolants are approximate geodesics learned by minimizing the kinetic energy of a data-induced Riemannian metric. This way, the generative model matches vector fields on the data manifold, which corresponds to lower uncertainty and more meaningful interpolations. We prescribe general metrics to instantiate MFM, independent of the task, and test it on a suite of challenging problems including LiDAR navigation, unpaired image translation, and modeling cellular dynamics. We observe that MFM outperforms the Euclidean baselines, particularly achieving SOTA on single-cell trajectory prediction. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Anomaly Detection by Adapting a pre-trained Vision Language Model

Authors: Yuxuan Cai, Xinwei He, Dingkang Liang, Ao Tong, Xiang Bai

Abstract: Recently, large vision and language models have shown their success when adapting them to many downstream tasks. In this paper, we present a unified framework named CLIP-ADA for Anomaly Detection by Adapting a pre-trained CLIP model. To this end, we make two important improvements: 1) To acquire unified anomaly detection across industrial images of multiple categories, we introduce the learnable p… ▽ More Recently, large vision and language models have shown their success when adapting them to many downstream tasks. In this paper, we present a unified framework named CLIP-ADA for Anomaly Detection by Adapting a pre-trained CLIP model. To this end, we make two important improvements: 1) To acquire unified anomaly detection across industrial images of multiple categories, we introduce the learnable prompt and propose to associate it with abnormal patterns through self-supervised learning. 2) To fully exploit the representation power of CLIP, we introduce an anomaly region refinement strategy to refine the localization quality. During testing, the anomalies are localized by directly calculating the similarity between the representation of the learnable prompt and the image. Comprehensive experiments demonstrate the superiority of our framework, e.g., we achieve the state-of-the-art 97.5/55.6 and 89.3/33.1 on MVTec-AD and VisA for anomaly detection and localization. In addition, the proposed method also achieves encouraging performance with marginal training data, which is more challenging. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Iterated Denoising Energy Matching for Sampling from Boltzmann Densities

Authors: Tara Akhound-Sadegh, Jarrid Rector-Brooks, Avishek Joey Bose, Sarthak Mittal, Pablo Lemos, Cheng-Hao Liu, Marcin Sendera, Siamak Ravanbakhsh, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, Alexander Tong

Abstract: Efficiently generating statistically independent samples from an unnormalized probability distribution, such as equilibrium samples of many-body systems, is a foundational problem in science. In this paper, we propose Iterated Denoising Energy Matching (iDEM), an iterative algorithm that uses a novel stochastic score matching objective leveraging solely the energy function and its gradient -- and… ▽ More Efficiently generating statistically independent samples from an unnormalized probability distribution, such as equilibrium samples of many-body systems, is a foundational problem in science. In this paper, we propose Iterated Denoising Energy Matching (iDEM), an iterative algorithm that uses a novel stochastic score matching objective leveraging solely the energy function and its gradient -- and no data samples -- to train a diffusion-based sampler. Specifically, iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our stochastic matching objective to further improve the sampler. iDEM is scalable to high dimensions as the inner matching objective, is simulation-free, and requires no MCMC samples. Moreover, by leveraging the fast mode mixing behavior of diffusion, iDEM smooths out the energy landscape enabling efficient exploration and learning of an amortized sampler. We evaluate iDEM on a suite of tasks ranging from standard synthetic energy functions to invariant $n$-body particle systems. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5\times$ faster, which allows it to be the first method to train using energy on the challenging $55$-particle Lennard-Jones system. △ Less

Submitted 26 June, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: Published at ICML 2024. Code for iDEM is available at https://github.com/jarridrb/dem

A Geometric VOF Method for Interface Flow Simulations

Authors: Dezhi Dai, Haomin Yuan, Albert Y. Tong, Adrian Tentner

Abstract: A novel numerical technique designed for interface flow simulations using the Volume of Fluid (VOF) method on arbitrary unstructured meshes has been introduced. The method is called SimPLIC, which seamlessly integrates Piecewise Linear Interface Calculation (PLIC) and Simpson's rule. The main focus of the proposed method is to compute the volume of the primary phase that moves across a mesh face w… ▽ More A novel numerical technique designed for interface flow simulations using the Volume of Fluid (VOF) method on arbitrary unstructured meshes has been introduced. The method is called SimPLIC, which seamlessly integrates Piecewise Linear Interface Calculation (PLIC) and Simpson's rule. The main focus of the proposed method is to compute the volume of the primary phase that moves across a mesh face within a single time step. This is achieved by reconstructing the interface and assessing how the submerged face area evolves over time. Simpson's rule is employed to integrate the time evolution of this submerged face area, ensuring an accurate estimation of the volume of the transported primary phase. The method's robustness was validated by solving a spherical interface advection problem in a non-uniform three-dimensional flow across unstructured meshes with diverse cell types and dimensions. Key metrics such as volume conservation, shape retention, friction boundedness and solving efficiency were meticulously monitored and juxtaposed. Numerical outcomes underscored the precision and adequacy of the PLIC-VOF technique when complemented with Simpson's rule in advecting the interface. Furthermore, the SimPLIC method has been integrated into OpenFOAM v2312 as an unofficial extension and is now accessible to the community. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy

Authors: Danqi Liao, Chen Liu, Benjamin W. Christensen, Alexander Tong, Guillaume Huguet, Guy Wolf, Maximilian Nickel, Ian Adelstein, Smita Krishnaswamy

Abstract: Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying m… ▽ More Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. Specifically, we define diffusion spectral entropy (DSE) in neural representations of a dataset as well as diffusion spectral mutual information (DSMI) between different variables representing data. First, we show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data that outperform classic Shannon entropy, nonparametric estimation, and mutual information neural estimation (MINE). We then study the evolution of representations in classification networks with supervised learning, self-supervision, or overfitting. We observe that (1) DSE of neural representations increases during training; (2) DSMI with the class label increases during generalizable learning but stays stagnant during overfitting; (3) DSMI with the input signal shows differing trends: on MNIST it increases, while on CIFAR-10 and STL-10 it decreases. Finally, we show that DSE can be used to guide better network initialization and that DSMI can be used to predict downstream classification accuracy across 962 models on ImageNet. The official implementation is available at https://github.com/ChenLiu-1996/DiffusionSpectralEntropy. △ Less

Submitted 3 December, 2023; originally announced December 2023.

Journal ref: ICML 2023 Workshop on Topology, Algebra, and Geometry in Machine Learning

SigFormer: Signature Transformers for Deep Hedging

Authors: Anh Tong, Thanh Nguyen-Tang, Dongeun Lee, Toan Tran, Jaesik Choi

Abstract: Deep hedging is a promising direction in quantitative finance, incorporating models and techniques from deep learning research. While giving excellent hedging strategies, models inherently requires careful treatment in designing architectures for neural networks. To mitigate such difficulties, we introduce SigFormer, a novel deep learning model that combines the power of path signatures and transf… ▽ More Deep hedging is a promising direction in quantitative finance, incorporating models and techniques from deep learning research. While giving excellent hedging strategies, models inherently requires careful treatment in designing architectures for neural networks. To mitigate such difficulties, we introduce SigFormer, a novel deep learning model that combines the power of path signatures and transformers to handle sequential data, particularly in cases with irregularities. Path signatures effectively capture complex data patterns, while transformers provide superior sequential attention. Our proposed model is empirically compared to existing methods on synthetic data, showcasing faster learning and enhanced robustness, especially in the presence of irregular underlying price data. Additionally, we validate our model performance through a real-world backtest on hedging the SP 500 index, demonstrating positive outcomes. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: ICAIF 2023

A Computational Framework for Solving Wasserstein Lagrangian Flows

Authors: Kirill Neklyudov, Rob Brekelmans, Alexander Tong, Lazar Atanackovic, Qiang Liu, Alireza Makhzani

Abstract: The dynamical formulation of the optimal transport can be extended through various choices of the underlying geometry (kinetic energy), and the regularization of density paths (potential energy). These combinations yield different variational problems (Lagrangians), encompassing many variations of the optimal transport problem such as the Schrödinger bridge, unbalanced optimal transport, and optim… ▽ More The dynamical formulation of the optimal transport can be extended through various choices of the underlying geometry (kinetic energy), and the regularization of density paths (potential energy). These combinations yield different variational problems (Lagrangians), encompassing many variations of the optimal transport problem such as the Schrödinger bridge, unbalanced optimal transport, and optimal transport with physical constraints, among others. In general, the optimal density path is unknown, and solving these variational problems can be computationally challenging. We propose a novel deep learning based framework approaching all of these problems from a unified perspective. Leveraging the dual formulation of the Lagrangians, our method does not require simulating or backpropagating through the trajectories of the learned dynamics, and does not need access to optimal couplings. We showcase the versatility of the proposed framework by outperforming previous approaches for the single-cell trajectory inference, where incorporating prior knowledge into the dynamics is crucial for correct predictions. △ Less

Submitted 3 July, 2024; v1 submitted 16 October, 2023; originally announced October 2023.

Causal Inference in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems

Authors: Trang Nguyen, Alexander Tong, Kanika Madan, Yoshua Bengio, Dianbo Liu

Abstract: Understanding causal relationships within Gene Regulatory Networks (GRNs) is essential for unraveling the gene interactions in cellular processes. However, causal discovery in GRNs is a challenging problem for multiple reasons including the existence of cyclic feedback loops and uncertainty that yields diverse possible causal structures. Previous works in this area either ignore cyclic dynamics (a… ▽ More Understanding causal relationships within Gene Regulatory Networks (GRNs) is essential for unraveling the gene interactions in cellular processes. However, causal discovery in GRNs is a challenging problem for multiple reasons including the existence of cyclic feedback loops and uncertainty that yields diverse possible causal structures. Previous works in this area either ignore cyclic dynamics (assume acyclic structure) or struggle with scalability. We introduce Swift-DynGFN as a novel framework that enhances causal structure learning in GRNs while addressing scalability concerns. Specifically, Swift-DynGFN exploits gene-wise independence to boost parallelization and to lower computational cost. Experiments on real single-cell RNA velocity and synthetic GRN datasets showcase the advancement in learning causal structure in GRNs and scalability in larger systems. △ Less

Submitted 5 October, 2023; originally announced October 2023.

SE(3)-Stochastic Flow Matching for Protein Backbone Generation

Authors: Avishek Joey Bose, Tara Akhound-Sadegh, Guillaume Huguet, Kilian Fatras, Jarrid Rector-Brooks, Cheng-Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael Bronstein, Alexander Tong

Abstract: The computational design of novel protein structures has the potential to impact numerous scientific disciplines greatly. Toward this goal, we introduce FoldFlow, a series of novel generative models of increasing modeling power based on the flow-matching paradigm over $3\mathrm{D}$ rigid motions -- i.e. the group $\text{SE}(3)$ -- enabling accurate modeling of protein backbones. We first introduce… ▽ More The computational design of novel protein structures has the potential to impact numerous scientific disciplines greatly. Toward this goal, we introduce FoldFlow, a series of novel generative models of increasing modeling power based on the flow-matching paradigm over $3\mathrm{D}$ rigid motions -- i.e. the group $\text{SE}(3)$ -- enabling accurate modeling of protein backbones. We first introduce FoldFlow-Base, a simulation-free approach to learning deterministic continuous-time dynamics and matching invariant target distributions on $\text{SE}(3)$. We next accelerate training by incorporating Riemannian optimal transport to create FoldFlow-OT, leading to the construction of both more simple and stable flows. Finally, we design FoldFlow-SFM, coupling both Riemannian OT and simulation-free training to learn stochastic continuous-time dynamics over $\text{SE}(3)$. Our family of FoldFlow, generative models offers several key advantages over previous approaches to the generative modeling of proteins: they are more stable and faster to train than diffusion-based approaches, and our models enjoy the ability to map any invariant source distribution to any invariant target distribution over $\text{SE}(3)$. Empirically, we validate FoldFlow, on protein backbone generation of up to $300$ amino acids leading to high-quality designable, diverse, and novel samples. △ Less

Submitted 11 April, 2024; v1 submitted 3 October, 2023; originally announced October 2023.

Comments: ICLR 2024 Spotlight

Simulation-free Schrödinger bridges via score and flow matching

Authors: Alexander Tong, Nikolay Malkin, Kilian Fatras, Lazar Atanackovic, Yanlei Zhang, Guillaume Huguet, Guy Wolf, Yoshua Bengio

Abstract: We present simulation-free score and flow matching ([SF]$^2$M), a simulation-free objective for inferring stochastic dynamics given unpaired samples drawn from arbitrary source and target distributions. Our method generalizes both the score-matching loss used in the training of diffusion models and the recently proposed flow matching loss used in the training of continuous normalizing flows. [SF]… ▽ More We present simulation-free score and flow matching ([SF]$^2$M), a simulation-free objective for inferring stochastic dynamics given unpaired samples drawn from arbitrary source and target distributions. Our method generalizes both the score-matching loss used in the training of diffusion models and the recently proposed flow matching loss used in the training of continuous normalizing flows. [SF]$^2$M interprets continuous-time stochastic generative modeling as a Schrödinger bridge problem. It relies on static entropy-regularized optimal transport, or a minibatch approximation, to efficiently learn the SB without simulating the learned stochastic process. We find that [SF]$^2$M is more efficient and gives more accurate solutions to the SB problem than simulation-based methods from prior work. Finally, we apply [SF]$^2$M to the problem of learning cell dynamics from snapshot data. Notably, [SF]$^2$M is the first method to accurately model cell dynamics in high dimensions and can recover known gene regulatory networks from simulated data. Our code is available in the TorchCFM package at https://github.com/atong01/conditional-flow-matching. △ Less

Submitted 11 March, 2024; v1 submitted 7 July, 2023; originally announced July 2023.

Comments: AISTATS 2024. Code: https://github.com/atong01/conditional-flow-matching

Neural FIM for learning Fisher Information Metrics from point cloud data

Authors: Oluwadamilola Fasina, Guillaume Huguet, Alexander Tong, Yanlei Zhang, Guy Wolf, Maximilian Nickel, Ian Adelstein, Smita Krishnaswamy

Abstract: Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the underlying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifol… ▽ More Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the underlying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifold model for the data. Neural FIM creates an extensible metric space from discrete point cloud data such that information from the metric can inform us of manifold characteristics such as volume and geodesics. We demonstrate Neural FIM's utility in selecting parameters for the PHATE visualization method as well as its ability to obtain information pertaining to local volume illuminating branching points and cluster centers embeddings of a toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs (immune cells). △ Less

Submitted 11 June, 2023; v1 submitted 1 June, 2023; originally announced June 2023.

Comments: 13 pages, 11 figures, 1 table

Graph Fourier MMD for Signals on Graphs

Authors: Samuel Leone, Aarthi Venkat, Guillaume Huguet, Alexander Tong, Guy Wolf, Smita Krishnaswamy

Abstract: While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little attention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in… ▽ More While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little attention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in the biomedical sciences. Thus, it becomes important to find ways to compare signals defined on such graphs. Here, we propose Graph Fourier MMD (GFMMD), a novel distance between distributions and signals on graphs. GFMMD is defined via an optimal witness function that is both smooth on the graph and maximizes difference in expectation between the pair of distributions on the graph. We find an analytical solution to this optimization problem as well as an embedding of distributions that results from this method. We also prove several properties of this method including scale invariance and applicability to disconnected graphs. We showcase it on graph benchmark datasets as well on single cell RNA-sequencing data analysis. In the latter, we use the GFMMD-based gene embeddings to find meaningful gene clusters. We also propose a novel type of score for gene selection called "gene localization score" which helps select genes for cellular state space characterization. △ Less

Submitted 4 June, 2023; originally announced June 2023.

A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction

Authors: Guillaume Huguet, Alexander Tong, Edward De Brouwer, Yanlei Zhang, Guy Wolf, Ian Adelstein, Smita Krishnaswamy

Abstract: Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoret… ▽ More Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoretical links have been established. Here, we establish such a link via results in Riemannian geometry explicitly connecting heat diffusion to manifold distances. In this process, we also formulate a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. This novel perspective makes clearer the choices available in manifold learning and denoising. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances, and preserving cluster structure in toy datasets. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure, where our method enables interpolation of withheld timepoints of data. Finally, we show that parameters of our more general method can be configured to give results similar to PHATE (a state-of-the-art diffusion based manifold learning method) as well as SNE (an attraction/repulsion neighborhood based method that forms the basis of t-SNE). △ Less

Submitted 30 May, 2023; originally announced May 2023.

Comments: 31 pages, 13 figures, 10 tables

Conditional Support Alignment for Domain Adaptation with Label Shift

Authors: Anh T Nguyen, Lam Tran, Anh Tong, Tuan-Duy H. Nguyen, Toan Tran

Abstract: Unsupervised domain adaptation (UDA) refers to a domain adaptation framework in which a learning model is trained based on the labeled samples on the source domain and unlabelled ones in the target domain. The dominant existing methods in the field that rely on the classical covariate shift assumption to learn domain-invariant feature representation have yielded suboptimal performance under the la… ▽ More Unsupervised domain adaptation (UDA) refers to a domain adaptation framework in which a learning model is trained based on the labeled samples on the source domain and unlabelled ones in the target domain. The dominant existing methods in the field that rely on the classical covariate shift assumption to learn domain-invariant feature representation have yielded suboptimal performance under the label distribution shift between source and target domains. In this paper, we propose a novel conditional adversarial support alignment (CASA) whose aim is to minimize the conditional symmetric support divergence between the source's and target domain's feature representation distributions, aiming at a more helpful representation for the classification task. We also introduce a novel theoretical target risk bound, which justifies the merits of aligning the supports of conditional feature distributions compared to the existing marginal support alignment approach in the UDA settings. We then provide a complete training process for learning in which the objective optimization functions are precisely based on the proposed target risk bound. Our empirical results demonstrate that CASA outperforms other state-of-the-art methods on different UDA benchmark tasks under label shift conditions. △ Less

Submitted 29 May, 2023; originally announced May 2023.

Capping Layer Effects on $Sb_{2}S_{3}$-based Reconfigurable Photonic Devices

Authors: Ting Yu Teo, Nanxi Li, Landobasa Y. M. Tobing, Amy S. K. Tong, Doris K. T. Ng, Zhihao Ren, Chengkuo Lee, Lennon Y. T. Lee, Robert Edward Simpson

Abstract: Capping layers are essential for protecting phase change materials (PCMs) used in non-volatile photonics technologies. This work demonstrates how $(ZnS)_{0.8}-(SiO_2)_{0.2}$ caps radically influence the performance of $Sb_{2}S_{3}$ and Ag-doped $Sb_{2}S_{3}$ integrated photonic devices. We found that at least 30 nm of capping material is necessary to protect the material from Sulfur loss. However,… ▽ More Capping layers are essential for protecting phase change materials (PCMs) used in non-volatile photonics technologies. This work demonstrates how $(ZnS)_{0.8}-(SiO_2)_{0.2}$ caps radically influence the performance of $Sb_{2}S_{3}$ and Ag-doped $Sb_{2}S_{3}$ integrated photonic devices. We found that at least 30 nm of capping material is necessary to protect the material from Sulfur loss. However, adding this cap affects the crystallization temperatures of the two PCMs in different ways. The crystallization temperature of $Sb_{2}S_{3}$ and Ag-doped $Sb_{2}S_{3}$ increased and decreased respectively, which is attributed to interfacial energy differences. Capped and uncapped Ag-doped $Sb_{2}S_{3}$ microring resonator (MRR) devices were fabricated and measured to understand how the cap affects the device performance. Surprisingly, the resonant frequency of the MRR exhibited a larger red-shift upon crystallization for the capped PCMs. This effect was due to the cap increasing the modal overlap with the PCM layer. Caps can, therefore, be used to provide a greater optical phase shift per unit length, thus reducing the overall footprint of these programmable devices. Overall, we conclude that caps on PCMs are not just useful for stabilizing the PCM layer, but can also be used to tune the PCM crystallization temperature and reduce device footprint. Moreover, the capping layer can be exploited to enhance light-matter interactions with the PCM element. △ Less

Submitted 5 May, 2023; originally announced May 2023.

FastMRI Prostate: A Publicly Available, Biparametric MRI Dataset to Advance Machine Learning for Prostate Cancer Imaging

Authors: Radhika Tibrewala, Tarun Dutt, Angela Tong, Luke Ginocchio, Mahesh B Keerthivasan, Steven H Baete, Sumit Chopra, Yvonne W Lui, Daniel K Sodickson, Hersh Chandarana, Patricia M Johnson

Abstract: The fastMRI brain and knee dataset has enabled significant advances in exploring reconstruction methods for improving speed and image quality for Magnetic Resonance Imaging (MRI) via novel, clinically relevant reconstruction approaches. In this study, we describe the April 2023 expansion of the fastMRI dataset to include biparametric prostate MRI data acquired on a clinical population. The dataset… ▽ More The fastMRI brain and knee dataset has enabled significant advances in exploring reconstruction methods for improving speed and image quality for Magnetic Resonance Imaging (MRI) via novel, clinically relevant reconstruction approaches. In this study, we describe the April 2023 expansion of the fastMRI dataset to include biparametric prostate MRI data acquired on a clinical population. The dataset consists of raw k-space and reconstructed images for T2-weighted and diffusion-weighted sequences along with slice-level labels that indicate the presence and grade of prostate cancer. As has been the case with fastMRI, increasing accessibility to raw prostate MRI data will further facilitate research in MR image reconstruction and evaluation with the larger goal of improving the utility of MRI for prostate cancer detection and evaluation. The dataset is available at https://fastmri.med.nyu.edu. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Comments: 4 pages, 1 figure

The Intrinsic Alignment of Red Galaxies in DES Y1 redMaPPer Galaxy Clusters

Authors: C. Zhou, A. Tong, M. A. Troxel, J. Blazek, C. Lin, D. Bacon, L. Bleem, A. Carnero Rosell, C. Chang, M. Costanzi, J. DeRose, J. P. Dietrich, A. Drlica-Wagner, D. Gruen, R. A. Gruendl, B. Hoyle, M. Jarvis, N. MacCrann, B. Mawdsley, T. McClintock, P. Melchior, J. Prat, A. Pujol, E. Rozo, E. S. Rykoff , et al. (57 additional authors not shown)

Abstract: Clusters of galaxies are sensitive to the most nonlinear peaks in the cosmic density field. The weak gravitational lensing of background galaxies by clusters can allow us to infer their masses. However, galaxies associated with the local environment of the cluster can also be intrinsically aligned due to the local tidal gradient, contaminating any cosmology derived from the lensing signal. We meas… ▽ More Clusters of galaxies are sensitive to the most nonlinear peaks in the cosmic density field. The weak gravitational lensing of background galaxies by clusters can allow us to infer their masses. However, galaxies associated with the local environment of the cluster can also be intrinsically aligned due to the local tidal gradient, contaminating any cosmology derived from the lensing signal. We measure this intrinsic alignment in Dark Energy Survey (DES) Year 1 redMaPPer clusters. We find evidence of a non-zero mean radial alignment of galaxies within clusters between redshift 0.1-0.7. We find a significant systematic in the measured ellipticities of cluster satellite galaxies that we attribute to the central galaxy flux and other intracluster light. We attempt to correct this signal, and fit a simple model for intrinsic alignment amplitude ($A_{\textrm{IA}}$) to the measurement, finding $A_{\textrm{IA}}=0.15\pm 0.04$, when excluding data near the edge of the cluster. We find a significantly stronger alignment of the central galaxy with the cluster dark matter halo at low redshift and with higher richness and central galaxy absolute magnitude (proxies for cluster mass). This is an important demonstration of the ability of large photometric data sets like DES to provide direct constraints on the intrinsic alignment of galaxies within clusters. These measurements can inform improvements to small-scale modeling and simulation of the intrinsic alignment of galaxies to help improve the separation of the intrinsic alignment signal in weak lensing studies. △ Less

Submitted 5 September, 2023; v1 submitted 23 February, 2023; originally announced February 2023.

Comments: 14 pages, 13 figures. Accepted to MNRAS

DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets

Authors: Lazar Atanackovic, Alexander Tong, Bo Wang, Leo J. Lee, Yoshua Bengio, Jason Hartford

Abstract: One of the grand challenges of cell biology is inferring the gene regulatory network (GRN) which describes interactions between genes and their products that control gene expression and cellular function. We can treat this as a causal discovery problem but with two non-standard challenges: (1) regulatory networks are inherently cyclic so we should not model a GRN as a directed acyclic graph (DAG),… ▽ More One of the grand challenges of cell biology is inferring the gene regulatory network (GRN) which describes interactions between genes and their products that control gene expression and cellular function. We can treat this as a causal discovery problem but with two non-standard challenges: (1) regulatory networks are inherently cyclic so we should not model a GRN as a directed acyclic graph (DAG), and (2) observations have significant measurement noise, so for typical sample sizes there will always be a large equivalence class of graphs that are likely given the data, and we want methods that capture this uncertainty. Existing methods either focus on challenge (1), identifying cyclic structure from dynamics, or on challenge (2) learning complex Bayesian posteriors over DAGs, but not both. In this paper we leverage the fact that it is possible to estimate the "velocity" of gene expression with RNA velocity techniques to develop an approach that addresses both challenges. Because we have access to velocity information, we can treat the Bayesian structure learning problem as a problem of sparse identification of a dynamical system, capturing cyclic feedback loops through time. Since our objective is to model uncertainty over discrete structures, we leverage Generative Flow Networks (GFlowNets) to estimate the posterior distribution over the combinatorial space of possible sparse dependencies. Our results indicate that our method learns posteriors that better encapsulate the distributions of cyclic structures compared to counterpart state-of-the-art Bayesian structure learning approaches. △ Less

Submitted 22 December, 2023; v1 submitted 8 February, 2023; originally announced February 2023.

Improving and generalizing flow-based generative models with minibatch optimal transport

Authors: Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, Yoshua Bengio

Abstract: Continuous normalizing flows (CNFs) are an attractive generative modeling technique, but they have been held back by limitations in their simulation-based maximum likelihood training. We introduce the generalized conditional flow matching (CFM) technique, a family of simulation-free training objectives for CNFs. CFM features a stable regression objective like that used to train the stochastic flow… ▽ More Continuous normalizing flows (CNFs) are an attractive generative modeling technique, but they have been held back by limitations in their simulation-based maximum likelihood training. We introduce the generalized conditional flow matching (CFM) technique, a family of simulation-free training objectives for CNFs. CFM features a stable regression objective like that used to train the stochastic flow in diffusion models but enjoys the efficient inference of deterministic flow models. In contrast to both diffusion models and prior CNF training algorithms, CFM does not require the source distribution to be Gaussian or require evaluation of its density. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference, as evaluated in our experiments. Furthermore, we show that when the true OT plan is available, our OT-CFM method approximates dynamic OT. Training CNFs with CFM improves results on a variety of conditional and unconditional generation tasks, such as inferring single cell dynamics, unsupervised image translation, and Schrödinger bridge inference. △ Less

Submitted 11 March, 2024; v1 submitted 1 February, 2023; originally announced February 2023.

Comments: TMLR. Code: https://github.com/atong01/conditional-flow-matching

On the Feasibility of Machine Learning Augmented Magnetic Resonance for Point-of-Care Identification of Disease

Authors: Raghav Singhal, Mukund Sudarshan, Anish Mahishi, Sri Kaushik, Luke Ginocchio, Angela Tong, Hersh Chandarana, Daniel K. Sodickson, Rajesh Ranganath, Sumit Chopra

Abstract: Early detection of many life-threatening diseases (e.g., prostate and breast cancer) within at-risk population can improve clinical outcomes and reduce cost of care. While numerous disease-specific "screening" tests that are closer to Point-of-Care (POC) are in use for this task, their low specificity results in unnecessary biopsies, leading to avoidable patient trauma and wasteful healthcare spen… ▽ More Early detection of many life-threatening diseases (e.g., prostate and breast cancer) within at-risk population can improve clinical outcomes and reduce cost of care. While numerous disease-specific "screening" tests that are closer to Point-of-Care (POC) are in use for this task, their low specificity results in unnecessary biopsies, leading to avoidable patient trauma and wasteful healthcare spending. On the other hand, despite the high accuracy of Magnetic Resonance (MR) imaging in disease diagnosis, it is not used as a POC disease identification tool because of poor accessibility. The root cause of poor accessibility of MR stems from the requirement to reconstruct high-fidelity images, as it necessitates a lengthy and complex process of acquiring large quantities of high-quality k-space measurements. In this study we explore the feasibility of an ML-augmented MR pipeline that directly infers the disease sidestepping the image reconstruction process. We hypothesise that the disease classification task can be solved using a very small tailored subset of k-space data, compared to image reconstruction. Towards that end, we propose a method that performs two tasks: 1) identifies a subset of the k-space that maximizes disease identification accuracy, and 2) infers the disease directly using the identified k-space subset, bypassing the image reconstruction step. We validate our hypothesis by measuring the performance of the proposed system across multiple diseases and anatomies. We show that comparable performance to image-based classifiers, trained on images reconstructed with full k-space data, can be achieved using small quantities of data: 8% of the data for detecting multiple abnormalities in prostate and brain scans, and 5% of the data for knee abnormalities. To better understand the proposed approach and instigate future research, we provide an extensive analysis and release code. △ Less

Submitted 2 February, 2023; v1 submitted 27 January, 2023; originally announced January 2023.

Geodesic Sinkhorn for Fast and Accurate Optimal Transport on Manifolds

Authors: Guillaume Huguet, Alexander Tong, María Ramos Zapatero, Christopher J. Tape, Guy Wolf, Smita Krishnaswamy

Abstract: Efficient computation of optimal transport distance between distributions is of growing importance in data science. Sinkhorn-based methods are currently the state-of-the-art for such computations, but require $O(n^2)$ computations. In addition, Sinkhorn-based methods commonly use an Euclidean ground distance between datapoints. However, with the prevalence of manifold structured scientific data, i… ▽ More Efficient computation of optimal transport distance between distributions is of growing importance in data science. Sinkhorn-based methods are currently the state-of-the-art for such computations, but require $O(n^2)$ computations. In addition, Sinkhorn-based methods commonly use an Euclidean ground distance between datapoints. However, with the prevalence of manifold structured scientific data, it is often desirable to consider geodesic ground distance. Here, we tackle both issues by proposing Geodesic Sinkhorn -- based on diffusing a heat kernel on a manifold graph. Notably, Geodesic Sinkhorn requires only $O(n\log n)$ computation, as we approximate the heat kernel with Chebyshev polynomials based on the sparse graph Laplacian. We apply our method to the computation of barycenters of several distributions of high dimensional single cell data from patient samples undergoing chemotherapy. In particular, we define the barycentric distance as the distance between two such barycenters. Using this definition, we identify an optimal transport distance and path associated with the effect of treatment on cellular data. △ Less

Submitted 26 September, 2023; v1 submitted 1 November, 2022; originally announced November 2022.

Comments: A shorter version without the appendix appeared in the IEEE International Workshop on Machine Learning for Signal Processing (2023)

Learnable Filters for Geometric Scattering Modules

Authors: Alexander Tong, Frederik Wenkel, Dhananjay Bhaskar, Kincaid Macdonald, Jackson Grady, Michael Perlmutter, Smita Krishnaswamy, Guy Wolf

Abstract: We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the lear… ▽ More We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks. Our results show that LEGS-based networks match or outperforms popular GNNs, as well as the original geometric scattering construction, on many datasets, in particular in biochemical domains, while retaining certain mathematical properties of handcrafted (non-learned) geometric scattering. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Comments: 14 pages, 3 figures, 10 tables. arXiv admin note: substantial text overlap with arXiv:2010.02415

Manifold Interpolating Optimal-Transport Flows for Trajectory Inference

Authors: Guillaume Huguet, D. S. Magruder, Alexander Tong, Oluwadamilola Fasina, Manik Kuchroo, Guy Wolf, Smita Krishnaswamy

Abstract: We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. MIOFlow combines dynamic models, manifold learning, and optimal transport by training neural ordinary differential equations (Neural ODE) to interpolate between static population snapshots as penalized b… ▽ More We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. MIOFlow combines dynamic models, manifold learning, and optimal transport by training neural ordinary differential equations (Neural ODE) to interpolate between static population snapshots as penalized by optimal transport with manifold ground distance. Further, we ensure that the flow follows the geometry by operating in the latent space of an autoencoder that we call a geodesic autoencoder (GAE). In GAE the latent space distance between points is regularized to match a novel multiscale geodesic distance on the data manifold that we define. We show that this method is superior to normalizing flows, Schrödinger bridges and other generative models that are designed to flow from noise to data in terms of interpolating between populations. Theoretically, we link these trajectories with dynamic optimal transport. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment. △ Less

Submitted 3 November, 2022; v1 submitted 29 June, 2022; originally announced June 2022.

Comments: Presented at NeurIPS 2022, 24 pages, 7 tables, 14 figures

Optically enhanced discharge excitation and trapping of $^{39}Ar$

Authors: Y. -Q. Chu, Z. -F. Wan, F. Ritterbusch, W. -K. Hu, J. -Q. Gu, S. -M. Hu, Z. -H. Jia, W. Jiang, Z. -T. Lu, L. -T. Sun, A. -M. Tong, J. S. Wang, G. -M. Yang

Abstract: We report on a two-fold increase of the $^{39}Ar$ loading rate in an atom trap by enhancing the generation of metastable atoms in a discharge source. Additional atoms in the metastable $1s_5$ level (Paschen notation) are obtained via optically pumping both the $1s_4$ - $2p_6$ transition at 801 nm and the $1s_2$ - $2p_6$ transition at 923 nm. By solving the master equation for the corresponding six… ▽ More We report on a two-fold increase of the $^{39}Ar$ loading rate in an atom trap by enhancing the generation of metastable atoms in a discharge source. Additional atoms in the metastable $1s_5$ level (Paschen notation) are obtained via optically pumping both the $1s_4$ - $2p_6$ transition at 801 nm and the $1s_2$ - $2p_6$ transition at 923 nm. By solving the master equation for the corresponding six-level system, we identify these two transitions to be the most suitable ones and encounter a transfer process between $1s_2$ and $1s_4$ when pumping both transitions simultaneously. We calculate the previously unknown frequency shifts of the two transitions in $^{39}Ar$ and confirm the results with trap loading measurements. The demonstrated increase in the loading rate enables a corresponding decrease in the required sample size, uncertainty and measurement time for $^{39}Ar$ dating, a significant improvement for applications such as dating of ocean water and alpine ice cores. △ Less

Submitted 24 June, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: arXiv admin note: text overlap with arXiv:2005.11056

Journal ref: Phys. Rev. A 105, 063108(2022)

Time-inhomogeneous diffusion geometry and topology

Authors: Guillaume Huguet, Alexander Tong, Bastian Rieck, Jessie Huang, Manik Kuchroo, Matthew Hirn, Guy Wolf, Smita Krishnaswamy

Abstract: Diffusion condensation is a dynamic process that yields a sequence of multiscale data representations that aim to encode meaningful abstractions. It has proven effective for manifold learning, denoising, clustering, and visualization of high-dimensional data. Diffusion condensation is constructed as a time-inhomogeneous process where each step first computes and then applies a diffusion operator t… ▽ More Diffusion condensation is a dynamic process that yields a sequence of multiscale data representations that aim to encode meaningful abstractions. It has proven effective for manifold learning, denoising, clustering, and visualization of high-dimensional data. Diffusion condensation is constructed as a time-inhomogeneous process where each step first computes and then applies a diffusion operator to the data. We theoretically analyze the convergence and evolution of this process from geometric, spectral, and topological perspectives. From a geometric perspective, we obtain convergence bounds based on the smallest transition probability and the radius of the data, whereas from a spectral perspective, our bounds are based on the eigenspectrum of the diffusion kernel. Our spectral results are of particular interest since most of the literature on data diffusion is focused on homogeneous processes. From a topological perspective, we show diffusion condensation generalizes centroid-based hierarchical clustering. We use this perspective to obtain a bound based on the number of data points, independent of their location. To understand the evolution of the data geometry beyond convergence, we use topological data analysis. We show that the condensation process itself defines an intrinsic condensation homology. We use this intrinsic topology as well as the ambient persistent homology of the condensation process to study how the data changes over diffusion time. We demonstrate both types of topological information in well-understood toy examples. Our work gives theoretical insights into the convergence of diffusion condensation, and shows that it provides a link between topological and geometric data analysis. △ Less

Submitted 5 January, 2023; v1 submitted 28 March, 2022; originally announced March 2022.

MURAL: An Unsupervised Random Forest-Based Embedding for Electronic Health Record Data

Authors: Michal Gerasimiuk, Dennis Shung, Alexander Tong, Adrian Stanley, Michael Schultz, Jeffrey Ngu, Loren Laine, Guy Wolf, Smita Krishnaswamy

Abstract: A major challenge in embedding or visualizing clinical patient data is the heterogeneity of variable types including continuous lab values, categorical diagnostic codes, as well as missing or incomplete data. In particular, in EHR data, some variables are {\em missing not at random (MNAR)} but deliberately not collected and thus are a source of information. For example, lab tests may be deemed nec… ▽ More A major challenge in embedding or visualizing clinical patient data is the heterogeneity of variable types including continuous lab values, categorical diagnostic codes, as well as missing or incomplete data. In particular, in EHR data, some variables are {\em missing not at random (MNAR)} but deliberately not collected and thus are a source of information. For example, lab tests may be deemed necessary for some patients on the basis of suspected diagnosis, but not for others. Here we present the MURAL forest -- an unsupervised random forest for representing data with disparate variable types (e.g., categorical, continuous, MNAR). MURAL forests consist of a set of decision trees where node-splitting variables are chosen at random, such that the marginal entropy of all other variables is minimized by the split. This allows us to also split on MNAR variables and discrete variables in a way that is consistent with the continuous variables. The end goal is to learn the MURAL embedding of patients using average tree distances between those patients. These distances can be fed to nonlinear dimensionality reduction method like PHATE to derive visualizable embeddings. While such methods are ubiquitous in continuous-valued datasets (like single cell RNA-sequencing) they have not been used extensively in mixed variable data. We showcase the use of our method on one artificial and two clinical datasets. We show that using our approach, we can visualize and classify data more accurately than competing approaches. Finally, we show that MURAL can also be used to compare cohorts of patients via the recently proposed tree-sliced Wasserstein distances. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance

Authors: Alexander Tong, Guillaume Huguet, Dennis Shung, Amine Natik, Manik Kuchroo, Guillaume Lajoie, Guy Wolf, Smita Krishnaswamy

Abstract: In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observations in many domains. Further, in many cases the target entities for analysis are actually signals on such graphs. We propose to compare and organize such datasets of graph signals by using an earth mover's distance (EMD) with a geodesic cost over the underlying… ▽ More In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observations in many domains. Further, in many cases the target entities for analysis are actually signals on such graphs. We propose to compare and organize such datasets of graph signals by using an earth mover's distance (EMD) with a geodesic cost over the underlying graph. Typically, EMD is computed by optimizing over the cost of transporting one probability distribution to another over an underlying metric space. However, this is inefficient when computing the EMD between many signals. Here, we propose an unbalanced graph EMD that efficiently embeds the unbalanced EMD on an underlying graph into an $L^1$ space, whose metric we call unbalanced diffusion earth mover's distance (UDEMD). Next, we show how this gives distances between graph signals that are robust to noise. Finally, we apply this to organizing patients based on clinical notes, embedding cells modeled as signals on a gene graph, and organizing genes modeled as signals over a large cell graph. In each case, we show that UDEMD-based embeddings find accurate distances that are highly efficient compared to other methods. △ Less

Submitted 28 March, 2022; v1 submitted 26 July, 2021; originally announced July 2021.

Comments: 5 pages, 5 figures, ICASSP 2022

Diffusion Earth Mover's Distance and Distribution Embeddings

Authors: Alexander Tong, Guillaume Huguet, Amine Natik, Kincaid MacDonald, Manik Kuchroo, Ronald Coifman, Guy Wolf, Smita Krishnaswamy

Abstract: We propose a new fast method of measuring distances between large numbers of related high dimensional datasets called the Diffusion Earth Mover's Distance (EMD). We model the datasets as distributions supported on common data graph that is derived from the affinity matrix computed on the combined data. In such cases where the graph is a discretization of an underlying Riemannian closed manifold, w… ▽ More We propose a new fast method of measuring distances between large numbers of related high dimensional datasets called the Diffusion Earth Mover's Distance (EMD). We model the datasets as distributions supported on common data graph that is derived from the affinity matrix computed on the combined data. In such cases where the graph is a discretization of an underlying Riemannian closed manifold, we prove that Diffusion EMD is topologically equivalent to the standard EMD with a geodesic ground distance. Diffusion EMD can be computed in $\tilde{O}(n)$ time and is more accurate than similarly fast algorithms such as tree-based EMDs. We also show Diffusion EMD is fully differentiable, making it amenable to future uses in gradient-descent frameworks such as deep neural networks. Finally, we demonstrate an application of Diffusion EMD to single cell data collected from 210 COVID-19 patient samples at Yale New Haven Hospital. Here, Diffusion EMD can derive distances between patients on the manifold of cells at least two orders of magnitude faster than equally accurate methods. This distance matrix between patients can be embedded into a higher level patient manifold which uncovers structure and heterogeneity in patients. More generally, Diffusion EMD is applicable to all datasets that are massively collected in parallel in many medical and biological systems. △ Less

Submitted 27 July, 2021; v1 submitted 25 February, 2021; originally announced February 2021.

Comments: Presented at ICML 2021

COVID-19 Heterogeneity in Islands Chain Environment

Authors: Monique Chyba, Alice Koniges, Prateek Kunwar, Winnie Lau, Yuriy Mileyko, Alan Tong

Abstract: As 2021 dawns, the COVID-19 pandemic is still raging strongly as vaccines finally appear and hopes for a return to normalcy start to materialize. There is much to be learned from the pandemic's first year data that will likely remain applicable to future epidemics and possible pandemics. With only minor variants in virus strain, countries across the globe have suffered roughly the same pandemic by… ▽ More As 2021 dawns, the COVID-19 pandemic is still raging strongly as vaccines finally appear and hopes for a return to normalcy start to materialize. There is much to be learned from the pandemic's first year data that will likely remain applicable to future epidemics and possible pandemics. With only minor variants in virus strain, countries across the globe have suffered roughly the same pandemic by first glance, yet few locations exhibit the same patterns of viral spread, growth, and control as the state of Hawai'i. In this paper, we examine the data and compare the COVID-19 spread statistics between the counties of Hawai'i as well as examine several locations with similar properties to Hawai'i. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Multimodal Data Visualization and Denoising with Integrated Diffusion

Authors: Manik Kuchroo, Abhinav Godavarthi, Alexander Tong, Guy Wolf, Smita Krishnaswamy

Abstract: We propose a method called integrated diffusion for combining multimodal datasets, or data gathered via several different measurements on the same system, to create a joint data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information from both modalities. We show the… ▽ More We propose a method called integrated diffusion for combining multimodal datasets, or data gathered via several different measurements on the same system, to create a joint data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information from both modalities. We show the utility of this joint operator in data denoising, visualization and clustering, performing better than other methods to integrate and analyze multimodal data. We apply our method to multi-omic data generated from blood cells, measuring both gene expression and chromatin accessibility. Our approach better visualizes the geometry of the joint data, captures known cross-modality associations and identifies known cellular populations. More generally, integrated diffusion is broadly applicable to multimodal datasets generated in many medical and biological systems. △ Less

Submitted 3 March, 2022; v1 submitted 12 February, 2021; originally announced February 2021.

Dark Energy Survey Year 3 Results: Deep Field Optical + Near-Infrared Images and Catalogue

Authors: W. G. Hartley, A. Choi, A. Amon, R. A. Gruendl, E. Sheldon, I. Harrison, G. M. Bernstein, I. Sevilla-Noarbe, B. Yanny, K. Eckert, H. T. Diehl, A. Alarcon, M. Banerji, K. Bechtol, R. Buchs, S. Cantu, C. Conselice, J. Cordero, C. Davis, T. M. Davis, S. Dodelson, A. Drlica-Wagner, S. Everett, A. Ferté, D. Gruen , et al. (93 additional authors not shown)

Abstract: We describe the Dark Energy Survey (DES) Deep Fields, a set of images and associated multi-wavelength catalogue ($ugrizJHKs$) built from Dark Energy Camera (DECam) and Visible and Infrared Survey Telescope for Astronomy (VISTA) data. The DES Deep Fields comprise 11 fields (10 DES supernova fields plus COSMOS), with a total area of $\sim30~$ square degrees in $ugriz$ bands and reaching a maximum… ▽ More We describe the Dark Energy Survey (DES) Deep Fields, a set of images and associated multi-wavelength catalogue ($ugrizJHKs$) built from Dark Energy Camera (DECam) and Visible and Infrared Survey Telescope for Astronomy (VISTA) data. The DES Deep Fields comprise 11 fields (10 DES supernova fields plus COSMOS), with a total area of $\sim30~$ square degrees in $ugriz$ bands and reaching a maximum $i$-band depth of 26.75 (AB, $10σ$, 2 arcsec). We present a catalogue for the DES 3-year cosmology analysis of those four fields with full 8-band coverage, totalling $5.88~$ sq. deg. after masking. Numbering $2.8~$million objects ($1.6~$million post masking), our catalogue is drawn from images coadded to consistent depths of $r=25.7, i=25, z=24.3$ mag. We use a new model-fitting code, built upon established methods, to deblend sources and ensure consistent colours across the $u$-band to $Ks$-band wavelength range. We further detail the tight control we maintain over the point-spread function modelling required for the model fitting, astrometry and consistency of photometry between the four fields. The catalogue allows us to perform a careful star-galaxy separation and produces excellent photometric redshift performance (${\rm NMAD} = 0.023$ at $i<23$). The Deep-Fields catalogue will be made available as part of the cosmology data products release, following the completion of the DES 3-year weak lensing and galaxy clustering cosmology work. △ Less

Submitted 16 February, 2022; v1 submitted 23 December, 2020; originally announced December 2020.

Comments: 32 pages, 23 figures, version accepted by MNRAS. See https://www.darkenergysurvey.org/des-year-3-cosmology-results-papers/ for the full DES Y3 cosmology release

Report number: FERMILAB-PUB-20-670-AE

Journal ref: Monthly Notices of the Royal Astronomical Society, Volume 509, Issue 3, pp.3547-3579, 2022

Learning Compositional Sparse Gaussian Processes with a Shrinkage Prior

Authors: Anh Tong, Toan Tran, Hung Bui, Jaesik Choi

Abstract: Choosing a proper set of kernel functions is an important problem in learning Gaussian Process (GP) models since each kernel structure has different model complexity and data fitness. Recently, automatic kernel composition methods provide not only accurate prediction but also attractive interpretability through search-based methods. However, existing methods suffer from slow kernel composition lea… ▽ More Choosing a proper set of kernel functions is an important problem in learning Gaussian Process (GP) models since each kernel structure has different model complexity and data fitness. Recently, automatic kernel composition methods provide not only accurate prediction but also attractive interpretability through search-based methods. However, existing methods suffer from slow kernel composition learning. To tackle large-scaled data, we propose a new sparse approximate posterior for GPs, MultiSVGP, constructed from groups of inducing points associated with individual additive kernels in compositional kernels. We demonstrate that this approximation provides a better fit to learn compositional kernels given empirical observations. We also provide theoretically justification on error bound when compared to the traditional sparse GP. In contrast to the search-based approach, we present a novel probabilistic algorithm to learn a kernel composition by handling the sparsity in the kernel selection with Horseshoe prior. We demonstrate that our model can capture characteristics of time series with significant reductions in computational time and have competitive regression performance on real-world data sets. △ Less

Submitted 24 February, 2021; v1 submitted 21 December, 2020; originally announced December 2020.

Comments: AAAI 2021

Characterizing Deep Gaussian Processes via Nonlinear Recurrence Systems

Authors: Anh Tong, Jaesik Choi

Abstract: Recent advances in Deep Gaussian Processes (DGPs) show the potential to have more expressive representation than that of traditional Gaussian Processes (GPs). However, there exists a pathology of deep Gaussian processes that their learning capacities reduce significantly when the number of layers increases. In this paper, we present a new analysis in DGPs by studying its corresponding nonlinear dy… ▽ More Recent advances in Deep Gaussian Processes (DGPs) show the potential to have more expressive representation than that of traditional Gaussian Processes (GPs). However, there exists a pathology of deep Gaussian processes that their learning capacities reduce significantly when the number of layers increases. In this paper, we present a new analysis in DGPs by studying its corresponding nonlinear dynamic systems to explain the issue. Existing work reports the pathology for the squared exponential kernel function. We extend our investigation to four types of common stationary kernel functions. The recurrence relations between layers are analytically derived, providing a tighter bound and the rate of convergence of the dynamic systems. We demonstrate our finding with a number of experimental results. △ Less

Submitted 21 December, 2020; v1 submitted 19 October, 2020; originally announced October 2020.

Comments: AAAI 2021

Data-Driven Learning of Geometric Scattering Networks

Authors: Alexander Tong, Frederik Wenkel, Kincaid MacDonald, Smita Krishnaswamy, Guy Wolf

Abstract: We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the lear… ▽ More We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks. △ Less

Submitted 28 March, 2022; v1 submitted 5 October, 2020; originally announced October 2020.

Comments: 6 pages, 2 figures, 3 tables, Presented at IEEE MLSP 2021

Uncovering the Folding Landscape of RNA Secondary Structure with Deep Graph Embeddings

Authors: Egbert Castro, Andrew Benz, Alexander Tong, Guy Wolf, Smita Krishnaswamy

Abstract: Biomolecular graph analysis has recently gained much attention in the emerging field of geometric deep learning. Here we focus on organizing biomolecular graphs in ways that expose meaningful relations and variations between them. We propose a geometric scattering autoencoder (GSAE) network for learning such graph embeddings. Our embedding network first extracts rich graph features using the recen… ▽ More Biomolecular graph analysis has recently gained much attention in the emerging field of geometric deep learning. Here we focus on organizing biomolecular graphs in ways that expose meaningful relations and variations between them. We propose a geometric scattering autoencoder (GSAE) network for learning such graph embeddings. Our embedding network first extracts rich graph features using the recently proposed geometric scattering transform. Then, it leverages a semi-supervised variational autoencoder to extract a low-dimensional embedding that retains the information in these features that enable prediction of molecular properties as well as characterize graphs. We show that GSAE organizes RNA graphs both by structure and energy, accurately reflecting bistable RNA structures. Also, the model is generative and can sample new folding trajectories. △ Less

Submitted 28 March, 2022; v1 submitted 11 June, 2020; originally announced June 2020.

Comments: 10 pages, 10 figures, 4 tables, Presented at IEEE Big Data 2020

Chip Placement with Deep Reinforcement Learning

Authors: Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Sungmin Bae, Azade Nazi, Jiwoo Pak, Andy Tong, Kavya Srinivasa, William Hang, Emre Tuncer, Anand Babu, Quoc V. Le, James Laudon, Richard Ho, Roger Carpenter, Jeff Dean

Abstract: In this work, we present a learning-based approach to chip placement, one of the most complex and time-consuming stages of the chip design process. Unlike prior methods, our approach has the ability to learn from past experience and improve over time. In particular, as we train over a greater number of chip blocks, our method becomes better at rapidly generating optimized placements for previously… ▽ More In this work, we present a learning-based approach to chip placement, one of the most complex and time-consuming stages of the chip design process. Unlike prior methods, our approach has the ability to learn from past experience and improve over time. In particular, as we train over a greater number of chip blocks, our method becomes better at rapidly generating optimized placements for previously unseen chip blocks. To achieve these results, we pose placement as a Reinforcement Learning (RL) problem and train an agent to place the nodes of a chip netlist onto a chip canvas. To enable our RL policy to generalize to unseen blocks, we ground representation learning in the supervised task of predicting placement quality. By designing a neural architecture that can accurately predict reward across a wide variety of netlists and their placements, we are able to generate rich feature embeddings of the input netlists. We then use this architecture as the encoder of our policy and value networks to enable transfer learning. Our objective is to minimize PPA (power, performance, and area), and we show that, in under 6 hours, our method can generate placements that are superhuman or comparable on modern accelerator netlists, whereas existing baselines require human experts in the loop and take several weeks. △ Less

Submitted 22 April, 2020; originally announced April 2020.

DISIR: Deep Image Segmentation with Interactive Refinement

Authors: Gaston Lenczner, Bertrand Le Saux, Nicola Luminari, Adrien Chan Hon Tong, Guy Le Besnerais

Abstract: This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify t… ▽ More This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify the inputs of the network - not its weights - enabling a fast and smooth process. Through experiments on two public aerial datasets, we show that user annotations are extremely rewarding: each click corrects roughly 5000 pixels. We analyze the impact of different aspects of our framework such as the representation of the annotations, the volume of training data or the network architecture. Code is available at https://github.com/delair-ai/DISIR. △ Less

Submitted 20 August, 2020; v1 submitted 31 March, 2020; originally announced March 2020.

Comments: 8 pages, 12 figures. Published in the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences

Journal ref: XXIV ISPRS Congress, Commission II (Volume V-2-2020)

TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics

Authors: Alexander Tong, Jessie Huang, Guy Wolf, David van Dijk, Smita Krishnaswamy

Abstract: It is increasingly common to encounter data from dynamic processes captured by static cross-sectional measurements over time, particularly in biomedical settings. Recent attempts to model individual trajectories from this data use optimal transport to create pairwise matchings between time points. However, these methods cannot model continuous dynamics and non-linear paths that entities can take i… ▽ More It is increasingly common to encounter data from dynamic processes captured by static cross-sectional measurements over time, particularly in biomedical settings. Recent attempts to model individual trajectories from this data use optimal transport to create pairwise matchings between time points. However, these methods cannot model continuous dynamics and non-linear paths that entities can take in these systems. To address this issue, we establish a link between continuous normalizing flows and dynamic optimal transport, that allows us to model the expected paths of points over time. Continuous normalizing flows are generally under constrained, as they are allowed to take an arbitrary path from the source to the target distribution. We present TrajectoryNet, which controls the continuous paths taken between distributions to produce dynamic optimal transport. We show how this is particularly applicable for studying cellular dynamics in data from single-cell RNA sequencing (scRNA-seq) technologies, and that TrajectoryNet improves upon recently proposed static optimal transport-based models that can be used for interpolating cellular distributions. △ Less

Submitted 26 July, 2020; v1 submitted 9 February, 2020; originally announced February 2020.

Comments: Presented at ICML 2020

Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms

Authors: Michael Perlmutter, Alexander Tong, Feng Gao, Guy Wolf, Matthew Hirn

Abstract: The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. Recently, several works have introduced generalizations of the scattering transform for non-Euclidean settings such as graphs. Our work builds upon these constructions by introducing windowed and non-windowed geometric scattering transforms for graphs based upo… ▽ More The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. Recently, several works have introduced generalizations of the scattering transform for non-Euclidean settings such as graphs. Our work builds upon these constructions by introducing windowed and non-windowed geometric scattering transforms for graphs based upon a very general class of asymmetric wavelets. We show that these asymmetric graph scattering transforms have many of the same theoretical guarantees as their symmetric counterparts. As a result, the proposed construction unifies and extends known theoretical results for many of the existing graph scattering architectures. In doing so, this work helps bridge the gap between geometric scattering and other graph neural networks by introducing a large family of networks with provable stability and invariance guarantees. These results lay the groundwork for future deep learning architectures for graph-structured data that have learned filters and also provably have desirable theoretical properties. △ Less

Submitted 28 June, 2023; v1 submitted 14 November, 2019; originally announced November 2019.

Confirmatory Bayesian Online Change Point Detection in the Covariance Structure of Gaussian Processes

Authors: Jiyeon Han, Kyowoon Lee, Anh Tong, Jaesik Choi

Abstract: In the analysis of sequential data, the detection of abrupt changes is important in predicting future changes. In this paper, we propose statistical hypothesis tests for detecting covariance structure changes in locally smooth time series modeled by Gaussian Processes (GPs). We provide theoretically justified thresholds for the tests, and use them to improve Bayesian Online Change Point Detection… ▽ More In the analysis of sequential data, the detection of abrupt changes is important in predicting future changes. In this paper, we propose statistical hypothesis tests for detecting covariance structure changes in locally smooth time series modeled by Gaussian Processes (GPs). We provide theoretically justified thresholds for the tests, and use them to improve Bayesian Online Change Point Detection (BOCPD) by confirming statistically significant changes and non-changes. Our Confirmatory BOCPD (CBOCPD) algorithm finds multiple structural breaks in GPs even when hyperparameters are not tuned precisely. We also provide conditions under which CBOCPD provides the lower prediction error compared to BOCPD. Experimental results on synthetic and real-world datasets show that our new tests correctly detect changes in the covariance structure in GPs. The proposed algorithm also outperforms existing methods for the prediction of nonstationarity in terms of both regression error and log likelihood. △ Less

Submitted 7 February, 2020; v1 submitted 30 May, 2019; originally announced May 2019.

Comments: IJCAI 2019 Comments: 12 pages, LaTeX; Revised conditions of Theorems in section 4, results unchanged

Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz Discriminators

Authors: Alexander Tong, Guy Wolf, Smita Krishnaswamy

Abstract: Anomaly detection is of great interest in fields where abnormalities need to be identified and corrected (e.g., medicine and finance). Deep learning methods for this task often rely on autoencoder reconstruction error, sometimes in conjunction with other errors. We show that this approach exhibits intrinsic biases that lead to undesirable results. Reconstruction-based methods are sensitive to trai… ▽ More Anomaly detection is of great interest in fields where abnormalities need to be identified and corrected (e.g., medicine and finance). Deep learning methods for this task often rely on autoencoder reconstruction error, sometimes in conjunction with other errors. We show that this approach exhibits intrinsic biases that lead to undesirable results. Reconstruction-based methods are sensitive to training-data outliers and simple-to-reconstruct points. Instead, we introduce a new unsupervised Lipschitz anomaly discriminator that does not suffer from these biases. Our anomaly discriminator is trained, similar to the ones used in GANs, to detect the difference between the training data and corruptions of the training data. We show that this procedure successfully detects unseen anomalies with guarantees on those that have a certain Wasserstein distance from the data or corrupted training set. These additions allow us to show improved performance on MNIST, CIFAR10, and health record data. △ Less

Submitted 26 July, 2020; v1 submitted 25 May, 2019; originally announced May 2019.

Comments: 6 pages, 4 figures, 2 tables, presented at IEEE MLSP

Finding Archetypal Spaces Using Neural Networks

Authors: David van Dijk, Daniel Burkhardt, Matthew Amodio, Alex Tong, Guy Wolf, Smita Krishnaswamy

Abstract: Archetypal analysis is a data decomposition method that describes each observation in a dataset as a convex combination of "pure types" or archetypes. These archetypes represent extrema of a data space in which there is a trade-off between features, such as in biology where different combinations of traits provide optimal fitness for different environments. Existing methods for archetypal analysis… ▽ More Archetypal analysis is a data decomposition method that describes each observation in a dataset as a convex combination of "pure types" or archetypes. These archetypes represent extrema of a data space in which there is a trade-off between features, such as in biology where different combinations of traits provide optimal fitness for different environments. Existing methods for archetypal analysis work well when a linear relationship exists between the feature space and the archetypal space. However, such methods are not applicable to systems where the feature space is generated non-linearly from the combination of archetypes, such as in biological systems or image transformations. Here, we propose a reformulation of the problem such that the goal is to learn a non-linear transformation of the data into a latent archetypal space. To solve this problem, we introduce Archetypal Analysis network (AAnet), which is a deep neural network framework for learning and generating from a latent archetypal representation of data. We demonstrate state-of-the-art recovery of ground-truth archetypes in non-linear data domains, show AAnet can generate from data geometry rather than from data density, and use AAnet to identify biologically meaningful archetypes in single-cell gene expression data. △ Less

Submitted 13 November, 2019; v1 submitted 25 January, 2019; originally announced January 2019.

Comments: 9 pages, 10 figures, to be presented at IEEE Big Data 2019

Interpretable Neuron Structuring with Graph Spectral Regularization

Authors: Alexander Tong, David van Dijk, Jay S. Stanley III, Matthew Amodio, Kristina Yim, Rebecca Muhle, James Noonan, Guy Wolf, Smita Krishnaswamy

Abstract: While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we propose Graph Spectral Regularization for making hidden layers more interpretable without significantly impacting performance on the primary task. Taking inspiration from spatial organization and localization of… ▽ More While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we propose Graph Spectral Regularization for making hidden layers more interpretable without significantly impacting performance on the primary task. Taking inspiration from spatial organization and localization of neuron activations in biological networks, we use a graph Laplacian penalty to structure the activations within a layer. This penalty encourages activations to be smooth either on a predetermined graph or on a feature-space graph learned from the data via co-activations of a hidden layer of the neural network. We show numerous uses for this additional structure including cluster indication and visualization in biological and image data sets. △ Less

Submitted 14 February, 2020; v1 submitted 30 September, 2018; originally announced October 2018.

Comments: 12 pages, 6 figures, presented at IDA 2020

Distributed Rumor Blocking with Multiple Positive Cascades

Authors: Guangmo Amo Tong, Weili Wu, Ding-Zhu Du

Abstract: Misinformation and rumor can spread rapidly and widely through online social networks and therefore rumor controlling has become a critical issue. It is often assumed that there is a single authority whose goal is to minimize the spread of rumor by generating a positive cascade. In this paper, we study a more realistic scenario when there are multiple positive cascades generated by different agent… ▽ More Misinformation and rumor can spread rapidly and widely through online social networks and therefore rumor controlling has become a critical issue. It is often assumed that there is a single authority whose goal is to minimize the spread of rumor by generating a positive cascade. In this paper, we study a more realistic scenario when there are multiple positive cascades generated by different agents. For the multiple-cascade diffusion, we propose the P2P independent cascade (PIC) model for private social communications. The main part of this paper is an analysis of the rumor blocking effect (i.e. the number of the users activated by rumor) when the agents non-cooperatively generate the positive cascades. We show that the rumor blocking effect provided by the Nash equilibrium will not be arbitrarily worse even if the positive cascades are generated non-cooperatively. In addition, we give a discussion on how the cascade priority and activation order affect the rumor blocking problem. We experimentally examine the Nash equilibrium of the proposed games by simulations done on real social network structures. △ Less

Submitted 1 December, 2017; v1 submitted 20 November, 2017; originally announced November 2017.

Comments: under review

Cell Sequence and Mitosis Affect Fibroblast Directional Decision-Making during Chemotaxis in Tissue-Mimicking Microfluidic Mazes

Authors: Q. L. Pham, D. Chege, T. Dijamco, J. Brito, E. Stein, N. A. N. Tong, S. Basuray, R. S. Voronov

Abstract: Directed fibroblast migration is central to highly proliferative processes in regenerative medicine and developmental biology, such as wound healing and embryogenesis. However, the mechanisms by which single fibroblasts affect each other's directional decisions, while chemotaxing in microscopic tissue pores, are not well understood. Therefore, we explored the effects of two types of relevant socia… ▽ More Directed fibroblast migration is central to highly proliferative processes in regenerative medicine and developmental biology, such as wound healing and embryogenesis. However, the mechanisms by which single fibroblasts affect each other's directional decisions, while chemotaxing in microscopic tissue pores, are not well understood. Therefore, we explored the effects of two types of relevant social interactions on fibroblast PDGF-BB-induced migration in microfluidic tissue-mimicking mazes: cell sequence and mitosis. Surprisingly, it was found that in both cases, the cells display behavior that is contradictory to the chemoattractant gradient established in the maze. In case of the sequence, the cells do not like to take the same path through the maze as their predecessor, when faced with a bifurcation. To the contrary, they tend to alternate - if a leading cell takes the shorter (steeper gradient) path, the cell following it chooses the longer (weaker gradient) path, and vice versa. Additionally, we found that when a mother cell divides, its two daughters go in opposite directions (even if it means migrating against the chemoattractant gradient and overcoming on-going cell traffic). Therefore, it is apparent that fibroblasts modify each other's directional decisions in a manner that is counter-intuitive to what is expected from classical chemotaxis theory. Consequently, accounting for these effects could lead to a better understanding of tissue generation in vivo, and result in more advanced engineered tissue products in vitro. △ Less

Submitted 18 September, 2017; originally announced September 2017.

Comments: see last page for supplemental materials

Journal ref: Pham, Q.L., Rodrigues, L.N., Maximov, M.A. et al. Cel. Mol. Bioeng. (2018)

Discovering Latent Covariance Structures for Multiple Time Series

Authors: Anh Tong, Jaesik Choi

Abstract: Analyzing multivariate time series data is important to predict future events and changes of complex systems in finance, manufacturing, and administrative decisions. The expressiveness power of Gaussian Process (GP) regression methods has been significantly improved by compositional covariance structures. In this paper, we present a new GP model which naturally handles multiple time series by plac… ▽ More Analyzing multivariate time series data is important to predict future events and changes of complex systems in finance, manufacturing, and administrative decisions. The expressiveness power of Gaussian Process (GP) regression methods has been significantly improved by compositional covariance structures. In this paper, we present a new GP model which naturally handles multiple time series by placing an Indian Buffet Process (IBP) prior on the presence of shared kernels. Our selective covariance structure decomposition allows exploiting shared parameters over a set of multiple, selected time series. We also investigate the well-definedness of the models when infinite latent components are introduced. We present a pragmatic search algorithm which explores a larger structure space efficiently. Experiments conducted on five real-world data sets demonstrate that our new model outperforms existing methods in term of structure discoveries and predictive performances. △ Less

Submitted 22 May, 2019; v1 submitted 28 March, 2017; originally announced March 2017.

Comments: ICML2019, 13 pages

Automatic Generation of Probabilistic Programming from Time Series Data

Authors: Anh Tong, Jaesik Choi

Abstract: Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of model is given, constructing a probabilistic program is rather straightforward. Th… ▽ More Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of model is given, constructing a probabilistic program is rather straightforward. Thus, main focus have been to learn the best model parameters and compute marginal probabilities. In this paper, we provide a new perspective to build expressive probabilistic program from continue time series data when the structure of model is not given. The intuition behind of our method is to find a descriptive covariance structure of time series data in nonparametric Gaussian process regression. We report that such descriptive covariance structure efficiently derives a probabilistic programming description accurately. △ Less

Submitted 13 July, 2016; v1 submitted 3 July, 2016; originally announced July 2016.

Searching for Topological Symmetry in Data Haystack

Authors: Kallol Roy, Anh Tong, Jaesik Choi

Abstract: Finding interesting symmetrical topological structures in high-dimensional systems is an important problem in statistical machine learning. Limited amount of available high-dimensional data and its sensitivity to noise pose computational challenges to find symmetry. Our paper presents a new method to find local symmetries in a low-dimensional 2-D grid structure which is embedded in high-dimensiona… ▽ More Finding interesting symmetrical topological structures in high-dimensional systems is an important problem in statistical machine learning. Limited amount of available high-dimensional data and its sensitivity to noise pose computational challenges to find symmetry. Our paper presents a new method to find local symmetries in a low-dimensional 2-D grid structure which is embedded in high-dimensional structure. To compute the symmetry in a grid structure, we introduce three legal grid moves (i) Commutation (ii) Cyclic Permutation (iii) Stabilization on sets of local grid squares, grid blocks. The three grid moves are legal transformations as they preserve the statistical distribution of hamming distances in each grid block. We propose and coin the term of grid symmetry of data on the 2-D data grid as the invariance of statistical distributions of hamming distance are preserved after a sequence of grid moves. We have computed and analyzed the grid symmetry of data on multivariate Gaussian distributions and Gamma distributions with noise. △ Less

Submitted 11 March, 2016; originally announced March 2016.