Showing 1–40 of 40 results for author: Hemati, S

DFML: Decentralized Federated Mutual Learning

Authors: Yasser H. Khalil, Amir H. Estiri, Mahdi Beitollahi, Nader Asadi, Sobhan Hemati, Xu Li, Guojun Zhang, Xi Chen

Abstract: In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure. Additionally, contemporary devices inherently exhibit model and data heterogeneity. Existing work lacks a Decentralized FL (DFL) framework capable of accommodating such heterogeneity without imposing architectural… ▽ More In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure. Additionally, contemporary devices inherently exhibit model and data heterogeneity. Existing work lacks a Decentralized FL (DFL) framework capable of accommodating such heterogeneity without imposing architectural restrictions or assuming the availability of public data. To address these issues, we propose a Decentralized Federated Mutual Learning (DFML) framework that is serverless, supports nonrestrictive heterogeneous models, and avoids reliance on public data. DFML effectively handles model and data heterogeneity through mutual learning, which distills knowledge between clients, and cyclically varying the amount of supervision and distillation signals. Extensive experimental results demonstrate consistent effectiveness of DFML in both convergence speed and global accuracy, outperforming prevalent baselines under various conditions. For example, with the CIFAR-100 dataset and 50 clients, DFML achieves a substantial increase of +17.20% and +19.95% in global accuracy under Independent and Identically Distributed (IID) and non-IID data shifts, respectively. △ Less

Submitted 2 February, 2024; originally announced February 2024.

Parametric Feature Transfer: One-shot Federated Learning with Foundation Models

Authors: Mahdi Beitollahi, Alex Bie, Sobhan Hemati, Leo Maxime Brunswic, Xu Li, Xi Chen, Guojun Zhang

Abstract: In one-shot federated learning (FL), clients collaboratively train a global model in a single round of communication. Existing approaches for one-shot FL enhance communication efficiency at the expense of diminished accuracy. This paper introduces FedPFT (Federated Learning with Parametric Feature Transfer), a methodology that harnesses the transferability of foundation models to enhance both accu… ▽ More In one-shot federated learning (FL), clients collaboratively train a global model in a single round of communication. Existing approaches for one-shot FL enhance communication efficiency at the expense of diminished accuracy. This paper introduces FedPFT (Federated Learning with Parametric Feature Transfer), a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL. The approach involves transferring per-client parametric models (specifically, Gaussian mixtures) of features extracted from foundation models. Subsequently, each parametric model is employed to generate synthetic features for training a classifier head. Experimental results on eight datasets demonstrate that FedPFT enhances the communication-accuracy frontier in both centralized and decentralized FL scenarios, as well as across diverse data-heterogeneity settings such as covariate shift and task shift, with improvements of up to 20.6%. Additionally, FedPFT adheres to the data minimization principle of FL, as clients do not send real features. We demonstrate that sending real features is vulnerable to potent reconstruction attacks. Moreover, we show that FedPFT is amenable to formal privacy guarantees via differential privacy, demonstrating favourable privacy-accuracy tradeoffs. △ Less

Submitted 2 February, 2024; originally announced February 2024.

Comments: 20 pages, 12 figures

A Convex Optimization Approach to Compute Trapping Regions for Lossless Quadratic Systems

Authors: Shih-Chi Liao, A. Leonid Heide, Maziar S. Hemati, Peter J. Seiler

Abstract: Quadratic systems with lossless quadratic terms arise in many applications, including models of atmosphere and incompressible fluid flows. Such systems have a trapping region if all trajectories eventually converge to and stay within a bounded set. Conditions for the existence and characterization of trapping regions have been established in prior works for boundedness analysis. However, prior sol… ▽ More Quadratic systems with lossless quadratic terms arise in many applications, including models of atmosphere and incompressible fluid flows. Such systems have a trapping region if all trajectories eventually converge to and stay within a bounded set. Conditions for the existence and characterization of trapping regions have been established in prior works for boundedness analysis. However, prior solutions have used non-convex optimization methods, resulting in conservative estimates. In this paper, we build on this prior work and provide a convex semidefinite programming condition for the existence of a trapping region. The condition allows precise verification or falsification of the existence of a trapping region. If a trapping region exists, then we provide a second semidefinite program to compute the least conservative trapping region in the form of a ball. Two low-dimensional systems are provided as examples to illustrate the results. A third high-dimensional example is also included to demonstrate that the computation required for the analysis can be scaled to systems of up to $\sim O(100)$ states. The proposed method provides a precise and computationally efficient numerical approach for computing trapping regions. We anticipate this work will benefit future studies on modeling and control of lossless quadratic dynamical systems. △ Less

Submitted 9 January, 2024; originally announced January 2024.

Analysis and Validation of Image Search Engines in Histopathology

Authors: Isaiah Lahr, Saghir Alfasly, Peyman Nejat, Jibran Khan, Luke Kottom, Vaishnavi Kumbhar, Areej Alsaafin, Abubakr Shafique, Sobhan Hemati, Ghazal Alabtah, Nneka Comfere, Dennis Murphee, Aaron Mangold, Saba Yasir, Chady Meroueh, Lisa Boardman, Vijay H. Shah, Joaquin J. Garcia, H. R. Tizhoosh

Abstract: Searching for similar images in archives of histology and histopathology images is a crucial task that may aid in patient matching for various purposes, ranging from triaging and diagnosis to prognosis and prediction. Whole slide images (WSIs) are highly detailed digital representations of tissue specimens mounted on glass slides. Matching WSI to WSI can serve as the critical method for patient ma… ▽ More Searching for similar images in archives of histology and histopathology images is a crucial task that may aid in patient matching for various purposes, ranging from triaging and diagnosis to prognosis and prediction. Whole slide images (WSIs) are highly detailed digital representations of tissue specimens mounted on glass slides. Matching WSI to WSI can serve as the critical method for patient matching. In this paper, we report extensive analysis and validation of four search methods bag of visual words (BoVW), Yottixel, SISH, RetCCL, and some of their potential variants. We analyze their algorithms and structures and assess their performance. For this evaluation, we utilized four internal datasets ($1269$ patients) and three public datasets ($1207$ patients), totaling more than $200,000$ patches from $38$ different classes/subtypes across five primary sites. Certain search engines, for example, BoVW, exhibit notable efficiency and speed but suffer from low accuracy. Conversely, search engines like Yottixel demonstrate efficiency and speed, providing moderately accurate results. Recent proposals, including SISH, display inefficiency and yield inconsistent outcomes, while alternatives like RetCCL prove inadequate in both accuracy and efficiency. Further research is imperative to address the dual aspects of accuracy and minimal storage requirements in histopathological image search. △ Less

Submitted 8 June, 2024; v1 submitted 6 January, 2024; originally announced January 2024.

Cross Domain Generative Augmentation: Domain Generalization with Latent Diffusion Models

Authors: Sobhan Hemati, Mahdi Beitollahi, Amir Hossein Estiri, Bassel Al Omari, Xi Chen, Guojun Zhang

Abstract: Despite the huge effort in developing novel regularizers for Domain Generalization (DG), adding simple data augmentation to the vanilla ERM which is a practical implementation of the Vicinal Risk Minimization principle (VRM) \citep{chapelle2000vicinal} outperforms or stays competitive with many of the proposed regularizers. The VRM reduces the estimation error in ERM by replacing the point-wise ke… ▽ More Despite the huge effort in developing novel regularizers for Domain Generalization (DG), adding simple data augmentation to the vanilla ERM which is a practical implementation of the Vicinal Risk Minimization principle (VRM) \citep{chapelle2000vicinal} outperforms or stays competitive with many of the proposed regularizers. The VRM reduces the estimation error in ERM by replacing the point-wise kernel estimates with a more precise estimation of true data distribution that reduces the gap between data points \textbf{within each domain}. However, in the DG setting, the estimation error of true data distribution by ERM is mainly caused by the distribution shift \textbf{between domains} which cannot be fully addressed by simple data augmentation techniques within each domain. Inspired by this limitation of VRM, we propose a novel data augmentation named Cross Domain Generative Augmentation (CDGA) that replaces the pointwise kernel estimates in ERM with new density estimates in the \textbf{vicinity of domain pairs} so that the gap between domains is further reduced. To this end, CDGA, which is built upon latent diffusion models (LDM), generates synthetic images to fill the gap between all domains and as a result, reduces the non-iidness. We show that CDGA outperforms SOTA DG methods under the Domainbed benchmark. To explain the effectiveness of CDGA, we generate more than 5 Million synthetic images and perform extensive ablation studies including data scaling laws, distribution visualization, domain shift quantification, adversarial robustness, and loss landscape analysis. △ Less

Submitted 8 December, 2023; originally announced December 2023.

An optimization framework for analyzing nonlinear stability due to sparse finite-amplitude perturbations

Authors: A. Leonid Heide, Maziar S. Hemati

Abstract: Recent works have established the utility of sparsity-promoting norms for extracting spatially-localized instability mechanisms in fluid flows, with possible implications for flow control. However, these prior works have focused on linear dynamics of infinitesimal perturbations about a given baseflow. In this paper, we propose an optimization framework for computing sparse finite-amplitude perturb… ▽ More Recent works have established the utility of sparsity-promoting norms for extracting spatially-localized instability mechanisms in fluid flows, with possible implications for flow control. However, these prior works have focused on linear dynamics of infinitesimal perturbations about a given baseflow. In this paper, we propose an optimization framework for computing sparse finite-amplitude perturbations that maximize transient growth in nonlinear systems. A variational approach is used to derive the first-order necessary conditions for optimality, which form the basis of our iterative direct-adjoint looping numerical solution algorithm. When applied to a reduced-order model of a sinusoidal shear flow at $Re=20$, our framework identifies that energy injection into a single vortical mode yields comparable energy amplification as the non-sparse optimal solution with energy distributed across all modes. Energy injection into three additional modes results in an identical transient growth as the non-sparse case. Subsequent analysis of the dynamic response of the flow establishes that these sparse optimal perturbations trigger many of the same nonlinear modal interactions that give rise to transient growth when all modes are perturbed in an optimal manner. It is also observed that as perturbation amplitude is increased, the maximum transient growth is achieved at an earlier time. Our results highlight the power of the proposed optimization framework for revealing dominant nonlinear modal interactions and sparse perturbation mechanisms for transient growth and instability in fluid flows. We anticipate the approach will be a useful tool in guiding the design of flow control strategies in the future. △ Less

Submitted 15 November, 2023; originally announced November 2023.

When is a Foundation Model a Foundation Model

Authors: Saghir Alfasly, Peyman Nejat, Sobhan Hemati, Jibran Khan, Isaiah Lahr, Areej Alsaafin, Abubakr Shafique, Nneka Comfere, Dennis Murphree, Chady Meroueh, Saba Yasir, Aaron Mangold, Lisa Boardman, Vijay Shah, Joaquin J. Garcia, H. R. Tizhoosh

Abstract: Recently, several studies have reported on the fine-tuning of foundation models for image-text modeling in the field of medicine, utilizing images from online data sources such as Twitter and PubMed. Foundation models are large, deep artificial neural networks capable of learning the context of a specific domain through training on exceptionally extensive datasets. Through validation, we have obse… ▽ More Recently, several studies have reported on the fine-tuning of foundation models for image-text modeling in the field of medicine, utilizing images from online data sources such as Twitter and PubMed. Foundation models are large, deep artificial neural networks capable of learning the context of a specific domain through training on exceptionally extensive datasets. Through validation, we have observed that the representations generated by such models exhibit inferior performance in retrieval tasks within digital pathology when compared to those generated by significantly smaller, conventional deep networks. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Understanding Hessian Alignment for Domain Generalization

Authors: Sobhan Hemati, Guojun Zhang, Amir Estiri, Xi Chen

Abstract: Out-of-distribution (OOD) generalization is a critical ability for deep learning models in many real-world scenarios including healthcare and autonomous vehicles. Recently, different techniques have been proposed to improve OOD generalization. Among these methods, gradient-based regularizers have shown promising performance compared with other competitors. Despite this success, our understanding o… ▽ More Out-of-distribution (OOD) generalization is a critical ability for deep learning models in many real-world scenarios including healthcare and autonomous vehicles. Recently, different techniques have been proposed to improve OOD generalization. Among these methods, gradient-based regularizers have shown promising performance compared with other competitors. Despite this success, our understanding of the role of Hessian and gradient alignment in domain generalization is still limited. To address this shortcoming, we analyze the role of the classifier's head Hessian matrix and gradient in domain generalization using recent OOD theory of transferability. Theoretically, we show that spectral norm between the classifier's head Hessian matrices across domains is an upper bound of the transfer measure, a notion of distance between target and source domains. Furthermore, we analyze all the attributes that get aligned when we encourage similarity between Hessians and gradients. Our analysis explains the success of many regularizers like CORAL, IRM, V-REx, Fish, IGA, and Fishr as they regularize part of the classifier's head Hessian and/or gradient. Finally, we propose two simple yet effective methods to match the classifier's head Hessians and gradients in an efficient way, based on the Hessian Gradient Product (HGP) and Hutchinson's method (Hutchinson), and without directly calculating Hessians. We validate the OOD generalization ability of proposed methods in different scenarios, including transferability, severe correlation shift, label shift and diversity shift. Our results show that Hessian alignment methods achieve promising performance on various OOD benchmarks. The code is available at \url{https://github.com/huawei-noah/Federated-Learning/tree/main/HessianAlignment}. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: ICCV 2023

On the convexity of static output feedback control synthesis for systems with lossless nonlinearities

Authors: Talha Mushtaq, Peter Seiler, Maziar S. Hemati

Abstract: Computing a stabilizing static output-feedback (SOF) controller is an NP-hard problem, in general. Yet, these controllers have amassed popularity in recent years because of their practical use in feedback control applications, such as fluid flow control and sensor/actuator selection. The inherent difficulty of synthesizing SOF controllers is rooted in solving a series of non-convex problems that m… ▽ More Computing a stabilizing static output-feedback (SOF) controller is an NP-hard problem, in general. Yet, these controllers have amassed popularity in recent years because of their practical use in feedback control applications, such as fluid flow control and sensor/actuator selection. The inherent difficulty of synthesizing SOF controllers is rooted in solving a series of non-convex problems that make the solution computationally intractable. In this note, we show that SOF synthesis is a convex problem for the specific case of systems with a lossless (i.e., energy-conserving) nonlinearity. Our proposed method ensures asymptotic stability of an SOF controller by enforcing the lossless behavior of the nonlinearity using a quadratic constraint approach. In particular, we formulate a bilinear matrix inequality~(BMI) using the approach, then show that the resulting BMI can be recast as a linear matrix inequality (LMI). The resulting LMI is a convex problem whose feasible solution, if one exists, yields an asymptotically stabilizing SOF controller. △ Less

Submitted 6 July, 2023; originally announced July 2023.

Comments: Submitted to Automatica as a Technical Communique

Exact Solution for the Rank-One Structured Singular Value with Repeated Complex Full-Block Uncertainty

Authors: Talha Mushtaq, Peter Seiler, Maziar S. Hemati

Abstract: In this note, we present an exact solution for the structured singular value (SSV) of rank-one complex matrices with repeated complex full-block uncertainty. A key step in the proof is the use of Von Neumman's trace inequality. Previous works provided exact solutions for rank-one SSV when the uncertainty contains repeated (real or complex) scalars and/or non-repeated complex full-block uncertainti… ▽ More In this note, we present an exact solution for the structured singular value (SSV) of rank-one complex matrices with repeated complex full-block uncertainty. A key step in the proof is the use of Von Neumman's trace inequality. Previous works provided exact solutions for rank-one SSV when the uncertainty contains repeated (real or complex) scalars and/or non-repeated complex full-block uncertainties. Our result with repeated complex full-blocks contains, as special cases, the previous results for repeated complex scalars and/or non-repeated complex full-block uncertainties. The repeated complex full-block uncertainty has recently gained attention in the context of incompressible fluid flows. Specifically, it has been used to analyze the effect of the convective nonlinearity in the incompressible Navier-Stokes equation (NSE). SSV analysis with repeated full-block uncertainty has led to an improved understanding of the underlying flow physics. We demonstrate our method on a turbulent channel flow model as an example. △ Less

Submitted 5 July, 2023; originally announced July 2023.

Mathematical Challenges in Deep Learning

Authors: Vahid Partovi Nia, Guojun Zhang, Ivan Kobyzev, Michael R. Metel, Xinlin Li, Ke Sun, Sobhan Hemati, Masoud Asgharian, Linglong Kong, Wulong Liu, Boxing Chen

Abstract: Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challenges to this field with applications in cell phones, personal computers, autonomous cars, and wireless base stations. Here we list a set of problems, ranging from training, inference, generalization bound, and optimizati… ▽ More Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challenges to this field with applications in cell phones, personal computers, autonomous cars, and wireless base stations. Here we list a set of problems, ranging from training, inference, generalization bound, and optimization with some formalism to communicate these challenges with mathematicians, statisticians, and theoretical computer scientists. This is a subjective view of the research questions in deep learning that benefits the tech industry in long run. △ Less

Submitted 24 March, 2023; originally announced March 2023.

Robust Local Stabilization of Nonlinear Systems with Controller-Dependent Norm Bounds: A Convex Approach with Input-Output Sampling

Authors: Sze Kwan Cheah, Diganta Bhattacharjee, Maziar S. Hemati, Ryan J. Caverly

Abstract: This letter presents a framework for synthesizing a robust full-state feedback controller for systems with unknown nonlinearities. Our approach characterizes input-output behavior of the nonlinearities in terms of local norm bounds using available sampled data corresponding to a known region about an equilibrium point. A challenge in this approach is that if the nonlinearities have explicit depend… ▽ More This letter presents a framework for synthesizing a robust full-state feedback controller for systems with unknown nonlinearities. Our approach characterizes input-output behavior of the nonlinearities in terms of local norm bounds using available sampled data corresponding to a known region about an equilibrium point. A challenge in this approach is that if the nonlinearities have explicit dependence on the control inputs, an a priori selection of the control input sampling region is required to determine the local norm bounds. This leads to a "chicken and egg" problem, where the local norm bounds are required for controller synthesis, but the region of control inputs needed to be characterized cannot be known prior to synthesis of the controller. To tackle this issue, we constrain the closed-loop control inputs within the sampling region while synthesizing the controller. As the resulting synthesis problem is non-convex, three semi-definite programs (SDPs) are obtained through convex relaxations of the main problem, and an iterative algorithm is constructed using these SDPs for control synthesis. Two numerical examples are included to demonstrate the effectiveness of the proposed algorithm. △ Less

Submitted 6 December, 2022; originally announced December 2022.

Comments: Accepted for publication in the IEEE Control Systems Letters (L-CSS)

Structured Singular Value of a Repeated Complex Full-Block Uncertainty

Authors: Talha Mushtaq, Diganta Bhattacharjee, Peter Seiler, Maziar S. Hemati

Abstract: The structured singular value (SSV), or mu, is used to assess the robust stability and performance of an uncertain linear time-invariant system. Existing algorithms compute upper and lower bounds on the SSV for structured uncertainties that contain repeated (real or complex) scalars and/or non-repeated complex full blocks. This paper presents algorithms to compute bounds on the SSV for the case of… ▽ More The structured singular value (SSV), or mu, is used to assess the robust stability and performance of an uncertain linear time-invariant system. Existing algorithms compute upper and lower bounds on the SSV for structured uncertainties that contain repeated (real or complex) scalars and/or non-repeated complex full blocks. This paper presents algorithms to compute bounds on the SSV for the case of repeated complex full blocks. This specific class of uncertainty is relevant for the input output analysis of many convective systems, such as fluid flows. Specifically, we present a power iteration to compute a lower bound on SSV for the case of repeated complex full blocks. This generalizes existing power iterations for repeated complex scalar and non-repeated complex full blocks. The upper bound can be formulated as a semi-definite program (SDP), which we solve using a standard interior-point method to compute optimal scaling matrices associated with the repeated full blocks. Our implementation of the method only requires gradient information, which improves the computational efficiency of the method. Finally, we test our proposed algorithms on an example model of incompressible fluid flow. The proposed methods provide less conservative bounds as compared to prior results, which ignore the repeated full block structure. △ Less

Submitted 5 January, 2024; v1 submitted 10 November, 2022; originally announced November 2022.

Comments: Submitted to the International Journal of Robust and Nonlinear Control

Quadratic Constraints for Local Stability Analysis of Quadratic Systems

Authors: Shih-Chi Liao, Maziar S. Hemati, Peter Seiler

Abstract: This paper proposes new quadratic constraints (QCs) to bound a quadratic polynomial. Such QCs can be used in dissipation ineqaulities to analyze the stability and performance of nonlinear systems with quadratic vector fields. The proposed QCs utilize the sign-indefiniteness of certain classes of quadratic polynomials. These new QCs provide a tight bound on the quadratic terms along specific direct… ▽ More This paper proposes new quadratic constraints (QCs) to bound a quadratic polynomial. Such QCs can be used in dissipation ineqaulities to analyze the stability and performance of nonlinear systems with quadratic vector fields. The proposed QCs utilize the sign-indefiniteness of certain classes of quadratic polynomials. These new QCs provide a tight bound on the quadratic terms along specific directions. This reduces the conservatism of the QC bounds as compared to the QCs in previous work. Two numerical examples of local stability analysis are provided to demonstrate the effectiveness of the proposed QCs. △ Less

Submitted 8 September, 2022; originally announced September 2022.

Comments: 6 pages, 4 figures, to be published in IEEE Conference on Decision and Control 2022

Learning Binary and Sparse Permutation-Invariant Representations for Fast and Memory Efficient Whole Slide Image Search

Authors: Sobhan Hemati, Shivam Kalra, Morteza Babaie, H. R. Tizhoosh

Abstract: Learning suitable Whole slide images (WSIs) representations for efficient retrieval systems is a non-trivial task. The WSI embeddings obtained from current methods are in Euclidean space not ideal for efficient WSI retrieval. Furthermore, most of the current methods require high GPU memory due to the simultaneous processing of multiple sets of patches. To address these challenges, we propose a nov… ▽ More Learning suitable Whole slide images (WSIs) representations for efficient retrieval systems is a non-trivial task. The WSI embeddings obtained from current methods are in Euclidean space not ideal for efficient WSI retrieval. Furthermore, most of the current methods require high GPU memory due to the simultaneous processing of multiple sets of patches. To address these challenges, we propose a novel framework for learning binary and sparse WSI representations utilizing a deep generative modelling and the Fisher Vector. We introduce new loss functions for learning sparse and binary permutation-invariant WSI representations that employ instance-based training achieving better memory efficiency. The learned WSI representations are validated on The Cancer Genomic Atlas (TCGA) and Liver-Kidney-Stomach (LKS) datasets. The proposed method outperforms Yottixel (a recent search engine for histopathology images) both in terms of retrieval accuracy and speed. Further, we achieve competitive performance against SOTA on the public benchmark LKS dataset for WSI classification. △ Less

Submitted 23 September, 2022; v1 submitted 29 August, 2022; originally announced August 2022.

Beyond Neighbourhood-Preserving Transformations for Quantization-Based Unsupervised Hashing

Authors: Sobhan Hemati, H. R. Tizhoosh

Abstract: An effective unsupervised hashing algorithm leads to compact binary codes preserving the neighborhood structure of data as much as possible. One of the most established schemes for unsupervised hashing is to reduce the dimensionality of data and then find a rigid (neighbourhood-preserving) transformation that reduces the quantization error. Although employing rigid transformations is effective, we… ▽ More An effective unsupervised hashing algorithm leads to compact binary codes preserving the neighborhood structure of data as much as possible. One of the most established schemes for unsupervised hashing is to reduce the dimensionality of data and then find a rigid (neighbourhood-preserving) transformation that reduces the quantization error. Although employing rigid transformations is effective, we may not reduce quantization loss to the ultimate limits. As well, reducing dimensionality and quantization loss in two separate steps seems to be sub-optimal. Motivated by these shortcomings, we propose to employ both rigid and non-rigid transformations to reduce quantization error and dimensionality simultaneously. We relax the orthogonality constraint on the projection in a PCA-formulation and regularize this by a quantization term. We show that both the non-rigid projection matrix and rotation matrix contribute towards minimizing quantization loss but in different ways. A scalable nested coordinate descent approach is proposed to optimize this mixed-integer optimization problem. We evaluate the proposed method on five public benchmark datasets providing almost half a million images. Comparative results indicate that the proposed method mostly outperforms state-of-art linear methods and competes with end-to-end deep solutions. △ Less

Submitted 1 October, 2021; originally announced October 2021.

Comments: Under revision on Pattern Recognition Letter

Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images

Authors: Shivam Kalra, Mohammed Adnan, Sobhan Hemati, Taher Dehkharghanian, Shahryar Rahnamayan, Hamid Tizhoosh

Abstract: Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The… ▽ More Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The feature extractor model is fine-tuned using hierarchical target labels of WSIs, i.e., anatomic site and primary diagnosis. In the second stage, a set of encoded patch-level features from a WSI is used to compute the primary diagnosis probability through the proposed Pay Attention with Focus scheme, an attention-weighted averaging of predicted probabilities for all patches of a mosaic modulated by a trainable focal factor. Experimental results show that the proposed model can be robust, and effective for the classification of WSIs. △ Less

Submitted 11 June, 2021; originally announced June 2021.

Comments: Accepted in MICCAI, 2021

Estimating Regions of Attraction for Transitional Flows using Quadratic Constraints

Authors: Aniketh Kalur, Talha Mushtaq, Peter Seiler, Maziar S. Hemati

Abstract: This letter describes a method for estimating regions of attraction and bounds on permissible perturbation amplitudes in nonlinear fluids systems. The proposed approach exploits quadratic constraints between the inputs and outputs of the nonlinearity on elliptical sets. This approach reduces conservatism and improves estimates for regions of attraction and bounds on permissible perturbation amplit… ▽ More This letter describes a method for estimating regions of attraction and bounds on permissible perturbation amplitudes in nonlinear fluids systems. The proposed approach exploits quadratic constraints between the inputs and outputs of the nonlinearity on elliptical sets. This approach reduces conservatism and improves estimates for regions of attraction and bounds on permissible perturbation amplitudes over related methods that employ quadratic constraints on spherical sets. We present and investigate two algorithms for performing the analysis: an iterative method that refines the analysis by solving a sequence of semi-definite programs, and another based on solving a generalized eigenvalue problem with lower computational complexity, but at the cost of some precision in the final solution. The proposed algorithms are demonstrated on low-order mechanistic models of transitional flows. We further compare accuracy and computational complexity with analysis based on sum-of-squares optimization and direct-adjoint looping methods. △ Less

Submitted 17 May, 2021; v1 submitted 9 March, 2021; originally announced March 2021.

Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides

Authors: Abtin Riasatian, Morteza Babaie, Danial Maleki, Shivam Kalra, Mojtaba Valipour, Sobhan Hemati, Manit Zaveri, Amir Safarpoor, Sobhan Shafiei, Mehdi Afshari, Maral Rasoolijaberi, Milad Sikaroudi, Mohd Adnan, Sultaan Shah, Charles Choi, Savvas Damaskinos, Clinton JV Campbell, Phedias Diamandis, Liron Pantanowitz, Hany Kashani, Ali Ghodsi, H. R. Tizhoosh

Abstract: Feature vectors provided by pre-trained deep artificial neural networks have become a dominant source for image representation in recent literature. Their contribution to the performance of image analysis can be improved through finetuning. As an ultimate solution, one might even train a deep network from scratch with the domain-relevant images, a highly desirable option which is generally impeded… ▽ More Feature vectors provided by pre-trained deep artificial neural networks have become a dominant source for image representation in recent literature. Their contribution to the performance of image analysis can be improved through finetuning. As an ultimate solution, one might even train a deep network from scratch with the domain-relevant images, a highly desirable option which is generally impeded in pathology by lack of labeled images and the computational expense. In this study, we propose a new network, namely KimiaNet, that employs the topology of the DenseNet with four dense blocks, fine-tuned and trained with histopathology images in different configurations. We used more than 240,000 image patches with 1000x1000 pixels acquired at 20x magnification through our proposed "highcellularity mosaic" approach to enable the usage of weak labels of 7,126 whole slide images of formalin-fixed paraffin-embedded human pathology samples publicly available through the The Cancer Genome Atlas (TCGA) repository. We tested KimiaNet using three public datasets, namely TCGA, endometrial cancer images, and colorectal cancer images by evaluating the performance of search and classification when corresponding features of different networks are used for image representation. As well, we designed and trained multiple convolutional batch-normalized ReLU (CBR) networks. The results show that KimiaNet provides superior results compared to the original DenseNet and smaller CBR networks when used as feature extractor to represent histopathology images. △ Less

Submitted 19 January, 2021; originally announced January 2021.

Model-based multi-sensor fusion for reconstructing wall-bounded turbulence

Authors: Mengying Wang, C. Vamsi Krishna, Mitul Luhar, Maziar S. Hemati

Abstract: Wall-bounded turbulent flows can be challenging to measure within experiments due to the breadth of spatial and temporal scales inherent in such flows. Instrumentation capable of obtaining time-resolved data (e.g., Hot-Wire Anemometers) tends to be restricted to spatially-localized point measurements; likewise, instrumentation capable of achieving spatially-resolved field measurements (e.g., Parti… ▽ More Wall-bounded turbulent flows can be challenging to measure within experiments due to the breadth of spatial and temporal scales inherent in such flows. Instrumentation capable of obtaining time-resolved data (e.g., Hot-Wire Anemometers) tends to be restricted to spatially-localized point measurements; likewise, instrumentation capable of achieving spatially-resolved field measurements (e.g., Particle Image Velocimetry) tends to lack the sampling rates needed to attain time-resolution in many such flows. In this study, we propose to fuse measurements from multi-rate and multi-fidelity sensors with predictions from a physics-based model to reconstruct the spatiotemporal evolution of a wall-bounded turbulent flow. A "fast" filter is formulated to assimilate high-rate point measurements with estimates from a linear model derived from the Navier-Stokes equations. Additionally, a "slow" filter is used to update the reconstruction every time a new field measurement becomes available. By marching through the data both forward and backward in time, we are able to reconstruct the turbulent flow with greater spatiotemporal resolution than either sensing modality alone. We demonstrate the approach using direct numerical simulations of a turbulent channel flow from the Johns Hopkins Turbulence Database. A statistical analysis of the model-based multi-sensor fusion approach is also conducted. △ Less

Submitted 13 January, 2021; originally announced January 2021.

Reducing transient energy growth in a channel flow using static output feedback control

Authors: Huaijin Yao, Yiyang Sun, Talha Mushtaq, Maziar S. Hemati

Abstract: Transient energy growth of flow perturbations is an important mechanism for laminar-to-turbulent transition that can be mitigated with feedback control. Linear quadratic optimal control strategies have shown some success in reducing transient energy growth and suppressing transition, but acceptable worst-case performance can be difficult to achieve using sensor-based output feedback control. In th… ▽ More Transient energy growth of flow perturbations is an important mechanism for laminar-to-turbulent transition that can be mitigated with feedback control. Linear quadratic optimal control strategies have shown some success in reducing transient energy growth and suppressing transition, but acceptable worst-case performance can be difficult to achieve using sensor-based output feedback control. In this study, we investigate static output feedback controllers for reducing transient energy growth of flow perturbations within linear and nonlinear simulations of a sub-critical channel flow. A static output feedback linear quadratic regulator~(SOF-LQR) is designed to reduce the worst-case transient energy growth due to flow perturbations. The controller directly uses wall-based measurements to optimally regulate the flow with wall-normal blowing and suction from the upper and lower channel walls. Optimal static output feedback gains are computed using a modified Anderson-Moore algorithm that accelerates the iterative solution of the synthesis problem by leveraging Armijo-type adaptations. We show that SOF-LQR controllers can reduce the worst-case transient energy growth due to flow perturbations. Our results also indicate that SOF-LQR controllers exhibit robustness to Reynolds number variations. Further, direct numerical simulations show that the designed SOF-LQR controllers increase laminar-to-turbulent transition thresholds under streamwise disturbances and delay transition under spanwise disturbances. The results of this study highlight the advantages of SOF-LQR controllers and create opportunities for realizing improved transition control strategies in the future. △ Less

Submitted 24 December, 2020; originally announced December 2020.

A non-alternating graph hashing algorithm for large scale image search

Authors: Sobhan Hemati, Mohammad Hadi Mehdizavareh, Shojaeddin Chenouri, Hamid R Tizhoosh

Abstract: In the era of big data, methods for improving memory and computational efficiency have become crucial for successful deployment of technologies. Hashing is one of the most effective approaches to deal with computational limitations that come with big data. One natural way for formulating this problem is spectral hashing that directly incorporates affinity to learn binary codes. However, due to bin… ▽ More In the era of big data, methods for improving memory and computational efficiency have become crucial for successful deployment of technologies. Hashing is one of the most effective approaches to deal with computational limitations that come with big data. One natural way for formulating this problem is spectral hashing that directly incorporates affinity to learn binary codes. However, due to binary constraints, the optimization becomes intractable. To mitigate this challenge, different relaxation approaches have been proposed to reduce the computational load of obtaining binary codes and still attain a good solution. The problem with all existing relaxation methods is resorting to one or more additional auxiliary variables to attain high quality binary codes while relaxing the problem. The existence of auxiliary variables leads to coordinate descent approach which increases the computational complexity. We argue that introducing these variables is unnecessary. To this end, we propose a novel relaxed formulation for spectral hashing that adds no additional variables to the problem. Furthermore, instead of solving the problem in original space where number of variables is equal to the data points, we solve the problem in a much smaller space and retrieve the binary codes from this solution. This trick reduces both the memory and computational complexity at the same time. We apply two optimization techniques, namely projected gradient and optimization on manifold, to obtain the solution. Using comprehensive experiments on four public datasets, we show that the proposed efficient spectral hashing (ESH) algorithm achieves highly competitive retrieval performance compared with state of the art at low complexity. △ Less

Submitted 19 June, 2021; v1 submitted 24 December, 2020; originally announced December 2020.

Comments: The paper is under consideration at Computer Vision and Image Understanding journal

Feedback control of transitional shear flows: Sensor selection for performance recovery

Authors: Huaijin Yao, Yiyang Sun, Maziar S. Hemati

Abstract: The choice and placement of sensors and actuators is an essential factor determining the performance that can be realized using feedback control. This determination is especially important, but difficult, in the context of controlling transitional flows. The highly non-normal nature of the linearized Navier-Stokes equations makes the flow sensitive to small perturbations, with potentially drastic… ▽ More The choice and placement of sensors and actuators is an essential factor determining the performance that can be realized using feedback control. This determination is especially important, but difficult, in the context of controlling transitional flows. The highly non-normal nature of the linearized Navier-Stokes equations makes the flow sensitive to small perturbations, with potentially drastic performance consequences on closed-loop flow control performance. Full-information controllers, such as the linear quadratic regulator (LQR), have demonstrated some success in reducing transient energy growth and suppressing transition; however, sensor-based output feedback controllers with comparable performance have been difficult to realize. In this study, we propose two methods for sensor selection that enable sensor-based output feedback controllers to recover full-information control performance: one based on a sparse controller synthesis approach, and one based on a balanced truncation procedure for model reduction. Both approaches are investigated within linear and nonlinear simulations of a sub-critical channel flow with blowing and suction actuation at the walls. We find that sensor configurations identified by both approaches allow sensor-based static output feedback LQR controllers to recover full-information LQR control performance, both in reducing transient energy growth and suppressing transition. Further, our results indicate that both the sensor selection methods and the resulting controllers exhibit robustness to Reynolds number variations. △ Less

Submitted 7 September, 2020; originally announced September 2020.

Personality Assessment from Text for Machine Commonsense Reasoning

Authors: Niloofar Hezarjaribi, Zhila Esna Ashari, James F. Frenzel, Hassan Ghasemzadeh, Saied Hemati

Abstract: This article presents PerSense, a framework to estimate human personality traits based on expressed texts and to use them for commonsense reasoning analysis. The personality assessment approaches include an aggregated Probability Density Functions (PDF), and Machine Learning (ML) models. Our goal is to demonstrate the feasibility of using machine learning algorithms on personality trait data to pr… ▽ More This article presents PerSense, a framework to estimate human personality traits based on expressed texts and to use them for commonsense reasoning analysis. The personality assessment approaches include an aggregated Probability Density Functions (PDF), and Machine Learning (ML) models. Our goal is to demonstrate the feasibility of using machine learning algorithms on personality trait data to predict humans' responses to open-ended commonsense questions. We assess the performance of the PerSense algorithms for personality assessment by conducting an experiment focused on Neuroticism, an important personality trait crucial in mental health analysis and suicide prevention by collecting data from a diverse population with different Neuroticism scores. Our analysis shows that the algorithms achieve comparable results to the ground truth data. Specifically, the PDF approach achieves 97% accuracy when the confidence factor, the logarithmic ratio of the first to the second guess probability, is greater than 3. Additionally, ML approach obtains its highest accuracy, 82.2%, with a multilayer Perceptron classifier. To assess the feasibility of commonsense reasoning analysis, we train ML algorithms to predict responses to commonsense questions. Our analysis of data collected with 300 participants demonstrate that PerSense predicts answers to commonsense questions with 82.3% accuracy using a Random Forest classifier. △ Less

Submitted 15 April, 2020; originally announced April 2020.

Nonlinear Stability Analysis of Transitional Flows using Quadratic Constraints

Authors: Aniketh Kalur, Peter Seiler, Maziar S. Hemati

Abstract: The dynamics of transitional flows are governed by an interplay between the non-normal linear dynamics and quadratic nonlinearity in the incompressible Navier-Stokes equations. In this work, we propose a framework for nonlinear stability analysis that exploits the fact that nonlinear flow interactions are constrained by the physics encoded in the nonlinearity. In particular, we show that nonlinear… ▽ More The dynamics of transitional flows are governed by an interplay between the non-normal linear dynamics and quadratic nonlinearity in the incompressible Navier-Stokes equations. In this work, we propose a framework for nonlinear stability analysis that exploits the fact that nonlinear flow interactions are constrained by the physics encoded in the nonlinearity. In particular, we show that nonlinear stability analysis problems can be posed as convex feasibility and optimization problems based on Lyapunov matrix inequalities, and a set of quadratic constraints that represent the nonlinear flow physics. The proposed framework can be used to conduct global stability, local stability, and transient energy growth analysis. The approach is demonstrated on the low-dimensional Waleffe-Kim-Hamilton model of transition and sustained turbulence. Our analysis correctly determines the critical Reynolds number for global instability. For local stability analysis, we show that the framework can estimate the size of the region of attraction as well as the amplitude of the largest permissible perturbation such that all trajectories converge back to the equilibrium point. Additionally, we show that the framework can predict bounds on the maximum transient energy growth. Finally, we show that careful analysis of the multipliers used to enforce the quadratic constraints can be used to extract dominant nonlinear flow interactions that drive the dynamics and associated instabilities. △ Less

Submitted 29 March, 2021; v1 submitted 11 April, 2020; originally announced April 2020.

Journal ref: Phys. Rev. Fluids 6, 044401 (2021)

Reconstructing the time evolution of wall-bounded turbulent flows from non-time resolved PIV measurements

Authors: C. Vamsi Krishna, Mengying Wang, Maziar S. Hemati, Mitul Luhar

Abstract: Particle Image Velocimetry (PIV) systems are often limited in their ability to fully resolve the spatiotemporal fluctuations inherent in turbulent flows due to hardware constraints. In this study, we develop models based on Rapid Distortion Theory (RDT) and Taylor's Hypothesis (TH) to reconstruct the time evolution of a turbulent flow field in the intermediate period between consecutive PIV snapsh… ▽ More Particle Image Velocimetry (PIV) systems are often limited in their ability to fully resolve the spatiotemporal fluctuations inherent in turbulent flows due to hardware constraints. In this study, we develop models based on Rapid Distortion Theory (RDT) and Taylor's Hypothesis (TH) to reconstruct the time evolution of a turbulent flow field in the intermediate period between consecutive PIV snapshots obtained using a non-time resolved system. The linear governing equations are evolved forwards and backwards in time using the PIV snapshots as initial conditions. The flow field in the intervening period is then reconstructed by taking a weighted sum of the forward and backward estimates. This spatiotemporal weighting function is designed to account for the advective nature of the RDT and TH equations. Reconstruction accuracy is evaluated as a function of spatial resolution and reconstruction time horizon using Direct Numerical Simulation data for turbulent channel flow from the Johns Hopkins Turbulence Database. This method reconstructs single-point turbulence statistics well and resolves velocity spectra at frequencies higher than the temporal Nyquist limit of the acquisition system. Reconstructions obtained using a characteristics-based evolution of the flow field under TH prove to be more accurate compared to reconstructions obtained from numerical integration of the discretized forms of RDT and TH. The effect of measurement noise on reconstruction error is also evaluated. △ Less

Submitted 14 April, 2020; v1 submitted 10 March, 2020; originally announced March 2020.

Comments: 21 pages, 11 captioned figures

A Hybrid Model for Lift Response to Dynamic Actuation On A Stalled Airfoil

Authors: Xuanhong An, David R. Williams, Maziar S. Hemati

Abstract: The current research focuses on modeling the lift response due to dynamic (time-varying) 'burst-type' actuation on a stalled airfoil. Dynamic `burst-type' actuation exhibits two different characteristic dynamic behaviors within the system, namely the high-frequency and low-frequency components. These characteristics introduce modeling challenges. In this paper, we propose a hybrid model composed o… ▽ More The current research focuses on modeling the lift response due to dynamic (time-varying) 'burst-type' actuation on a stalled airfoil. Dynamic `burst-type' actuation exhibits two different characteristic dynamic behaviors within the system, namely the high-frequency and low-frequency components. These characteristics introduce modeling challenges. In this paper, we propose a hybrid model composed of two individual sub-models, one for each of the two frequencies. The lift response due to high-frequency single burst actuation is captured using a convolution model. The low-frequency component due to nonlinear burst-burst interactions are captured using a Wiener model, consisting of linear time-invariant dynamics and a static output nonlinearity. The hybrid model is validated using data from wind tunnel experiments. △ Less

Submitted 18 December, 2019; originally announced December 2019.

Data-Driven Selection of Actuators for Optimal Control of Airfoil Separation

Authors: Debraj Bhattacharjee, Bjoern Klose, Gustaaf B. Jacobs, Maziar S. Hemati

Abstract: We present a systematic approach for determining the optimal actuator location for separation control from input-output response data, gathered from numerical simulations or physical experiments. The Eigensystem Realization Algorithm is used to extract state-space descriptions from the response data associated with a candidate set of actuator locations. These system realizations are then used to d… ▽ More We present a systematic approach for determining the optimal actuator location for separation control from input-output response data, gathered from numerical simulations or physical experiments. The Eigensystem Realization Algorithm is used to extract state-space descriptions from the response data associated with a candidate set of actuator locations. These system realizations are then used to determine the actuator location among the set that can drive the system output to an arbitrary value with minimal control effort. The solution of the corresponding minimum energy optimal control problem is evaluated by computing the generalized output controllability Gramian. We use the method to analyze high-fidelity numerical simulation data of the lift and separation-angle responses to a pulse of localized body-force actuation from six distinct locations on the upper surface of a NACA 65(1)-412 airfoil. We find that the optimal location for controlling lift is different from the optimal location for controlling separation angle. In order to explain the physical mechanisms underlying these differences, we conduct controllability analyses of the flowfield by leveraging the dynamic mode decomposition with control algorithm. These modal analyses of flowfield response data reveal that excitation of coherent structures in the wake benefit lift control; whereas, excitation of coherent structures in the shear layer benefit separation-angle control. △ Less

Submitted 10 October, 2019; originally announced October 2019.

Feedback control for transition suppression in direct numerical simulations of channel flow

Authors: Yiyang Sun, Maziar S. Hemati

Abstract: For channel flow at subcritical Reynolds numbers ($Re<5772$), a laminar-to-turbulent transition can emerge due to a large transient amplification in the kinetic energy of small perturbations, resulting in an increase in drag at the walls. The objectives of the present study are three-fold: (1) to study the nonlinear effects on transient energy growth, (2) to design a feedback control strategy to p… ▽ More For channel flow at subcritical Reynolds numbers ($Re<5772$), a laminar-to-turbulent transition can emerge due to a large transient amplification in the kinetic energy of small perturbations, resulting in an increase in drag at the walls. The objectives of the present study are three-fold: (1) to study the nonlinear effects on transient energy growth, (2) to design a feedback control strategy to prevent this subcritical transition, and (3) to examine the control mechanisms that enable transition suppression. We investigate transient energy growth of linear optimal disturbance in plane Poiseuille flow at a subcritical Reynolds number of $Re=3000$ using linear analysis and nonlinear simulation. We find that the amplification of the given initial perturbation is reduced when the nonlinear effect is substantial, with larger perturbations being less amplified in general. Moreover, we design linear quadratic optimal controllers to delay transition via wall-normal blowing and suction actuation at the channel walls. We demonstrate that these feedback controllers are capable of reducing transient energy growth in the linear setting. The performance of the same controllers is evaluated for nonlinear flows where a laminar-to-turbulent transition emerges without control. Nonlinear simulations reveal that the controllers can reduce transient energy growth and suppress transition. Further, we identify and characterize the underlying physical mechanisms that enable feedback control to suppress and delay laminar-to-turbulent transition. △ Less

Submitted 11 September, 2019; originally announced September 2019.

Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs

Authors: Mohammad Hadi Mehdizavareh, Sobhan Hemati, Hamid Soltanian-Zadeh

Abstract: Recently, brain-computer interface (BCI) systems developed based on steady-state visual evoked potential (SSVEP) have attracted much attention due to their high information transfer rate (ITR) and increasing number of targets. However, SSVEP-based methods can be improved in terms of their accuracy and target detection time. We propose a new method based on canonical correlation analysis (CCA) to i… ▽ More Recently, brain-computer interface (BCI) systems developed based on steady-state visual evoked potential (SSVEP) have attracted much attention due to their high information transfer rate (ITR) and increasing number of targets. However, SSVEP-based methods can be improved in terms of their accuracy and target detection time. We propose a new method based on canonical correlation analysis (CCA) to integrate subject-specific models and subject-independent information and enhance BCI performance. We propose to use training data of other subjects to optimize hyperparameters for CCA-based model of a specific subject. An ensemble version of the proposed method is also developed for a fair comparison with ensemble task-related component analysis (TRCA). The proposed method is compared with TRCA and extended CCA methods. A publicly available, 35-subject SSVEP benchmark dataset is used for comparison studies and performance is quantified by classification accuracy and ITR. The ITR of the proposed method is higher than those of TRCA and extended CCA. The proposed method outperforms extended CCA in all conditions and TRCA for time windows greater than 0.3 s. The proposed method also outperforms TRCA when there are limited training blocks and electrodes. This study illustrates that adding subject-independent information to subject-specific models can improve performance of SSVEP-based BCIs. △ Less

Submitted 15 January, 2020; v1 submitted 19 July, 2019; originally announced July 2019.

Comments: 22 pages, 8 figures, 1 table, 1 appendix, published in PLOS ONE journal. This is a draft version. The published version is available in the following link: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0226048

Journal ref: PLOS ONE 15(1): e0226048 (2020)

Control-oriented model reduction for minimizing transient energy growth in shear flows

Authors: Aniketh Kalur, Maziar S. Hemati

Abstract: A linear non-modal mechanism for transient amplification of perturbation energy is known to trigger sub-critical transition to turbulence in many shear flows. Feedback control strategies for minimizing this transient energy growth can be formulated as convex optimization problems based on linear matrix inequalities. Unfortunately, solving the requisite linear matrix inequality problem can be compu… ▽ More A linear non-modal mechanism for transient amplification of perturbation energy is known to trigger sub-critical transition to turbulence in many shear flows. Feedback control strategies for minimizing this transient energy growth can be formulated as convex optimization problems based on linear matrix inequalities. Unfortunately, solving the requisite linear matrix inequality problem can be computationally prohibitive within the context of high-dimensional fluid flows. In this work, we investigate the utility of control-oriented reduced-order models to facilitate the design of feedback flow control strategies that minimize the maximum transient energy growth. An output projection onto proper orthogonal decomposition modes is used to faithfully capture the system energy. Subsequently, a balanced truncation is performed to reduce the state dimension, while preserving the system's input-output properties. The model reduction and control approaches are studied within the context of a linearized channel flow with blowing and suction actuation at the walls. Controller synthesis for this linearized channel flow system becomes tractable through the use of the proposed control-oriented reduced-order models. Further, the resulting controllers are found to reduce the maximum transient energy growth compared with more conventional linear quadratic optimal control strategies. △ Less

Submitted 2 July, 2019; originally announced July 2019.

Modal Analysis of Fluid Flows: Applications and Outlook

Authors: Kunihiko Taira, Maziar S. Hemati, Steven L. Brunton, Yiyang Sun, Karthik Duraisamy, Shervin Bagheri, Scott T. M. Dawson, Chi-An Yeh

Abstract: We present applications of modal analysis techniques to study, model, and control canonical aerodynamic flows. To illustrate how modal analysis techniques can provide physical insights in a complementary manner, we selected four fundamental examples of cylinder wakes, wall-bounded flows, airfoil wakes, and cavity flows. We also offer brief discussions on the outlook for modal analysis techniques,… ▽ More We present applications of modal analysis techniques to study, model, and control canonical aerodynamic flows. To illustrate how modal analysis techniques can provide physical insights in a complementary manner, we selected four fundamental examples of cylinder wakes, wall-bounded flows, airfoil wakes, and cavity flows. We also offer brief discussions on the outlook for modal analysis techniques, in light of rapid developments in data science. △ Less

Submitted 26 July, 2019; v1 submitted 13 March, 2019; originally announced March 2019.

Comments: 37 pages, 19 figures, 2 tables

Detecting exotic wakes with hydrodynamic sensors

Authors: Mengying Wang, Maziar S. Hemati

Abstract: Wake sensing for bioinspired robotic swimmers has been the focus of much investigation owing to its relevance to locomotion control, especially in the context of schooling and target following. Many successful wake sensing strategies have been devised based on models of von Karman-type wakes; however, such wake sensing technologies are invalid in the context of exotic wake types that commonly aris… ▽ More Wake sensing for bioinspired robotic swimmers has been the focus of much investigation owing to its relevance to locomotion control, especially in the context of schooling and target following. Many successful wake sensing strategies have been devised based on models of von Karman-type wakes; however, such wake sensing technologies are invalid in the context of exotic wake types that commonly arise in swimming locomotion. Indeed, exotic wakes can exhibit markedly different dynamics, and so must be modeled and sensed accordingly. Here, we propose a general wake detection protocol for distinguishing between wake types from measured hydrodynamic signals alone. An ideal-flow model is formulated and used to demonstrate the general wake detection framework in a proof-of-concept study. We show that wakes with different underlying dynamics impart distinct signatures on a fish-like body, which can be observed in time-series measurements at a single location on the body surface. These hydrodynamic wake signatures are used to construct a wake classification library that is then used to classify unknown wakes from hydrodynamic signal measurements. The wake detection protocol is found to have an accuracy rate of over 95% in the majority of performance studies conducted here. Thus, exotic wake detection is shown to be viable, which suggests that such technologies have the potential to become key enablers of multi-model sensing and locomotion control strategies in the future. △ Less

Submitted 28 November, 2017; originally announced November 2017.

Performance limitations of observer-based feedback for transient energy growth suppression

Authors: Maziar S. Hemati, Huaijin Yao

Abstract: Transient energy growth suppression is a common control objective for feedback flow control aimed at delaying transition to turbulence. A prevailing control approach in this context is observer-based feedback, in which a full-state feedback controller is applied to state estimates from an observer. The present study identifies a fundamental performance limitation of observer-based feedback control… ▽ More Transient energy growth suppression is a common control objective for feedback flow control aimed at delaying transition to turbulence. A prevailing control approach in this context is observer-based feedback, in which a full-state feedback controller is applied to state estimates from an observer. The present study identifies a fundamental performance limitation of observer-based feedback control: whenever the uncontrolled system exhibits transient energy growth in response to optimal disturbances, control by observer-based feedback will necessarily lead to transient energy growth in response to optimal disturbances for the closed-loop system as well. Indeed, this result establishes that observer-based feedback can be a poor candidate for controller synthesis in the context of transient energy growth suppression and transition delay: the performance objective of transient energy growth suppression can never be achieved by means of observer-based feedback. Further, an illustrative example is used to show that alternative forms of output feedback are not necessarily subject to these same performance limitations, and should also be considered in the context of transient energy growth suppression and transition control. △ Less

Submitted 17 April, 2018; v1 submitted 14 November, 2017; originally announced November 2017.

Comments: 7 pages; 1 figure

Journal ref: AIAA Journal, Vol. 56, No. 6 (2018), pp. 2119-2123

Symbolic Stochastic Chase Decoding of Reed-Solomon and BCH Codes

Authors: Hossein Mani, Saied Hemati

Abstract: This paper proposes the Symbolic-Stochastic Chase Decoding Algorithm (S-SCA) for the Reed-Solomon (RS) and BCH codes. By efficient usage of void space between constellation points for $q$-ary modulations and using soft information at the input of the decoder, the S-SCA is capable of outperforming conventional Symbolic-Chase algorithm (S-CA) with less computational cost. Since the S-SCA starts with… ▽ More This paper proposes the Symbolic-Stochastic Chase Decoding Algorithm (S-SCA) for the Reed-Solomon (RS) and BCH codes. By efficient usage of void space between constellation points for $q$-ary modulations and using soft information at the input of the decoder, the S-SCA is capable of outperforming conventional Symbolic-Chase algorithm (S-CA) with less computational cost. Since the S-SCA starts with the randomized generation of likely test-vectors, it reduces the complexity to polynomial order and also it does not need to find the least reliable symbols to generate test-vectors. Our simulation results show that by increasing the number of test-vectors, the performance of the algorithm can approach the ML bound. The S-SCA($1K$) provides near $2$ dB gain in comparison with S-CA($1K$) for $(31, 25)$ RS code using $32$-QAM. Furthermore, the algorithm provides near $3$ dB further gain with $1K$ iteration compared with S-CA($65K$) when $(255, 239)$ RS code is used in an AWGN channel. For the Rayleigh fading channel and the same code, the algorithm provides more that $5$ dB gain. Also for $(63, 57)$ BCH codes and $8$-PSK modulation the proposed algorithm provides $3$dB gain with less complexity. This decoder is Soft-Input Soft-Output (SISO) decoder and is highly attractive in low power applications. Finally, the Symbolic-Search Bitwise-Transmission Stochastic Chase Algorithm (SSBT-SCA) was introduced for RS codes over BPSK transmission that is capable of generating symbolic test-vectors that reduce complexity and mitigate burst errors. △ Less

Submitted 14 July, 2017; originally announced July 2017.

Characterizing and correcting for the effect of sensor noise in the dynamic mode decomposition

Authors: Scott T. M. Dawson, Maziar S. Hemati, Matthew O. Williams, Clarence W. Rowley

Abstract: Dynamic mode decomposition (DMD) provides a practical means of extracting insightful dynamical information from fluids datasets. Like any data processing technique, DMD's usefulness is limited by its ability to extract real and accurate dynamical features from noise-corrupted data. Here we show analytically that DMD is biased to sensor noise, and quantify how this bias depends on the size and nois… ▽ More Dynamic mode decomposition (DMD) provides a practical means of extracting insightful dynamical information from fluids datasets. Like any data processing technique, DMD's usefulness is limited by its ability to extract real and accurate dynamical features from noise-corrupted data. Here we show analytically that DMD is biased to sensor noise, and quantify how this bias depends on the size and noise level of the data. We present three modifications to DMD that can be used to remove this bias: (i) a direct correction of the identified bias using known noise properties, (ii) combining the results of performing DMD forwards and backwards in time, and (iii) a total least-squares-inspired algorithm. We discuss the relative merits of each algorithm, and demonstrate the performance of these modifications on a range of synthetic, numerical, and experimental datasets. We further compare our modified DMD algorithms with other variants proposed in recent literature. △ Less

Submitted 18 January, 2016; v1 submitted 8 July, 2015; originally announced July 2015.

Mitigating Hardware Cyber-Security Risks in Error Correcting Decoders

Authors: Saied Hemati

Abstract: This paper investigates hardware cyber-security risks associated with channel decoders, which are commonly acquired as a black box in semiconductor industry. It is shown that channel decoders are potentially attractive targets for hardware cyber-security attacks and can be easily embedded with malicious blocks. Several attack scenarios are considered in this work and suitable methods for mitigatin… ▽ More This paper investigates hardware cyber-security risks associated with channel decoders, which are commonly acquired as a black box in semiconductor industry. It is shown that channel decoders are potentially attractive targets for hardware cyber-security attacks and can be easily embedded with malicious blocks. Several attack scenarios are considered in this work and suitable methods for mitigating the risks are proposed. These methods are based on randomizing the inputs of the channel decoder to obstruct the communications between attackers and the malicious blocks, ideally without changing the decoding performance. △ Less

Submitted 5 May, 2015; originally announced May 2015.

De-Biasing the Dynamic Mode Decomposition for Applied Koopman Spectral Analysis

Authors: Maziar S. Hemati, Clarence W. Rowley, Eric A. Deem, Louis N. Cattafesta

Abstract: The Dynamic Mode Decomposition (DMD)---a popular method for performing data-driven Koopman spectral analysis---has gained increased adoption as a technique for extracting dynamically meaningful spatio-temporal descriptions of fluid flows from snapshot measurements. Often times, DMD descriptions can be used for predictive purposes as well, which enables informed decision-making based on DMD model-f… ▽ More The Dynamic Mode Decomposition (DMD)---a popular method for performing data-driven Koopman spectral analysis---has gained increased adoption as a technique for extracting dynamically meaningful spatio-temporal descriptions of fluid flows from snapshot measurements. Often times, DMD descriptions can be used for predictive purposes as well, which enables informed decision-making based on DMD model-forecasts. Despite its widespread use and utility, DMD regularly fails to yield accurate dynamical descriptions when the measured snapshot data are imprecise due to, e.g., sensor noise. Here, we express DMD as a two-stage algorithm in order to isolate a source of systematic error. We show that DMD's first stage, a subspace projection step, systematically introduces bias errors by processing snapshots asymmetrically. To remove this systematic error, we propose utilizing an augmented snapshot matrix in a subspace projection step, as in problems of total least-squares, in order to account for the error present in all snapshots. The resulting unbiased and noise-aware total DMD (TDMD) formulation reduces to standard DMD in the absence of snapshot errors, while the two-stage perspective generalizes the de-biasing framework to other related methods as well. TDMD's performance is demonstrated in numerical and experimental fluids examples. △ Less

Submitted 26 October, 2015; v1 submitted 12 February, 2015; originally announced February 2015.

Dynamic Mode Decomposition for Large and Streaming Datasets

Authors: Maziar S. Hemati, Matthew O. Williams, Clarence W. Rowley

Abstract: We formulate a low-storage method for performing dynamic mode decomposition that can be updated inexpensively as new data become available; this formulation allows dynamical information to be extracted from large datasets and data streams. We present two algorithms: the first is mathematically equivalent to a standard "batch-processed" formulation; the second introduces a compression step that mai… ▽ More We formulate a low-storage method for performing dynamic mode decomposition that can be updated inexpensively as new data become available; this formulation allows dynamical information to be extracted from large datasets and data streams. We present two algorithms: the first is mathematically equivalent to a standard "batch-processed" formulation; the second introduces a compression step that maintains computational efficiency, while enhancing the ability to isolate pertinent dynamical information from noisy measurements. Both algorithms reliably capture dominant fluid dynamic behaviors, as demonstrated on cylinder wake data collected from both direct numerical simulations and particle image velocimetry experiments △ Less

Submitted 27 June, 2014; originally announced June 2014.

Relaxed Half-Stochastic Belief Propagation

Authors: François Leduc-Primeau, Saied Hemati, Shie Mannor, Warren J. Gross

Abstract: Low-density parity-check codes are attractive for high throughput applications because of their low decoding complexity per bit, but also because all the codeword bits can be decoded in parallel. However, achieving this in a circuit implementation is complicated by the number of wires required to exchange messages between processing nodes. Decoding algorithms that exchange binary messages are inte… ▽ More Low-density parity-check codes are attractive for high throughput applications because of their low decoding complexity per bit, but also because all the codeword bits can be decoded in parallel. However, achieving this in a circuit implementation is complicated by the number of wires required to exchange messages between processing nodes. Decoding algorithms that exchange binary messages are interesting for fully-parallel implementations because they can reduce the number and the length of the wires, and increase logic density. This paper introduces the Relaxed Half-Stochastic (RHS) decoding algorithm, a binary message belief propagation (BP) algorithm that achieves a coding gain comparable to the best known BP algorithms that use real-valued messages. We derive the RHS algorithm by starting from the well-known Sum-Product algorithm, and then derive a low-complexity version suitable for circuit implementation. We present extensive simulation results on two standardized codes having different rates and constructions, including low bit error rate results. These simulations show that RHS can be an advantageous replacement for the existing state-of-the-art decoding algorithms when targeting fully-parallel implementations. △ Less

Submitted 11 May, 2012; originally announced May 2012.