Showing 1–50 of 177 results for author: Huang, L

Applications and Advances of Artificial Intelligence in Music Generation:A Review

Authors: Yanxu Chen, Linshu Huang, Tian Gou

Abstract: In recent years, artificial intelligence (AI) has made significant progress in the field of music generation, driving innovation in music creation and applications. This paper provides a systematic review of the latest research advancements in AI music generation, covering key technologies, models, datasets, evaluation methods, and their practical applications across various fields. The main contr… ▽ More In recent years, artificial intelligence (AI) has made significant progress in the field of music generation, driving innovation in music creation and applications. This paper provides a systematic review of the latest research advancements in AI music generation, covering key technologies, models, datasets, evaluation methods, and their practical applications across various fields. The main contributions of this review include: (1) presenting a comprehensive summary framework that systematically categorizes and compares different technological approaches, including symbolic generation, audio generation, and hybrid models, helping readers better understand the full spectrum of technologies in the field; (2) offering an extensive survey of current literature, covering emerging topics such as multimodal datasets and emotion expression evaluation, providing a broad reference for related research; (3) conducting a detailed analysis of the practical impact of AI music generation in various application domains, particularly in real-time interaction and interdisciplinary applications, offering new perspectives and insights; (4) summarizing the existing challenges and limitations of music quality evaluation methods and proposing potential future research directions, aiming to promote the standardization and broader adoption of evaluation techniques. Through these innovative summaries and analyses, this paper serves as a comprehensive reference tool for researchers and practitioners in AI music generation, while also outlining future directions for the field. △ Less

Submitted 3 September, 2024; originally announced September 2024.

Adversarial Network Optimization under Bandit Feedback: Maximizing Utility in Non-Stationary Multi-Hop Networks

Authors: Yan Dai, Longbo Huang

Abstract: Stochastic Network Optimization (SNO) concerns scheduling in stochastic queueing systems. It has been widely studied in network theory. Classical SNO algorithms require network conditions to be stationary with time, which fails to capture the non-stationary components in many real-world scenarios. Many existing algorithms also assume knowledge of network conditions before decision, which rules out… ▽ More Stochastic Network Optimization (SNO) concerns scheduling in stochastic queueing systems. It has been widely studied in network theory. Classical SNO algorithms require network conditions to be stationary with time, which fails to capture the non-stationary components in many real-world scenarios. Many existing algorithms also assume knowledge of network conditions before decision, which rules out applications where unpredictability presents. Motivated by these issues, we consider Adversarial Network Optimization (ANO) under bandit feedback. Specifically, we consider the task of *i)* maximizing some unknown and time-varying utility function associated to scheduler's actions, where *ii)* the underlying network is a non-stationary multi-hop one whose conditions change arbitrarily with time, and *iii)* only bandit feedback (effect of actually deployed actions) is revealed after decisions. Our proposed `UMO2` algorithm ensures network stability and also matches the utility maximization performance of any "mildly varying" reference policy up to a polynomially decaying gap. To our knowledge, no previous ANO algorithm handled multi-hop networks or achieved utility guarantees under bandit feedback, whereas ours can do both. Technically, our method builds upon a novel integration of online learning into Lyapunov analyses: To handle complex inter-dependencies among queues in multi-hop networks, we propose meticulous techniques to balance online learning and Lyapunov arguments. To tackle the learning obstacles due to potentially unbounded queue sizes, we design a new online linear optimization algorithm that automatically adapts to loss magnitudes. To maximize utility, we propose a bandit convex optimization algorithm with novel queue-dependent learning rate scheduling that suites drastically varying queue lengths. Our new insights in online learning can be of independent interest. △ Less

Submitted 28 August, 2024; originally announced August 2024.

Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper

Authors: Tianyi Xu, Kaixun Huang, Pengcheng Guo, Yu Zhou, Longtao Huang, Hui Xue, Lei Xie

Abstract: Pre-trained multilingual speech foundation models, like Whisper, have shown impressive performance across different languages. However, adapting these models to new or specific languages is computationally extensive and faces catastrophic forgetting problems. Addressing these issues, our study investigates strategies to enhance the model on new languages in the absence of original training data, w… ▽ More Pre-trained multilingual speech foundation models, like Whisper, have shown impressive performance across different languages. However, adapting these models to new or specific languages is computationally extensive and faces catastrophic forgetting problems. Addressing these issues, our study investigates strategies to enhance the model on new languages in the absence of original training data, while also preserving the established performance on the original languages. Specifically, we first compare various LoRA-based methods to find out their vulnerability to forgetting. To mitigate this issue, we propose to leverage the LoRA parameters from the original model for approximate orthogonal gradient descent on the new samples. Additionally, we also introduce a learnable rank coefficient to allocate trainable parameters for more efficient training. Our experiments with a Chinese Whisper model (for Uyghur and Tibetan) yield better results with a more compact parameter set. △ Less

Submitted 20 August, 2024; originally announced August 2024.

ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosis

Authors: Yubao Zhao, Tian Zhang, Xu Wang, Puyu Han, Tong Chen, Linlin Huang, Youzhu Jin, Jiaju Kang

Abstract: The success of Multimodal Large Language Models (MLLMs) in the medical auxiliary field shows great potential, allowing patients to engage in conversations using physiological signal data. However, general MLLMs perform poorly in cardiac disease diagnosis, particularly in the integration of ECG data analysis and long-text medical report generation, mainly due to the complexity of ECG data analysis… ▽ More The success of Multimodal Large Language Models (MLLMs) in the medical auxiliary field shows great potential, allowing patients to engage in conversations using physiological signal data. However, general MLLMs perform poorly in cardiac disease diagnosis, particularly in the integration of ECG data analysis and long-text medical report generation, mainly due to the complexity of ECG data analysis and the gap between text and ECG signal modalities. Additionally, models often exhibit severe stability deficiencies in long-text generation due to the lack of precise knowledge strongly related to user queries. To address these issues, we propose ECG-Chat, the first multitask MLLMs focused on ECG medical report generation, providing multimodal conversational capabilities based on cardiology knowledge. We propose a contrastive learning approach that integrates ECG waveform data with text reports, aligning ECG features with reports in a fine-grained manner. This method also results in an ECG encoder that excels in zero-shot report retrieval tasks. Additionally, expanding existing datasets, we constructed a 19k ECG diagnosis dataset and a 25k multi-turn dialogue dataset for training and fine-tuning ECG-Chat, which provides professional diagnostic and conversational capabilities. Furthermore, ECG-Chat can generate comprehensive ECG analysis reports through an automated LaTeX generation pipeline. We established a benchmark for the ECG report generation task and tested our model on multiple baselines. ECG-Chat achieved the best performance in classification, retrieval, multimodal dialogue, and medical report generation tasks. Our report template design has also been widely recognized by medical practitioners. △ Less

Submitted 16 August, 2024; originally announced August 2024.

Semantics Guided Disentangled GAN for Chest X-ray Image Rib Segmentation

Authors: Lili Huang, Dexin Ma, Xiaowei Zhao, Chenglong Li, Haifeng Zhao, Jin Tang, Chuanfu Li

Abstract: The label annotations for chest X-ray image rib segmentation are time consuming and laborious, and the labeling quality heavily relies on medical knowledge of annotators. To reduce the dependency on annotated data, existing works often utilize generative adversarial network (GAN) to generate training data. However, GAN-based methods overlook the nuanced information specific to individual organs, w… ▽ More The label annotations for chest X-ray image rib segmentation are time consuming and laborious, and the labeling quality heavily relies on medical knowledge of annotators. To reduce the dependency on annotated data, existing works often utilize generative adversarial network (GAN) to generate training data. However, GAN-based methods overlook the nuanced information specific to individual organs, which degrades the generation quality of chest X-ray image. Hence, we propose a novel Semantics guided Disentangled GAN (SD-GAN), which can generate the high-quality training data by fully utilizing the semantic information of different organs, for chest X-ray image rib segmentation. In particular, we use three ResNet50 branches to disentangle features of different organs, then use a decoder to combine features and generate corresponding images. To ensure that the generated images correspond to the input organ labels in semantics tags, we employ a semantics guidance module to perform semantic guidance on the generated images. To evaluate the efficacy of SD-GAN in generating high-quality samples, we introduce modified TransUNet(MTUNet), a specialized segmentation network designed for multi-scale contextual information extracting and multi-branch decoding, effectively tackling the challenge of organ overlap. We also propose a new chest X-ray image dataset (CXRS). It includes 1250 samples from various medical institutions. Lungs, clavicles, and 24 ribs are simultaneously annotated on each chest X-ray image. The visualization and quantitative results demonstrate the efficacy of SD-GAN in generating high-quality chest X-ray image-mask pairs. Using generated data, our trained MTUNet overcomes the limitations of the data scale and outperforms other segmentation networks. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Multi-scale Conditional Generative Modeling for Microscopic Image Restoration

Authors: Luzhe Huang, Xiongye Xiao, Shixuan Li, Jiawen Sun, Yi Huang, Aydogan Ozcan, Paul Bogdan

Abstract: The advance of diffusion-based generative models in recent years has revolutionized state-of-the-art (SOTA) techniques in a wide variety of image analysis and synthesis tasks, whereas their adaptation on image restoration, particularly within computational microscopy remains theoretically and empirically underexplored. In this research, we introduce a multi-scale generative model that enhances con… ▽ More The advance of diffusion-based generative models in recent years has revolutionized state-of-the-art (SOTA) techniques in a wide variety of image analysis and synthesis tasks, whereas their adaptation on image restoration, particularly within computational microscopy remains theoretically and empirically underexplored. In this research, we introduce a multi-scale generative model that enhances conditional image restoration through a novel exploitation of the Brownian Bridge process within wavelet domain. By initiating the Brownian Bridge diffusion process specifically at the lowest-frequency subband and applying generative adversarial networks at subsequent multi-scale high-frequency subbands in the wavelet domain, our method provides significant acceleration during training and sampling while sustaining a high image generation quality and diversity on par with SOTA diffusion models. Experimental results on various computational microscopy and imaging tasks confirm our method's robust performance and its considerable reduction in its sampling steps and time. This pioneering technique offers an efficient image restoration framework that harmonizes efficiency with quality, signifying a major stride in incorporating cutting-edge generative models into computational microscopy workflows. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

Authors: Ye Bai, Jingping Chen, Jitong Chen, Wei Chen, Zhuo Chen, Chuang Ding, Linhao Dong, Qianqian Dong, Yujiao Du, Kepan Gao, Lu Gao, Yi Guo, Minglun Han, Ting Han, Wenchao Hu, Xinying Hu, Yuxiang Hu, Deyu Hua, Lu Huang, Mingkun Huang, Youjia Huang, Jishuo Jin, Fanliu Kong, Zongwei Lan, Tianyu Li , et al. (30 additional authors not shown)

Abstract: Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this wor… ▽ More Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this work, we introduce Seed-ASR, a large language model (LLM) based speech recognition model. Seed-ASR is developed based on the framework of audio conditioned LLM (AcLLM), leveraging the capabilities of LLMs by inputting continuous speech representations together with contextual information into the LLM. Through stage-wise large-scale training and the elicitation of context-aware capabilities in LLM, Seed-ASR demonstrates significant improvement over end-to-end models on comprehensive evaluation sets, including multiple domains, accents/dialects and languages. Additionally, Seed-ASR can be further deployed to support specific needs in various scenarios without requiring extra language models. Compared to recently released large ASR models, Seed-ASR achieves 10%-40% reduction in word (or character, for Chinese) error rates on Chinese and English public test sets, further demonstrating its powerful performance. △ Less

Submitted 10 July, 2024; v1 submitted 5 July, 2024; originally announced July 2024.

Segmenting Medical Images: From UNet to Res-UNet and nnUNet

Authors: Lina Huang, Alina Miron, Kate Hone, Yongmin Li

Abstract: This study provides a comparative analysis of deep learning models including UNet, Res-UNet, Attention Res-UNet, and nnUNet, and evaluates their performance in brain tumour, polyp, and multi-class heart segmentation tasks. The analysis focuses on precision, accuracy, recall, Dice Similarity Coefficient (DSC), and Intersection over Union (IoU) to assess their clinical applicability. In brain tumour… ▽ More This study provides a comparative analysis of deep learning models including UNet, Res-UNet, Attention Res-UNet, and nnUNet, and evaluates their performance in brain tumour, polyp, and multi-class heart segmentation tasks. The analysis focuses on precision, accuracy, recall, Dice Similarity Coefficient (DSC), and Intersection over Union (IoU) to assess their clinical applicability. In brain tumour segmentation, Res-UNet and nnUNet significantly outperformed UNet, with Res-UNet leading in DSC and IoU scores, indicating superior accuracy in tumour delineation. Meanwhile, nnUNet excelled in recall and accuracy, which are crucial for reliable tumour detection in clinical diagnosis and planning. In polyp detection, nnUNet was the most effective, achieving the highest metrics across all categories and proving itself as a reliable diagnostic tool in endoscopy. In the complex task of heart segmentation, Res-UNet and Attention Res-UNet were outstanding in delineating the left ventricle, with Res-UNet also leading in right ventricle segmentation. nnUNet was unmatched in myocardium segmentation, achieving top scores in precision, recall, DSC, and IoU. The conclusion notes that although Res-UNet occasionally outperforms nnUNet in specific metrics, the differences are quite small. Moreover, nnUNet consistently shows superior overall performance across the experiments. Particularly noted for its high recall and accuracy, which are crucial in clinical settings to minimize misdiagnosis and ensure timely treatment, nnUNet's robust performance in crucial metrics across all tested categories establishes it as the most effective model for these varied and complex segmentation tasks. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: 7 pages, 3 figures

Intelligent Reflecting Surface-Assisted NLOS Sensing With OFDM Signals

Authors: Jilin Wang, Jun Fang, Hongbin Li, Lei Huang

Abstract: This work addresses the problem of intelligent reflecting surface (IRS) assisted target sensing in a non-line-of-sight (NLOS) scenario, where an IRS is employed to facilitate the radar/access point (AP) to sense the targets when the line-of-sight (LOS) path between the AP and the target is blocked by obstacles. To sense the targets, the AP transmits a train of uniformly-spaced orthogonal frequency… ▽ More This work addresses the problem of intelligent reflecting surface (IRS) assisted target sensing in a non-line-of-sight (NLOS) scenario, where an IRS is employed to facilitate the radar/access point (AP) to sense the targets when the line-of-sight (LOS) path between the AP and the target is blocked by obstacles. To sense the targets, the AP transmits a train of uniformly-spaced orthogonal frequency division multiplexing (OFDM) pulses, and then perceives the targets based on the echoes from the AP-IRS-targets-IRS-AP channel. To resolve an inherent scaling ambiguity associated with IRS-assisted NLOS sensing, we propose a two-phase sensing scheme by exploiting the diversity in the illumination pattern of the IRS across two different phases. Specifically, the received echo signals from the two phases are formulated as third-order tensors. Then a canonical polyadic (CP) decomposition-based method is developed to estimate each target's parameters including the direction of arrival (DOA), Doppler shift and time delay. Our analysis reveals that the proposed method achieves reliable NLOS sensing using a modest quantity of pulse/subcarrier resources. Simulation results are provided to show the effectiveness of the proposed method under the challenging scenario where the degrees-of-freedom provided by the AP-IRS channel are not enough for resolving the scaling ambiguity. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Deep-learning-based groupwise registration for motion correction of cardiac $T_1$ mapping

Authors: Yi Zhang, Yidong Zhao, Lu Huang, Liming Xia, Qian Tao

Abstract: Quantitative $T_1$ mapping by MRI is an increasingly important tool for clinical assessment of cardiovascular diseases. The cardiac $T_1$ map is derived by fitting a known signal model to a series of baseline images, while the quality of this map can be deteriorated by involuntary respiratory and cardiac motion. To correct motion, a template image is often needed to register all baseline images, b… ▽ More Quantitative $T_1$ mapping by MRI is an increasingly important tool for clinical assessment of cardiovascular diseases. The cardiac $T_1$ map is derived by fitting a known signal model to a series of baseline images, while the quality of this map can be deteriorated by involuntary respiratory and cardiac motion. To correct motion, a template image is often needed to register all baseline images, but the choice of template is nontrivial, leading to inconsistent performance sensitive to image contrast. In this work, we propose a novel deep-learning-based groupwise registration framework, which omits the need for a template, and registers all baseline images simultaneously. We design two groupwise losses for this registration framework: the first is a linear principal component analysis (PCA) loss that enforces alignment of baseline images irrespective of the intensity variation, and the second is an auxiliary relaxometry loss that enforces adherence of intensity profile to the signal model. We extensively evaluated our method, termed ``PCA-Relax'', and other baseline methods on an in-house cardiac MRI dataset including both pre- and post-contrast $T_1$ sequences. All methods were evaluated under three distinct training-and-evaluation strategies, namely, standard, one-shot, and test-time-adaptation. The proposed PCA-Relax showed further improved performance of registration and mapping over well-established baselines. The proposed groupwise framework is generic and can be adapted to applications involving multiple images. △ Less

Submitted 21 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Comments: MICCAI 2024. Contents may slightly differ from the camera-ready version

CLDTA: Contrastive Learning based on Diagonal Transformer Autoencoder for Cross-Dataset EEG Emotion Recognition

Authors: Yuan Liao, Yuhong Zhang, Shenghuan Wang, Xiruo Zhang, Yiling Zhang, Wei Chen, Yuzhe Gu, Liya Huang

Abstract: Recent advances in non-invasive EEG technology have broadened its application in emotion recognition, yielding a multitude of related datasets. Yet, deep learning models struggle to generalize across these datasets due to variations in acquisition equipment and emotional stimulus materials. To address the pressing need for a universal model that fluidly accommodates diverse EEG dataset formats and… ▽ More Recent advances in non-invasive EEG technology have broadened its application in emotion recognition, yielding a multitude of related datasets. Yet, deep learning models struggle to generalize across these datasets due to variations in acquisition equipment and emotional stimulus materials. To address the pressing need for a universal model that fluidly accommodates diverse EEG dataset formats and bridges the gap between laboratory and real-world data, we introduce a novel deep learning framework: the Contrastive Learning based Diagonal Transformer Autoencoder (CLDTA), tailored for EEG-based emotion recognition. The CLDTA employs a diagonal masking strategy within its encoder to extracts full-channel EEG data's brain network knowledge, facilitating transferability to the datasets with fewer channels. And an information separation mechanism improves model interpretability by enabling straightforward visualization of brain networks. The CLDTA framework employs contrastive learning to distill subject-independent emotional representations and uses a calibration prediction process to enable rapid adaptation of the model to new subjects with minimal samples, achieving accurate emotion recognition. Our analysis across the SEED, SEED-IV, SEED-V, and DEAP datasets highlights CLDTA's consistent performance and proficiency in detecting both task-specific and general features of EEG signals related to emotions, underscoring its potential to revolutionize emotion recognition research. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Accelerating Ill-conditioned Hankel Matrix Recovery via Structured Newton-like Descent

Authors: HanQin Cai, Longxiu Huang, Xiliang Lu, Juntao You

Abstract: This paper studies the robust Hankel recovery problem, which simultaneously removes the sparse outliers and fulfills missing entries from the partial observation. We propose a novel non-convex algorithm, coined Hankel Structured Newton-Like Descent (HSNLD), to tackle the robust Hankel recovery problem. HSNLD is highly efficient with linear convergence, and its convergence rate is independent of th… ▽ More This paper studies the robust Hankel recovery problem, which simultaneously removes the sparse outliers and fulfills missing entries from the partial observation. We propose a novel non-convex algorithm, coined Hankel Structured Newton-Like Descent (HSNLD), to tackle the robust Hankel recovery problem. HSNLD is highly efficient with linear convergence, and its convergence rate is independent of the condition number of the underlying Hankel matrix. The recovery guarantee has been established under some mild conditions. Numerical experiments on both synthetic and real datasets show the superior performance of HSNLD against state-of-the-art algorithms. △ Less

Submitted 11 June, 2024; originally announced June 2024.

MSC Class: 15A29; 15A83; 47B35; 90C17; 90C26; 90C53

Compressed Meta-Optical Encoder for Image Classification

Authors: Anna Wirth-Singh, Jinlin Xiang, Minho Choi, Johannes E. Fröch, Luocheng Huang, Shane Colburn, Eli Shlizerman, Arka Majumdar

Abstract: Optical and hybrid convolutional neural networks (CNNs) recently have become of increasing interest to achieve low-latency, low-power image classification and computer vision tasks. However, implementing optical nonlinearity is challenging, and omitting the nonlinear layers in a standard CNN comes at a significant reduction in accuracy. In this work, we use knowledge distillation to compress modif… ▽ More Optical and hybrid convolutional neural networks (CNNs) recently have become of increasing interest to achieve low-latency, low-power image classification and computer vision tasks. However, implementing optical nonlinearity is challenging, and omitting the nonlinear layers in a standard CNN comes at a significant reduction in accuracy. In this work, we use knowledge distillation to compress modified AlexNet to a single linear convolutional layer and an electronic backend (two fully connected layers). We obtain comparable performance to a purely electronic CNN with five convolutional layers and three fully connected layers. We implement the convolution optically via engineering the point spread function of an inverse-designed meta-optic. Using this hybrid approach, we estimate a reduction in multiply-accumulate operations from 17M in a conventional electronic modified AlexNet to only 86K in the hybrid compressed network enabled by the optical frontend. This constitutes over two orders of magnitude reduction in latency and power consumption. Furthermore, we experimentally demonstrate that the classification accuracy of the system exceeds 93% on the MNIST dataset. △ Less

Submitted 14 June, 2024; v1 submitted 22 April, 2024; originally announced June 2024.

Soft Multipath Information-Based UWB Tracking in Cluttered Scenarios: Preliminaries and Validations

Authors: Chenglong Li, Zukun Lu, Long Huang, Shaojie Ni, Guangfu Sun, Emmeric Tanghe, Wout Joseph

Abstract: In this paper, we investigate ultra-wideband (UWB) localization and tracking in cluttered environments. Instead of mitigating the multipath, we exploit the specular reflections to enhance the localizability and improve the positioning accuracy. With the assistance of the multipath, it is also possible to achieve localization purposes using fewer anchors or when the line-of-sight propagations are b… ▽ More In this paper, we investigate ultra-wideband (UWB) localization and tracking in cluttered environments. Instead of mitigating the multipath, we exploit the specular reflections to enhance the localizability and improve the positioning accuracy. With the assistance of the multipath, it is also possible to achieve localization purposes using fewer anchors or when the line-of-sight propagations are blocked. Rather than using single-value distance, angle, or Doppler estimates for the localization, we model the likelihoods of both the line-of-sight and specular multipath components, namely soft multipath information, and propose the multipath-assisted probabilistic UWB tracking algorithm. Experimental results in a cluttered industrial scenario show that the proposed algorithm achieves 46.4 cm and 33.1 cm 90th percentile errors in the cases of 3 and 4 anchors, respectively, which outperforms conventional methods with more than 61.8% improvement given fewer anchors and strong multipath effect. △ Less

Submitted 28 May, 2024; v1 submitted 27 May, 2024; originally announced May 2024.

Autonomous Quality and Hallucination Assessment for Virtual Tissue Staining and Digital Pathology

Authors: Luzhe Huang, Yuzhu Li, Nir Pillar, Tal Keidar Haran, William Dean Wallace, Aydogan Ozcan

Abstract: Histopathological staining of human tissue is essential in the diagnosis of various diseases. The recent advances in virtual tissue staining technologies using AI alleviate some of the costly and tedious steps involved in the traditional histochemical staining process, permitting multiplexed rapid staining of label-free tissue without using staining reagents, while also preserving tissue. However,… ▽ More Histopathological staining of human tissue is essential in the diagnosis of various diseases. The recent advances in virtual tissue staining technologies using AI alleviate some of the costly and tedious steps involved in the traditional histochemical staining process, permitting multiplexed rapid staining of label-free tissue without using staining reagents, while also preserving tissue. However, potential hallucinations and artifacts in these virtually stained tissue images pose concerns, especially for the clinical utility of these approaches. Quality assessment of histology images is generally performed by human experts, which can be subjective and depends on the training level of the expert. Here, we present an autonomous quality and hallucination assessment method (termed AQuA), mainly designed for virtual tissue staining, while also being applicable to histochemical staining. AQuA achieves 99.8% accuracy when detecting acceptable and unacceptable virtually stained tissue images without access to ground truth, also presenting an agreement of 98.5% with the manual assessments made by board-certified pathologists. Besides, AQuA achieves super-human performance in identifying realistic-looking, virtually stained hallucinatory images that would normally mislead human diagnosticians by deceiving them into diagnosing patients that never existed. We further demonstrate the wide adaptability of AQuA across various virtually and histochemically stained tissue images and showcase its strong external generalization to detect unseen hallucination patterns of virtual staining network models as well as artifacts observed in the traditional histochemical staining workflow. This framework creates new opportunities to enhance the reliability of virtual staining and will provide quality assurance for various image generation and transformation tasks in digital pathology and computational imaging. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 37 Pages, 7 Figures

Cross-Forming Control and Fault Current Limiting for Grid-Forming Inverters

Authors: Xiuqiang He, Maitraya Avadhut Desai, Linbin Huang, Florian Dörfler

Abstract: This article proposes a "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming. It differs from classical grid-forming and grid-following paradigms that feature voltage magnitude-and-angle forming and voltage magnitude-and-angle following (or current magnitude-and-angle forming), respecti… ▽ More This article proposes a "cross-forming" control concept for grid-forming inverters operating against grid faults. Cross-forming refers to voltage angle forming and current magnitude forming. It differs from classical grid-forming and grid-following paradigms that feature voltage magnitude-and-angle forming and voltage magnitude-and-angle following (or current magnitude-and-angle forming), respectively. The cross-forming concept addresses the need for inverters to remain grid-forming (particularly voltage angle forming, as required by grid codes) while managing fault current limitation. Simple and feasible cross-forming control implementations are proposed, enabling inverters to quickly limit fault currents to a prescribed level while preserving voltage angle forming for grid-forming synchronization and providing dynamic ancillary services, during symmetrical or asymmetrical fault ride-through. Moreover, the cross-forming control yields an equivalent system featuring a constant virtual impedance and a "normal form" representation, allowing for the extension of previously established transient stability results to include scenarios involving current saturation. Simulations and experiments validate the efficacy of the proposed cross-forming control implementations. △ Less

Submitted 19 July, 2024; v1 submitted 20 April, 2024; originally announced April 2024.

Saturation-Informed Current-Limiting Control for Grid-Forming Converters

Authors: Maitraya Avadhut Desai, Xiuqiang He, Linbin Huang, Florian Dörfler

Abstract: In this paper, we investigate the transient stability of a state-of-the-art grid-forming complex-droop control (i.e., dispatchable virtual oscillator control, dVOC) under current saturation. We quantify the saturation level of a converter by introducing the concept of degree of saturation (DoS), and we propose a provably stable current-limiting control with saturation-informed feedback, which feed… ▽ More In this paper, we investigate the transient stability of a state-of-the-art grid-forming complex-droop control (i.e., dispatchable virtual oscillator control, dVOC) under current saturation. We quantify the saturation level of a converter by introducing the concept of degree of saturation (DoS), and we propose a provably stable current-limiting control with saturation-informed feedback, which feeds the degree of saturation back to the inner voltage-control loop and the outer grid-forming loop. As a result, although the output current is saturated, the voltage phase angle can still be generated from an internal virtual voltage-source node that is governed by an equivalent complex-droop control. We prove that the proposed control achieves transient stability during current saturation under grid faults. We also provide parametric stability conditions for multi-converter systems under grid-connected and islanded scenarios. The stability performance of the current-limiting control is validated with various case studies. △ Less

Submitted 1 July, 2024; v1 submitted 11 April, 2024; originally announced April 2024.

Journal ref: Electric Power Systems Research, 2024

Differentially Private Dual Gradient Tracking for Distributed Resource Allocation

Authors: Wei Huo, Xiaomeng Chen, Lingying Huang, Karl Henrik Johansson, Ling Shi

Abstract: This paper investigates privacy issues in distributed resource allocation over directed networks, where each agent holds a private cost function and optimizes its decision subject to a global coupling constraint through local interaction with other agents. Conventional methods for resource allocation over directed networks require all agents to transmit their original data to neighbors, which pose… ▽ More This paper investigates privacy issues in distributed resource allocation over directed networks, where each agent holds a private cost function and optimizes its decision subject to a global coupling constraint through local interaction with other agents. Conventional methods for resource allocation over directed networks require all agents to transmit their original data to neighbors, which poses the risk of disclosing sensitive and private information. To address this issue, we propose an algorithm called differentially private dual gradient tracking (DP-DGT) for distributed resource allocation, which obfuscates the exchanged messages using independent Laplacian noise. Our algorithm ensures that the agents' decisions converge to a neighborhood of the optimal solution almost surely. Furthermore, without the assumption of bounded gradients, we prove that the cumulative differential privacy loss under the proposed algorithm is finite even when the number of iterations goes to infinity. To the best of our knowledge, we are the first to simultaneously achieve these two goals in distributed resource allocation problems over directed networks. Finally, numerical simulations on economic dispatch problems within the IEEE 14-bus system illustrate the effectiveness of our proposed algorithm. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Unsupervised Learning for Joint Beamforming Design in RIS-aided ISAC Systems

Authors: Junjie Ye, Lei Huang, Zhen Chen, Peichang Zhang, Mohamed Rihan

Abstract: It is critical to design efficient beamforming in reconfigurable intelligent surface (RIS)-aided integrated sensing and communication (ISAC) systems for enhancing spectrum utilization. However, conventional methods often have limitations, either incurring high computational complexity due to iterative algorithms or sacrificing performance when using heuristic methods. To achieve both low complexit… ▽ More It is critical to design efficient beamforming in reconfigurable intelligent surface (RIS)-aided integrated sensing and communication (ISAC) systems for enhancing spectrum utilization. However, conventional methods often have limitations, either incurring high computational complexity due to iterative algorithms or sacrificing performance when using heuristic methods. To achieve both low complexity and high spectrum efficiency, an unsupervised learning-based beamforming design is proposed in this work. We tailor image-shaped channel samples and develop an ISAC beamforming neural network (IBF-Net) model for beamforming. By leveraging unsupervised learning, the loss function incorporates key performance metrics like sensing and communication channel correlation and sensing channel gain, eliminating the need of labeling. Simulations show that the proposed method achieves competitive performance compared to benchmarks while significantly reduces computational complexity. △ Less

Submitted 15 May, 2024; v1 submitted 25 March, 2024; originally announced March 2024.

Comments: Accpeted by IEEE Wireless Communications Letters

Ensuring Disturbance Rejection Performance by Synthesizing Grid-Following and Grid-Forming Inverters in Power Systems

Authors: Fuyilong Ma, Huanhai Xin, Zhiyi Li, Linbin Huang

Abstract: To satisfy dynamic requirements of power systems, it is imperative for grid-tied inverters to ensure good disturbance rejection performance (DRP) under variable grid conditions. This letter discovers and theoretically proves that for general networks, synthesizing grid-following (GFL) inverters and grid-forming (GFM) inverters can always more effectively ensure the DRP of multiple inverters, as co… ▽ More To satisfy dynamic requirements of power systems, it is imperative for grid-tied inverters to ensure good disturbance rejection performance (DRP) under variable grid conditions. This letter discovers and theoretically proves that for general networks, synthesizing grid-following (GFL) inverters and grid-forming (GFM) inverters can always more effectively ensure the DRP of multiple inverters, as compared to homogeneous inverter-based systems that solely utilize either GFL or GFM inverters. The synthesis of GFL inverters and GFM inverters can concurrently increase the smallest eigenvalue and decrease the largest eigenvalue of the network grounded Laplacian matrix. This can be equivalent to rematching the proper short-circuit ratio (SCR) for GFL and GFM inverters, thereby ensuring the DRP of inverters both in weak and strong grids. The results reveal the necessity of synthesizing diverse inverter control schemes from the network-based perspective. Sensitivity function-based tests and real-time simulations validate our results. △ Less

Submitted 26 March, 2024; v1 submitted 25 March, 2024; originally announced March 2024.

Comments: 6 pages

QEAN: Quaternion-Enhanced Attention Network for Visual Dance Generation

Authors: Zhizhen Zhou, Yejing Huo, Guoheng Huang, An Zeng, Xuhang Chen, Lian Huang, Zinuo Li

Abstract: The study of music-generated dance is a novel and challenging Image generation task. It aims to input a piece of music and seed motions, then generate natural dance movements for the subsequent music. Transformer-based methods face challenges in time series prediction tasks related to human movements and music due to their struggle in capturing the nonlinear relationship and temporal aspects. This… ▽ More The study of music-generated dance is a novel and challenging Image generation task. It aims to input a piece of music and seed motions, then generate natural dance movements for the subsequent music. Transformer-based methods face challenges in time series prediction tasks related to human movements and music due to their struggle in capturing the nonlinear relationship and temporal aspects. This can lead to issues like joint deformation, role deviation, floating, and inconsistencies in dance movements generated in response to the music. In this paper, we propose a Quaternion-Enhanced Attention Network (QEAN) for visual dance synthesis from a quaternion perspective, which consists of a Spin Position Embedding (SPE) module and a Quaternion Rotary Attention (QRA) module. First, SPE embeds position information into self-attention in a rotational manner, leading to better learning of features of movement sequences and audio sequences, and improved understanding of the connection between music and dance. Second, QRA represents and fuses 3D motion features and audio features in the form of a series of quaternions, enabling the model to better learn the temporal coordination of music and dance under the complex temporal cycle conditions of dance generation. Finally, we conducted experiments on the dataset AIST++, and the results show that our approach achieves better and more robust performance in generating accurate, high-quality dance movements. Our source code and dataset can be available from https://github.com/MarasyZZ/QEAN and https://google.github.io/aistplusplus_dataset respectively. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: Accepted by The Visual Computer Journal

Distributed Average Consensus via Noisy and Non-Coherent Over-the-Air Aggregation

Authors: Huiwen Yang, Xiaomeng Chen, Lingying Huang, Subhrakanti Dey, Ling Shi

Abstract: Over-the-air aggregation has attracted widespread attention for its potential advantages in task-oriented applications, such as distributed sensing, learning, and consensus. In this paper, we develop a communication-efficient distributed average consensus protocol by utilizing over-the-air aggregation, which exploits the superposition property of wireless channels rather than combat it. Noisy chan… ▽ More Over-the-air aggregation has attracted widespread attention for its potential advantages in task-oriented applications, such as distributed sensing, learning, and consensus. In this paper, we develop a communication-efficient distributed average consensus protocol by utilizing over-the-air aggregation, which exploits the superposition property of wireless channels rather than combat it. Noisy channels and non-coherent transmission are taken into account, and only half-duplex transceivers are required. We prove that the system can achieve average consensus in mean square and even almost surely under the proposed protocol. Furthermore, we extend the analysis to the scenarios with time-varying topology. Numerical simulation shows the effectiveness of the proposed protocol. △ Less

Submitted 11 March, 2024; originally announced March 2024.

One-Bit Target Detection in Collocated MIMO Radar with Colored Background Noise

Authors: Yu-Hang Xiao, David Ramírez, Lei Huang, Xiao Peng Li, Hing Cheung So

Abstract: One-bit sampling has emerged as a promising technique in multiple-input multiple-output (MIMO) radar systems due to its ability to significantly reduce data volume and processing requirements. Nevertheless, current detection methods have not adequately addressed the impact of colored noise, which is frequently encountered in real scenarios. In this paper, we present a novel detection method that a… ▽ More One-bit sampling has emerged as a promising technique in multiple-input multiple-output (MIMO) radar systems due to its ability to significantly reduce data volume and processing requirements. Nevertheless, current detection methods have not adequately addressed the impact of colored noise, which is frequently encountered in real scenarios. In this paper, we present a novel detection method that accounts for colored noise in MIMO radar systems. Specifically, we derive Rao's test by computing the derivative of the likelihood function with respect to the target reflectivity parameter and the Fisher information matrix, resulting in a detector that takes the form of a weighted matched filter. To ensure the constant false alarm rate (CFAR) property, we also consider noise covariance uncertainty and examine its effect on the probability of false alarm. The detection probability is also studied analytically. Simulation results demonstrate that the proposed detector provides considerable performance gains in the presence of colored noise. △ Less

Submitted 26 April, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Optimal Dynamic Ancillary Services Provision Based on Local Power Grid Perception

Authors: Verena Häberle, Xiuqiang He, Linbin Huang, Eduardo Prieto-Araujo, Florian Dörfler

Abstract: In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a… ▽ More In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a set of different feasible response behaviors, among which we aim to find the optimal one to be realized by the converter system. Our approach is based on a so-called "perceive-and-optimize" (P&O) strategy: First, we identify a grid dynamic equivalent at the interconnection terminals of the converter system. Second, we consider the closed-loop interconnection of the identified grid equivalent and the parametric transfer function matrix, which we optimize for the set of transfer function parameters, resulting in a stable and optimal closed-loop performance for ancillary services provision. In the process, we ensure that grid-code and device-level requirements are satisfied. Finally, we demonstrate the effectiveness of our approach in different numerical case studies based on a modified Kundur two-area test system. △ Less

Submitted 28 August, 2024; v1 submitted 31 January, 2024; originally announced January 2024.

Comments: 15 pages, 20 Figures

Self-Attention and Hybrid Features for Replay and Deep-Fake Audio Detection

Authors: Lian Huang, Chi-Man Pun

Abstract: Due to the successful application of deep learning, audio spoofing detection has made significant progress. Spoofed audio with speech synthesis or voice conversion can be well detected by many countermeasures. However, an automatic speaker verification system is still vulnerable to spoofing attacks such as replay or Deep-Fake audio. Deep-Fake audio means that the spoofed utterances are generated u… ▽ More Due to the successful application of deep learning, audio spoofing detection has made significant progress. Spoofed audio with speech synthesis or voice conversion can be well detected by many countermeasures. However, an automatic speaker verification system is still vulnerable to spoofing attacks such as replay or Deep-Fake audio. Deep-Fake audio means that the spoofed utterances are generated using text-to-speech (TTS) and voice conversion (VC) algorithms. Here, we propose a novel framework based on hybrid features with the self-attention mechanism. It is expected that hybrid features can be used to get more discrimination capacity. Firstly, instead of only one type of conventional feature, deep learning features and Mel-spectrogram features will be extracted by two parallel paths: convolution neural networks and a short-time Fourier transform (STFT) followed by Mel-frequency. Secondly, features will be concatenated by a max-pooling layer. Thirdly, there is a Self-attention mechanism for focusing on essential elements. Finally, ResNet and a linear layer are built to get the results. Experimental results reveal that the hybrid features, compared with conventional features, can cover more details of an utterance. We achieve the best Equal Error Rate (EER) of 9.67\% in the physical access (PA) scenario and 8.94\% in the Deep fake task on the ASVspoof 2021 dataset. Compared with the best baseline system, the proposed approach improves by 74.60\% and 60.05\%, respectively. △ Less

Submitted 10 January, 2024; originally announced January 2024.

GainNet: Coordinates the Odd Couple of Generative AI and 6G Networks

Authors: Ning Chen, Jie Yang, Zhipeng Cheng, Xuwei Fan, Zhang Liu, Bangzhen Huang, Yifeng Zhao, Lianfen Huang, Xiaojiang Du, Mohsen Guizani

Abstract: The rapid expansion of AI-generated content (AIGC) reflects the iteration from assistive AI towards generative AI (GAI) with creativity. Meanwhile, the 6G networks will also evolve from the Internet-of-everything to the Internet-of-intelligence with hybrid heterogeneous network architectures. In the future, the interplay between GAI and the 6G will lead to new opportunities, where GAI can learn th… ▽ More The rapid expansion of AI-generated content (AIGC) reflects the iteration from assistive AI towards generative AI (GAI) with creativity. Meanwhile, the 6G networks will also evolve from the Internet-of-everything to the Internet-of-intelligence with hybrid heterogeneous network architectures. In the future, the interplay between GAI and the 6G will lead to new opportunities, where GAI can learn the knowledge of personalized data from the massive connected 6G end devices, while GAI's powerful generation ability can provide advanced network solutions for 6G network and provide 6G end devices with various AIGC services. However, they seem to be an odd couple, due to the contradiction of data and resources. To achieve a better-coordinated interplay between GAI and 6G, the GAI-native networks (GainNet), a GAI-oriented collaborative cloud-edge-end intelligence framework, is proposed in this paper. By deeply integrating GAI with 6G network design, GainNet realizes the positive closed-loop knowledge flow and sustainable-evolution GAI model optimization. On this basis, the GAI-oriented generic resource orchestration mechanism with integrated sensing, communication, and computing (GaiRom-ISCC) is proposed to guarantee the efficient operation of GainNet. Two simple case studies demonstrate the effectiveness and robustness of the proposed schemes. Finally, we envision the key challenges and future directions concerning the interplay between GAI models and 6G networks. △ Less

Submitted 5 January, 2024; originally announced January 2024.

Comments: 10 pages, 5 figures, 1 table

USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis

Authors: Jing Jiao, Jin Zhou, Xiaokang Li, Menghua Xia, Yi Huang, Lihong Huang, Na Wang, Xiaofan Zhang, Shichong Zhou, Yuanyuan Wang, Yi Guo

Abstract: Inadequate generality across different organs and tasks constrains the application of ultrasound (US) image analysis methods in smart healthcare. Building a universal US foundation model holds the potential to address these issues. Nevertheless, the development of such foundational models encounters intrinsic challenges in US analysis, i.e., insufficient databases, low quality, and ineffective fea… ▽ More Inadequate generality across different organs and tasks constrains the application of ultrasound (US) image analysis methods in smart healthcare. Building a universal US foundation model holds the potential to address these issues. Nevertheless, the development of such foundational models encounters intrinsic challenges in US analysis, i.e., insufficient databases, low quality, and ineffective features. In this paper, we present a universal US foundation model, named USFM, generalized to diverse tasks and organs towards label efficient US image analysis. First, a large-scale Multi-organ, Multi-center, and Multi-device US database was built, comprehensively containing over two million US images. Organ-balanced sampling was employed for unbiased learning. Then, USFM is self-supervised pre-trained on the sufficient US database. To extract the effective features from low-quality US images, we proposed a spatial-frequency dual masked image modeling method. A productive spatial noise addition-recovery approach was designed to learn meaningful US information robustly, while a novel frequency band-stop masking learning approach was also employed to extract complex, implicit grayscale distribution and textural variations. Extensive experiments were conducted on the various tasks of segmentation, classification, and image enhancement from diverse organs and diseases. Comparisons with representative US image analysis models illustrate the universality and effectiveness of USFM. The label efficiency experiments suggest the USFM obtains robust performance with only 20% annotation, laying the groundwork for the rapid development of US models in clinical practices. △ Less

Submitted 2 January, 2024; v1 submitted 30 December, 2023; originally announced January 2024.

Comments: Submit to MedIA, 17 pages, 11 figures

Characterizing the Role of Complex Power in Small-Signal Synchronization Stability of Multi-Converter Power Systems

Authors: Fuyilong Ma, Huanhai Xin, Zhiyi Li, Linbin Huang

Abstract: Small-signal synchronization instability (SSI) may be triggered when a grid-connected converter is operated in weak grids. This problem is highly related to the active and reactive power (referred to as complex power) generation or consumption of the converter. Such an instability phenomenon manifests as power oscillations within the bandwidth frequency of phase-locked loop (PLL). However, in a mu… ▽ More Small-signal synchronization instability (SSI) may be triggered when a grid-connected converter is operated in weak grids. This problem is highly related to the active and reactive power (referred to as complex power) generation or consumption of the converter. Such an instability phenomenon manifests as power oscillations within the bandwidth frequency of phase-locked loop (PLL). However, in a multi-converter power system (MCPS), it is challenging to characterize the role of each converter's complex power in SSI issues due to the high dimension of system dynamics. In this paper, we focus on the impact regularity of complex power on the small-signal synchronization stability of the MCPS. Firstly, we formulate the modeling of the MCPS and develop a systematical stability indicator to explicitly quantify the stability margin under various operating complex power. Then, based on the proposed stability indicator, the "resultant force" of complex power on the stability margin of the MCPS can be analytically characterized into a weighted summation form. Furthermore, we elaborate on the impact of each converter's active power and reactive power, respectively. Notably, we demonstrate the cancellation effect between the active power generation and consumption on the stability margin, which ultimately helps prevent SSI issues in the MCPS. The correctness of theoretical findings is well verified by simulations results. △ Less

Submitted 13 February, 2024; v1 submitted 29 December, 2023; originally announced December 2023.

Comments: 11 pages

MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesis

Authors: Wenhao Guan, Yishuang Li, Tao Li, Hukai Huang, Feng Wang, Jiayan Lin, Lingyan Huang, Lin Li, Qingyang Hong

Abstract: The style transfer task in Text-to-Speech refers to the process of transferring style information into text content to generate corresponding speech with a specific style. However, most existing style transfer approaches are either based on fixed emotional labels or reference speech clips, which cannot achieve flexible style transfer. Recently, some methods have adopted text descriptions to guide… ▽ More The style transfer task in Text-to-Speech refers to the process of transferring style information into text content to generate corresponding speech with a specific style. However, most existing style transfer approaches are either based on fixed emotional labels or reference speech clips, which cannot achieve flexible style transfer. Recently, some methods have adopted text descriptions to guide style transfer. In this paper, we propose a more flexible multi-modal and style controllable TTS framework named MM-TTS. It can utilize any modality as the prompt in unified multi-modal prompt space, including reference speech, emotional facial images, and text descriptions, to control the style of the generated speech in a system. The challenges of modeling such a multi-modal style controllable TTS mainly lie in two aspects:1)aligning the multi-modal information into a unified style space to enable the input of arbitrary modality as the style prompt in a single system, and 2)efficiently transferring the unified style representation into the given text content, thereby empowering the ability to generate prompt style-related voice. To address these problems, we propose an aligned multi-modal prompt encoder that embeds different modalities into a unified style space, supporting style transfer for different modalities. Additionally, we present a new adaptive style transfer method named Style Adaptive Convolutions to achieve a better style representation. Furthermore, we design a Rectified Flow based Refiner to solve the problem of over-smoothing Mel-spectrogram and generate audio of higher fidelity. Since there is no public dataset for multi-modal TTS, we construct a dataset named MEAD-TTS, which is related to the field of expressive talking head. Our experiments on the MEAD-TTS dataset and out-of-domain datasets demonstrate that MM-TTS can achieve satisfactory results based on multi-modal prompts. △ Less

Submitted 31 January, 2024; v1 submitted 17 December, 2023; originally announced December 2023.

Comments: Accepted at AAAI2024

In-Sensor Radio Frequency Computing for Energy-Efficient Intelligent Radar

Authors: Yang Sui, Minning Zhu, Lingyi Huang, Chung-Tse Michael Wu, Bo Yuan

Abstract: Radio Frequency Neural Networks (RFNNs) have demonstrated advantages in realizing intelligent applications across various domains. However, as the model size of deep neural networks rapidly increases, implementing large-scale RFNN in practice requires an extensive number of RF interferometers and consumes a substantial amount of energy. To address this challenge, we propose to utilize low-rank dec… ▽ More Radio Frequency Neural Networks (RFNNs) have demonstrated advantages in realizing intelligent applications across various domains. However, as the model size of deep neural networks rapidly increases, implementing large-scale RFNN in practice requires an extensive number of RF interferometers and consumes a substantial amount of energy. To address this challenge, we propose to utilize low-rank decomposition to transform a large-scale RFNN into a compact RFNN while almost preserving its accuracy. Specifically, we develop a Tensor-Train RFNN (TT-RFNN) where each layer comprises a sequence of low-rank third-order tensors, leading to a notable reduction in parameter count, thereby optimizing RF interferometer utilization in comparison to the original large-scale RFNN. Additionally, considering the inherent physical errors when mapping TT-RFNN to RF device parameters in real-world deployment, from a general perspective, we construct the Robust TT-RFNN (RTT-RFNN) by incorporating a robustness solver on TT-RFNN to enhance its robustness. To adapt the RTT-RFNN to varying requirements of reshaping operations, we further provide a reconfigurable reshaping solution employing RF switch matrices. Empirical evaluations conducted on MNIST and CIFAR-10 datasets show the effectiveness of our proposed method. △ Less

Submitted 16 December, 2023; originally announced December 2023.

Energy Efficiency Optimization in Active Reconfigurable Intelligent Surface-Aided Integrated Sensing and Communication Systems

Authors: Junjie Ye, Mohamed Rihan, Peichang Zhang, Lei Huang, Stefano Buzzi, Zhen Chen

Abstract: Energy efficiency (EE) is a challenging task in integrated sensing and communication (ISAC) systems, where high spectral efficiency and low energy consumption appear as conflicting requirements. Although passive reconfigurable intelligent surface (RIS) has emerged as a promising technology for enhancing the EE of the ISAC system, the multiplicative fading feature hinders its effectiveness. This pa… ▽ More Energy efficiency (EE) is a challenging task in integrated sensing and communication (ISAC) systems, where high spectral efficiency and low energy consumption appear as conflicting requirements. Although passive reconfigurable intelligent surface (RIS) has emerged as a promising technology for enhancing the EE of the ISAC system, the multiplicative fading feature hinders its effectiveness. This paper proposes the use of active RIS with its amplification gains to assist the ISAC system for EE improvement. Specifically, we formulate an EE optimization problem in an active RIS-aided ISAC system under system power budgets, considering constraints on user communication quality of service and sensing signal-to-noise ratio (SNR). A novel alternating optimization algorithm is developed to address the highly non-convex problem by leveraging a combination of the generalized Rayleigh quotient optimization approach, semidefinite relaxation (SDR), and the majorization-minimization (MM) framework. Furthermore, to accelerate the algorithm and reduce computational complexity, we derive a semi-closed form for eigenvalue determination. Numerical results demonstrate the effectiveness of the proposed approach, showcasing significant improvements in EE compared to both passive RIS and spectrum efficiency optimization cases. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Improving Large-scale Deep Biasing with Phoneme Features and Text-only Data in Streaming Transducer

Authors: Jin Qiu, Lu Huang, Boyu Li, Jun Zhang, Lu Lu, Zejun Ma

Abstract: Deep biasing for the Transducer can improve the recognition performance of rare words or contextual entities, which is essential in practical applications, especially for streaming Automatic Speech Recognition (ASR). However, deep biasing with large-scale rare words remains challenging, as the performance drops significantly when more distractors exist and there are words with similar grapheme seq… ▽ More Deep biasing for the Transducer can improve the recognition performance of rare words or contextual entities, which is essential in practical applications, especially for streaming Automatic Speech Recognition (ASR). However, deep biasing with large-scale rare words remains challenging, as the performance drops significantly when more distractors exist and there are words with similar grapheme sequences in the bias list. In this paper, we combine the phoneme and textual information of rare words in Transducers to distinguish words with similar pronunciation or spelling. Moreover, the introduction of training with text-only data containing more rare words benefits large-scale deep biasing. The experiments on the LibriSpeech corpus demonstrate that the proposed method achieves state-of-the-art performance on rare word error rate for different scales and levels of bias lists. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: Submitted to ASRU 2023

Integrated Sensing, Communication, and Computing for Cost-effective Multimodal Federated Perception

Authors: Ning Chen, Zhipeng Cheng, Xuwei Fan, Bangzhen Huang, Yifeng Zhao, Lianfen Huang, Xiaojiang Du, Mohsen Guizani

Abstract: Federated learning (FL) is a classic paradigm of 6G edge intelligence (EI), which alleviates privacy leaks and high communication pressure caused by traditional centralized data processing in the artificial intelligence of things (AIoT). The implementation of multimodal federated perception (MFP) services involves three sub-processes, including sensing-based multimodal data generation, communicati… ▽ More Federated learning (FL) is a classic paradigm of 6G edge intelligence (EI), which alleviates privacy leaks and high communication pressure caused by traditional centralized data processing in the artificial intelligence of things (AIoT). The implementation of multimodal federated perception (MFP) services involves three sub-processes, including sensing-based multimodal data generation, communication-based model transmission, and computing-based model training, ultimately relying on available underlying multi-domain physical resources such as time, frequency, and computing power. How to reasonably coordinate the multi-domain resources scheduling among sensing, communication, and computing, therefore, is crucial to the MFP networks. To address the above issues, this paper investigates service-oriented resource management with integrated sensing, communication, and computing (ISCC). With the incentive mechanism of the MFP service market, the resources management problem is redefined as a social welfare maximization problem, where the idea of "expanding resources" and "reducing costs" is used to improve learning performance gain and reduce resource costs. Experimental results demonstrate the effectiveness and robustness of the proposed resource scheduling mechanisms. △ Less

Submitted 7 November, 2023; originally announced November 2023.

Over-the-air Federated Policy Gradient

Authors: Huiwen Yang, Lingying Huang, Subhrakanti Dey, Ling Shi

Abstract: In recent years, over-the-air aggregation has been widely considered in large-scale distributed learning, optimization, and sensing. In this paper, we propose the over-the-air federated policy gradient algorithm, where all agents simultaneously broadcast an analog signal carrying local information to a common wireless channel, and a central controller uses the received aggregated waveform to updat… ▽ More In recent years, over-the-air aggregation has been widely considered in large-scale distributed learning, optimization, and sensing. In this paper, we propose the over-the-air federated policy gradient algorithm, where all agents simultaneously broadcast an analog signal carrying local information to a common wireless channel, and a central controller uses the received aggregated waveform to update the policy parameters. We investigate the effect of noise and channel distortion on the convergence of the proposed algorithm, and establish the complexities of communication and sampling for finding an $ε$-approximate stationary point. Finally, we present some simulation results to show the effectiveness of the algorithm. △ Less

Submitted 25 February, 2024; v1 submitted 25 October, 2023; originally announced October 2023.

Comments: To appear at IEEE ICC 2024

Quantitative Stability Conditions for Grid-Forming Converters With Complex Droop Control

Authors: Xiuqiang He, Linbin Huang, Irina Subotić, Verena Häberle, Florian Dörfler

Abstract: In this paper, we analytically study the transient stability of grid-connected converters with grid-forming complex droop control, also known as dispatchable virtual oscillator control. We prove theoretically that complex droop control, as a state-of-the-art grid-forming control, always possesses steady-state equilibria whereas classical droop control does not. We provide quantitative conditions f… ▽ More In this paper, we analytically study the transient stability of grid-connected converters with grid-forming complex droop control, also known as dispatchable virtual oscillator control. We prove theoretically that complex droop control, as a state-of-the-art grid-forming control, always possesses steady-state equilibria whereas classical droop control does not. We provide quantitative conditions for complex droop control maintaining transient stability (global asymptotic stability) under grid disturbances, which is beyond the well-established local (non-global) stability for classical droop control. For the transient instability of complex droop control, we reveal that the unstable trajectories are bounded, manifesting as limit cycle oscillations. Moreover, we extend our stability results from second-order grid-forming control dynamics to full-order system dynamics that additionally encompass both circuit electromagnetic transients and inner-loop dynamics. Our theoretical results contribute an insightful understanding of the transient stability and instability of complex droop control and offer practical guidelines for parameter tuning and stability guarantees. △ Less

Submitted 23 May, 2024; v1 submitted 15 October, 2023; originally announced October 2023.

Journal ref: IEEE Transactions on Power Electronics, 2024

A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods

Authors: Ling Huang, Su Ruan, Yucheng Xing, Mengling Feng

Abstract: The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods hav… ▽ More The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the result. In this review, we offer a comprehensive overview of prevailing methods proposed to quantify uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models. △ Less

Submitted 9 October, 2023; originally announced October 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.03736 by other authors

Dementia Assessment Using Mandarin Speech with an Attention-based Speech Recognition Encoder

Authors: Zih-Jyun Lin, Yi-Ju Chen, Po-Chih Kuo, Likai Huang, Chaur-Jong Hu, Cheng-Yu Chen

Abstract: Dementia diagnosis requires a series of different testing methods, which is complex and time-consuming. Early detection of dementia is crucial as it can prevent further deterioration of the condition. This paper utilizes a speech recognition model to construct a dementia assessment system tailored for Mandarin speakers during the picture description task. By training an attention-based speech reco… ▽ More Dementia diagnosis requires a series of different testing methods, which is complex and time-consuming. Early detection of dementia is crucial as it can prevent further deterioration of the condition. This paper utilizes a speech recognition model to construct a dementia assessment system tailored for Mandarin speakers during the picture description task. By training an attention-based speech recognition model on voice data closely resembling real-world scenarios, we have significantly enhanced the model's recognition capabilities. Subsequently, we extracted the encoder from the speech recognition model and added a linear layer for dementia assessment. We collected Mandarin speech data from 99 subjects and acquired their clinical assessments from a local hospital. We achieved an accuracy of 92.04% in Alzheimer's disease detection and a mean absolute error of 9% in clinical dementia rating score prediction. △ Less

Submitted 15 December, 2023; v1 submitted 5 October, 2023; originally announced October 2023.

Comments: Accepted to IEEE ICASSP 2024

Dynamic Ancillary Services: From Grid Codes to Transfer Function-Based Converter Control

Authors: Verena Häberle, Linbin Huang, Xiuqiang He, Eduardo Prieto-Araujo, Florian Dörfler

Abstract: Conventional grid-code specifications for dynamic ancillary services provision such as fast frequency and voltage regulation are typically defined by means of piece-wise linear step-response capability curves in the time domain. However, although the specification of such time-domain curves is straightforward, their practical implementation in a converter-based generation system is not immediate,… ▽ More Conventional grid-code specifications for dynamic ancillary services provision such as fast frequency and voltage regulation are typically defined by means of piece-wise linear step-response capability curves in the time domain. However, although the specification of such time-domain curves is straightforward, their practical implementation in a converter-based generation system is not immediate, and no customary methods have been developed yet. In this paper, we thus propose a systematic approach for the practical implementation of piece-wise linear time-domain curves to provide dynamic ancillary services by converter-based generation systems, while ensuring grid-code and device-level requirements to be reliably satisfied. Namely, we translate the piece-wise linear time-domain curves for active and reactive power provision in response to a frequency and voltage step change into a desired rational parametric transfer function in the frequency domain, which defines a dynamic response behavior to be realized by the converter. The obtained transfer function can be easily implemented e.g. via a PI-based matching control in the power loop of standard converter control architectures. We demonstrate the performance of our method in numerical grid-code compliance tests, and reveal its superiority over classical droop and virtual inertia schemes which may not satisfy the grid codes due to their structural limitations. △ Less

Submitted 28 August, 2024; v1 submitted 2 October, 2023; originally announced October 2023.

Comments: 8 pages, 11 figures

Gain and Phase: Decentralized Stability Conditions for Power Electronics-Dominated Power Systems

Authors: Linbin Huang, Dan Wang, Xiongfei Wang, Huanhai Xin, Ping Ju, Karl H. Johansson, Florian Dörfler

Abstract: This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, w… ▽ More This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, which aim at evaluating the closed-loop system stability by comparing the individual converter dynamics with the network dynamics in a decentralized and open-loop manner. Moreover, our approach can handle heterogeneous converters' dynamics and is suitable to analyze large-scale multi-converter power systems that contain grid-following (GFL), grid-forming (GFM) converters, and synchronous generators. Compared with other decentralized stability conditions, e.g., passivity-based stability conditions, the proposed conditions are significantly less conservative and can be generally satisfied in practice across the whole frequency range. △ Less

Submitted 10 January, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

Authors: Ling Huang, Su Ruan, Pierre Decazes, Thierry Denoeux

Abstract: Single-modality medical images generally do not contain enough information to reach an accurate and reliable diagnosis. For this reason, physicians generally diagnose diseases based on multimodal medical images such as, e.g., PET/CT. The effective fusion of multimodal information is essential to reach a reliable decision and explain how the decision is made as well. In this paper, we propose a fus… ▽ More Single-modality medical images generally do not contain enough information to reach an accurate and reliable diagnosis. For this reason, physicians generally diagnose diseases based on multimodal medical images such as, e.g., PET/CT. The effective fusion of multimodal information is essential to reach a reliable decision and explain how the decision is made as well. In this paper, we propose a fusion framework for multimodal medical image segmentation based on deep learning and the Dempster-Shafer theory of evidence. In this framework, the reliability of each single modality image when segmenting different objects is taken into account by a contextual discounting operation. The discounted pieces of evidence from each modality are then combined by Dempster's rule to reach a final decision. Experimental results with a PET-CT dataset with lymphomas and a multi-MRI dataset with brain tumors show that our method outperforms the state-of-the-art methods in accuracy and reliability. △ Less

Submitted 18 August, 2024; v1 submitted 11 September, 2023; originally announced September 2023.

Joint Oscillation Damping and Inertia Provision Service for Converter-Interfaced Generation

Authors: Cheng Feng, Linbin Huang, Xiuqiang He, Yi Wang, Florian Dörfler, Qixin Chen

Abstract: As renewable generation becomes more prevalent, traditional power systems dominated by synchronous generators are transitioning to systems dominated by converter-interfaced generation. These devices, with their weaker damping capabilities and lower inertia, compromise the system's ability to withstand disturbances, pose a threat to system stability, and lead to oscillations and poor frequency resp… ▽ More As renewable generation becomes more prevalent, traditional power systems dominated by synchronous generators are transitioning to systems dominated by converter-interfaced generation. These devices, with their weaker damping capabilities and lower inertia, compromise the system's ability to withstand disturbances, pose a threat to system stability, and lead to oscillations and poor frequency response performance. While some new converter-interfaced generations are capable of providing superior damping and fast frequency control, there is a lack of effective measures to incentivize manufacturers to adopt them. To address this gap, this paper defines the joint oscillation damping and inertia provision services at the system level, seeking to encourage converter-interfaced generation to provide enhanced damping and fast frequency response capabilities. Our approach is anchored in a novel convex parametric formulation that combines oscillation mode and frequency stability constraints. These constraints ensure a sufficient damping ratio for all oscillation modes and maintain transient frequency trajectories within acceptable limits. They are designed to integrate smoothly into various operational and planning optimization frameworks. Using this formulation, we introduce a joint service for oscillation damping and inertia provision based on a cost-minimization problem. This facilitates the optimal allocation of damping and virtual inertia to converters, achieving both small-signal stability and frequency stability. Furthermore, we investigate the economic effects of introducing this service into a new ancillary service market, assessing its impact on system operations and cost-efficiency. Numerical tests highlight the service's efficacy in ensuring both small-signal stability and frequency stability, and offer insights into potential economic benefits. △ Less

Submitted 3 September, 2023; originally announced September 2023.

Comments: Submitted for IEEE PES journal for possible publications

Illumination strategies for space-bandwidth-time product improvement in Fourier ptychography

Authors: Haibo Xu, Cheng Li, Mingzhe Wei, Ziwen Zhou, Longqian Huang

Abstract: Fourier ptychography (FP) is a promising technique for high-throughput imaging. Reconstruction algorithms and illumination paradigm are two key aspects of FP. In this review, we mainly focus on illumination strategies in FP. We derive the space-bandwidth-time product (SBP-T) for the characterization of FP performance. Based on the analysis of SBP-T, we categorize the illumination strategy in FP ef… ▽ More Fourier ptychography (FP) is a promising technique for high-throughput imaging. Reconstruction algorithms and illumination paradigm are two key aspects of FP. In this review, we mainly focus on illumination strategies in FP. We derive the space-bandwidth-time product (SBP-T) for the characterization of FP performance. Based on the analysis of SBP-T, we categorize the illumination strategy in FP effectively and discuss each category △ Less

Submitted 26 August, 2023; originally announced August 2023.

Non-Intrusive Electric Load Monitoring Approach Based on Current Feature Visualization for Smart Energy Management

Authors: Yiwen Xu, Dengfeng Liu, Liangtao Huang, Zhiquan Lin, Tiesong Zhao, Sam Kwong

Abstract: The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-invasive load monitoring method for smart electric ener… ▽ More The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-invasive load monitoring method for smart electric energy management. First of all, we utilize both signal transforms (including wavelet transform and discrete Fourier transform) and Gramian Angular Field (GAF) methods to map one-dimensional current signals onto two-dimensional color feature images. Second, we propose to recognize all electric loads from color feature images using a U-shape deep neural network with multi-scale feature extraction and attention mechanism. Third, we design our method as a cloud-based, non-invasive monitoring of all users, thereby saving energy cost during electric power system control. Experimental results on both public and our private datasets have demonstrated our method achieves superior performances than its peers, and thus supports efficient energy management over large-scale Internet of Things (IoT). △ Less

Submitted 8 August, 2023; originally announced August 2023.

Virtual histological staining of unlabeled autopsy tissue

Authors: Yuzhu Li, Nir Pillar, Jingxi Li, Tairan Liu, Di Wu, Songyu Sun, Guangdong Ma, Kevin de Haan, Luzhe Huang, Sepehr Hamidi, Anatoly Urisman, Tal Keidar Haran, William Dean Wallace, Jonathan E. Zuckerman, Aydogan Ozcan

Abstract: Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, a… ▽ More Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, and time. These challenges can become more pronounced during global health crises when the availability of histopathology services is limited, resulting in further delays in tissue fixation and more severe staining artifacts. Here, we report the first demonstration of virtual staining of autopsy tissue and show that a trained neural network can rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images that match hematoxylin and eosin (H&E) stained versions of the same samples, eliminating autolysis-induced severe staining artifacts inherent in traditional histochemical staining of autopsied tissue. Our virtual H&E model was trained using >0.7 TB of image data and a data-efficient collaboration scheme that integrates the virtual staining network with an image registration network. The trained model effectively accentuated nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining failed to provide consistent staining quality. This virtual autopsy staining technique can also be extended to necrotic tissue, and can rapidly and cost-effectively generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining. △ Less

Submitted 1 August, 2023; originally announced August 2023.

Comments: 24 Pages, 7 Figures

Journal ref: Nature Communications (2024)

Sensor Selection for Remote State Estimation with QoS Requirement Constraints

Authors: Huiwen Yang, Lingying Huang, Chao Yang, Yilin Mo, Ling Shi

Abstract: In this paper, we study the sensor selection problem for remote state estimation under the Quality-of-Service (QoS) requirement constraints. Multiple sensors are employed to observe a linear time-invariant system, and their measurements should be transmitted to a remote estimator for state estimation. However, due to the limited communication resources and the QoS requirement constraints, only som… ▽ More In this paper, we study the sensor selection problem for remote state estimation under the Quality-of-Service (QoS) requirement constraints. Multiple sensors are employed to observe a linear time-invariant system, and their measurements should be transmitted to a remote estimator for state estimation. However, due to the limited communication resources and the QoS requirement constraints, only some of the sensors can be allowed to transmit their measurements. To estimate the system state as accurately as possible, it is essential to select sensors for transmission appropriately. We formulate the sensor selection problem as a non-convex optimization problem. It is difficult to solve such a problem and even to find a feasible solution. To obtain a solution which can achieve good estimation performance, we first reformulate and relax the formulated problem. Then, we propose an algorithm based on successive convex approximation (SCA) to solve the relaxed problem. By utilizing the solution of the relaxed problem, we propose a heuristic sensor selection algorithm which can provide a good suboptimal solution. Simulation results are presented to show the effectiveness of the proposed heuristic. △ Less

Submitted 26 June, 2023; originally announced June 2023.

One-Bit Spectrum Sensing for Cognitive Radio

Authors: Pei-Wen Wu, Lei Huang, David Ramírez, Yu-Hang Xiao, Hing Cheung So

Abstract: Spectrum sensing in cognitive radio necessitates effective monitoring of wide bandwidths, which requires high-rate sampling. Traditional spectrum sensing methods employing high-precision analog-to-digital converters (ADCs) result in increased power consumption and expensive hardware costs. In this paper, we explore blind spectrum sensing utilizing one-bit ADCs. We derive a closed-form detector bas… ▽ More Spectrum sensing in cognitive radio necessitates effective monitoring of wide bandwidths, which requires high-rate sampling. Traditional spectrum sensing methods employing high-precision analog-to-digital converters (ADCs) result in increased power consumption and expensive hardware costs. In this paper, we explore blind spectrum sensing utilizing one-bit ADCs. We derive a closed-form detector based on Rao's test and demonstrate its equivalence with the second-order eigenvalue-moment-ratio test. Furthermore, a near-exact distribution based on the moment-based method, and an approximate distribution in the low signal-to-noise ratio (SNR) regime with the use of the central limit theorem, are obtained. Theoretical analysis is then performed and our results show that the performance loss of the proposed detector is approximately $2$ dB ($π/2$) compared to detectors employing $\infty$-bit ADCs when SNR is low. This loss can be compensated for by using approximately $2.47$ ($π^2/4$) times more samples. In addition, we unveil that the efficiency of incoherent accumulation in one-bit detection is the square root of that of coherent accumulation. Simulation results corroborate the correctness of our theoretical calculations. △ Less

Submitted 23 June, 2023; originally announced June 2023.

Text-only Domain Adaptation using Unified Speech-Text Representation in Transducer

Authors: Lu Huang, Boyu Li, Jun Zhang, Lu Lu, Zejun Ma

Abstract: Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We present a method to learn Unified Speech-Text Representation in Conformer Transducer(USTR-CT) to enable fast domain adaptation using the text-only corpus. Different from the previous textogram method, an extra text encoder i… ▽ More Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We present a method to learn Unified Speech-Text Representation in Conformer Transducer(USTR-CT) to enable fast domain adaptation using the text-only corpus. Different from the previous textogram method, an extra text encoder is introduced in our work to learn text representation and is removed during inference, so there is no modification for online deployment. To improve the efficiency of adaptation, single-step and multi-step adaptations are also explored. The experiments on adapting LibriSpeech to SPGISpeech show the proposed method reduces the word error rate(WER) by relatively 44% on the target domain, which is better than those of TTS method and textogram method. Also, it is shown the proposed method can be combined with internal language model estimation(ILME) to further improve the performance. △ Less

Submitted 6 June, 2023; originally announced June 2023.

Comments: Submitted to Interspeech 2023

Modulate Your Spectrum in Self-Supervised Learning

Authors: Xi Weng, Yunhao Ni, Tengwei Song, Jie Luo, Rao Muhammad Anwer, Salman Khan, Fahad Shahbaz Khan, Lei Huang

Abstract: Whitening loss offers a theoretical guarantee against feature collapse in self-supervised learning (SSL) with joint embedding architectures. Typically, it involves a hard whitening approach, transforming the embedding and applying loss to the whitened output. In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond… ▽ More Whitening loss offers a theoretical guarantee against feature collapse in self-supervised learning (SSL) with joint embedding architectures. Typically, it involves a hard whitening approach, transforming the embedding and applying loss to the whitened output. In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse. We show that whitening is a special instance of ST by definition, and our empirical investigations unveil other ST instances capable of preventing collapse. Additionally, we propose a novel ST instance named IterNorm with trace loss (INTL). Theoretical analysis confirms INTL's efficacy in preventing collapse and modulating the spectrum of embedding toward equal-eigenvalues during optimization. Our experiments on ImageNet classification and COCO object detection demonstrate INTL's potential in learning superior representations. The code is available at https://github.com/winci-ai/INTL. △ Less

Submitted 21 January, 2024; v1 submitted 26 May, 2023; originally announced May 2023.

Comments: Accepted at ICLR 2024. The code is available at https://github.com/winci-ai/intl

Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems

Authors: Luzhe Huang, Jianing Li, Xiaofu Ding, Yijie Zhang, Hanlong Chen, Aydogan Ozcan

Abstract: Uncertainty estimation is critical for numerous applications of deep neural networks and draws growing attention from researchers. Here, we demonstrate an uncertainty quantification approach for deep neural networks used in inverse problems based on cycle consistency. We build forward-backward cycles using the physical forward model available and a trained deep neural network solving the inverse p… ▽ More Uncertainty estimation is critical for numerous applications of deep neural networks and draws growing attention from researchers. Here, we demonstrate an uncertainty quantification approach for deep neural networks used in inverse problems based on cycle consistency. We build forward-backward cycles using the physical forward model available and a trained deep neural network solving the inverse problem at hand, and accordingly derive uncertainty estimators through regression analysis on the consistency of these forward-backward cycles. We theoretically analyze cycle consistency metrics and derive their relationship with respect to uncertainty, bias, and robustness of the neural network inference. To demonstrate the effectiveness of these cycle consistency-based uncertainty estimators, we classified corrupted and out-of-distribution input image data using some of the widely used image deblurring and super-resolution neural networks as testbeds. The blind testing of our method outperformed other models in identifying unseen input data corruption and distribution shifts. This work provides a simple-to-implement and rapid uncertainty quantification method that can be universally applied to various neural networks used for solving inverse problems. △ Less

Submitted 22 May, 2023; originally announced May 2023.

Comments: 28 Pages, 4 Figures, 1 Table

Journal ref: Intelligent Computing, AAAS (2023)

MIMO Grid Impedance Identification of Three-Phase Power Systems: Parametric vs. Nonparametric Approaches

Authors: Verena Häberle, Linbin Huang, Xiuqiang He, Eduardo Prieto-Araujo, Roy S. Smith, Florian Dörfler

Abstract: A fast and accurate grid impedance measurement of three-phase power systems is crucial for online assessment of power system stability and adaptive control of grid-connected converters. Existing grid impedance measurement approaches typically rely on pointwise sinusoidal injections or sequential wideband perturbations to identify a nonparametric grid impedance curve via fast Fourier computations i… ▽ More A fast and accurate grid impedance measurement of three-phase power systems is crucial for online assessment of power system stability and adaptive control of grid-connected converters. Existing grid impedance measurement approaches typically rely on pointwise sinusoidal injections or sequential wideband perturbations to identify a nonparametric grid impedance curve via fast Fourier computations in the frequency domain. This is not only time-consuming, but also inaccurate during time-varying grid conditions, while on top of that, the identified nonparametric model cannot be immediately used for stability analysis or control design. To tackle these problems, we propose to use parametric system identification techniques (e.g., prediction error or subspace methods) to obtain a parametric impedance model directly from time-domain current and voltage data. Our approach relies on injecting wideband excitation signals in the converter's controller and allows to accurately identify the grid impedance in closed loop within one injection and measurement cycle. Even though the underlying parametric system identification techniques are well-studied in general, their utilization in a grid impedance identification setup poses specific challenges, is vastly underexplored, and has not gained adequate attention in urgent and timely power systems applications. To this end, we demonstrate in numerical experiments how the proposed parametric approach can accomplish a significant improvement compared to prevalent nonparametric methods. △ Less

Submitted 29 November, 2023; v1 submitted 29 April, 2023; originally announced May 2023.

Comments: 7 pages, 7 figures