Showing 1–50 of 164 results for author: Ju, Y

Robust Federated Finetuning of Foundation Models via Alternating Minimization of LoRA

Authors: Shuangyi Chen, Yue Ju, Hardik Dalal, Zhongwen Zhu, Ashish Khisti

Abstract: Parameter-Efficient Fine-Tuning (PEFT) has risen as an innovative training strategy that updates only a select few model parameters, significantly lowering both computational and memory demands. PEFT also helps to decrease data transfer in federated learning settings, where communication depends on the size of updates. In this work, we explore the constraints of previous studies that integrate a w… ▽ More Parameter-Efficient Fine-Tuning (PEFT) has risen as an innovative training strategy that updates only a select few model parameters, significantly lowering both computational and memory demands. PEFT also helps to decrease data transfer in federated learning settings, where communication depends on the size of updates. In this work, we explore the constraints of previous studies that integrate a well-known PEFT method named LoRA with federated fine-tuning, then introduce RoLoRA, a robust federated fine-tuning framework that utilizes an alternating minimization approach for LoRA, providing greater robustness against decreasing fine-tuning parameters and increasing data heterogeneity. Our results indicate that RoLoRA not only presents the communication benefits but also substantially enhances the robustness and effectiveness in multiple federated fine-tuning scenarios. △ Less

Submitted 3 September, 2024; originally announced September 2024.

Comments: Presented at ES-FOMO-II@ICML2024

Searching for MeV-scale Axion-like Particles and Dark Photons with PandaX-4T

Authors: PandaX Collaboration, Tao Li, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke HanChangda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (76 additional authors not shown)

Abstract: Axion-like particles (ALPs) and dark photons (DPs) are viable dark matter particle candidates. We have searched for possible ALP/DP signals in the PandaX-4T liquid xenon detector using 94.8 days of data. A binned likelihood fit is constructed to search for possible mono-energetic peaks induced by the absorption processes between ALPs/DPs and atomic electrons of xenon. A detailed temporal model of… ▽ More Axion-like particles (ALPs) and dark photons (DPs) are viable dark matter particle candidates. We have searched for possible ALP/DP signals in the PandaX-4T liquid xenon detector using 94.8 days of data. A binned likelihood fit is constructed to search for possible mono-energetic peaks induced by the absorption processes between ALPs/DPs and atomic electrons of xenon. A detailed temporal model of decays associated with xenon isotopes is introduced to constrain the number of background events. No signal excess over background expectations is observed, and we have established the most stringent exclusion limits for most ALP/DP masses ranging from 150 keV/$c^2$ to 1 MeV/$c^2$. △ Less

Submitted 1 September, 2024; originally announced September 2024.

Exploring New Physics with PandaX-4T Low Energy Electronic Recoil Data

Authors: PandaX Collaboration, Xinning Zeng, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke HanChangda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (76 additional authors not shown)

Abstract: New particles beyond the Standard Model of particle physics, such as axions, can be effectively searched through their interactions with electrons. We use the large liquid xenon detector PandaX-4T to search for novel electronic recoil signals induced by solar axions, neutrinos with anomalous magnetic moment, axion-like particles, dark photons, and light fermionic dark matter. A detailed background… ▽ More New particles beyond the Standard Model of particle physics, such as axions, can be effectively searched through their interactions with electrons. We use the large liquid xenon detector PandaX-4T to search for novel electronic recoil signals induced by solar axions, neutrinos with anomalous magnetic moment, axion-like particles, dark photons, and light fermionic dark matter. A detailed background model is established with the latest datasets with 1.54 $\rm tonne \cdot year$ exposure. No significant excess above the background has been observed, and we have obtained competitive constraints for axion couplings, neutrino magnetic moment, and fermionic dark matter interactions. △ Less

Submitted 14 August, 2024; originally announced August 2024.

AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies

Authors: Bo-Wen Zhang, Liangdong Wang, Ye Yuan, Jijie Li, Shuhao Gu, Mengdi Zhao, Xinya Wu, Guang Liu, Chengwei Wu, Hanyu Zhao, Li Du, Yiming Ju, Quanyue Ma, Yulong Ao, Yingli Zhao, Songhe Zhu, Zhou Cao, Dong Liang, Yonghua Lin, Ming Zhang, Shunfei Wang, Yanxin Zhou, Min Ye, Xuekai Chen, Xinyang Yu , et al. (2 additional authors not shown)

Abstract: In recent years, with the rapid application of large language models across various fields, the scale of these models has gradually increased, and the resources required for their pre-training have grown exponentially. Training an LLM from scratch will cost a lot of computation resources while scaling up from a smaller model is a more efficient approach and has thus attracted significant attention… ▽ More In recent years, with the rapid application of large language models across various fields, the scale of these models has gradually increased, and the resources required for their pre-training have grown exponentially. Training an LLM from scratch will cost a lot of computation resources while scaling up from a smaller model is a more efficient approach and has thus attracted significant attention. In this paper, we present AquilaMoE, a cutting-edge bilingual 8*16B Mixture of Experts (MoE) language model that has 8 experts with 16 billion parameters each and is developed using an innovative training methodology called EfficientScale. This approach optimizes performance while minimizing data requirements through a two-stage process. The first stage, termed Scale-Up, initializes the larger model with weights from a pre-trained smaller model, enabling substantial knowledge transfer and continuous pretraining with significantly less data. The second stage, Scale-Out, uses a pre-trained dense model to initialize the MoE experts, further enhancing knowledge transfer and performance. Extensive validation experiments on 1.8B and 7B models compared various initialization schemes, achieving models that maintain and reduce loss during continuous pretraining. Utilizing the optimal scheme, we successfully trained a 16B model and subsequently the 8*16B AquilaMoE model, demonstrating significant improvements in performance and training efficiency. △ Less

Submitted 12 August, 2024; originally announced August 2024.

Dark Matter Search Results from 1.54 Tonne$\cdot$Year Exposure of PandaX-4T

Authors: PandaX Collaboration, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (77 additional authors not shown)

Abstract: In this letter, we report the dark matter search results from the commissioning run and the first science run of the PandaX-4T experiment. A blind analysis is carried out on the entire data set. The data processing is improved compared to previous work, unifying the low-level signal reconstruction in a wide energy range up to 120 keV. With a total exposure of 1.54 tonne$\cdot$year, no significant… ▽ More In this letter, we report the dark matter search results from the commissioning run and the first science run of the PandaX-4T experiment. A blind analysis is carried out on the entire data set. The data processing is improved compared to previous work, unifying the low-level signal reconstruction in a wide energy range up to 120 keV. With a total exposure of 1.54 tonne$\cdot$year, no significant excess of nuclear recoil events is found. The lowest 90% confidence level exclusion on the spin-independent cross section is $1.6 \times 10^{-47} \mathrm{cm}^2$ at a dark matter mass of 40 GeV$/c^2$. Our results represent the most stringent constraint for a dark matter mass above 100 GeV$/c^2$. △ Less

Submitted 1 August, 2024; originally announced August 2024.

In-Situ Techniques on GPU-Accelerated Data-Intensive Applications

Authors: Yi Ju, Mingshuai Li, Adalberto Perez, Laura Bellentani, Niclas Jansson, Stefano Markidis, Philipp Schlatter, Erwin Laure

Abstract: The computational power of High-Performance Computing (HPC) systems is constantly increasing, however, their input/output (IO) performance grows relatively slowly, and their storage capacity is also limited. This unbalance presents significant challenges for applications such as Molecular Dynamics (MD) and Computational Fluid Dynamics (CFD), which generate massive amounts of data for further visua… ▽ More The computational power of High-Performance Computing (HPC) systems is constantly increasing, however, their input/output (IO) performance grows relatively slowly, and their storage capacity is also limited. This unbalance presents significant challenges for applications such as Molecular Dynamics (MD) and Computational Fluid Dynamics (CFD), which generate massive amounts of data for further visualization or analysis. At the same time, checkpointing is crucial for long runs on HPC clusters, due to limited walltimes and/or failures of system components, and typically requires the storage of large amount of data. Thus, restricted IO performance and storage capacity can lead to bottlenecks for the performance of full application workflows (as compared to computational kernels without IO). In-situ techniques, where data is further processed while still in memory rather to write it out over the I/O subsystem, can help to tackle these problems. In contrast to traditional post-processing methods, in-situ techniques can reduce or avoid the need to write or read data via the IO subsystem. They offer a promising approach for applications aiming to leverage the full power of large scale HPC systems. In-situ techniques can also be applied to hybrid computational nodes on HPC systems consisting of graphics processing units (GPUs) and central processing units (CPUs). On one node, the GPUs would have significant performance advantages over the CPUs. Therefore, current approaches for GPU-accelerated applications often focus on maximizing GPU usage, leaving CPUs underutilized. In-situ tasks using CPUs to perform data analysis or preprocess data concurrently to the running simulation, offer a possibility to improve this underutilization. △ Less

Submitted 30 July, 2024; originally announced July 2024.

Understanding the Impact of Synchronous, Asynchronous, and Hybrid In-Situ Techniques in Computational Fluid Dynamics Applications

Authors: Yi Ju, Adalberto Perez, Stefano Markidis, Philipp Schlatter, Erwin Laure

Abstract: High-Performance Computing (HPC) systems provide input/output (IO) performance growing relatively slowly compared to peak computational performance and have limited storage capacity. Computational Fluid Dynamics (CFD) applications aiming to leverage the full power of Exascale HPC systems, such as the solver Nek5000, will generate massive data for further processing. These data need to be efficient… ▽ More High-Performance Computing (HPC) systems provide input/output (IO) performance growing relatively slowly compared to peak computational performance and have limited storage capacity. Computational Fluid Dynamics (CFD) applications aiming to leverage the full power of Exascale HPC systems, such as the solver Nek5000, will generate massive data for further processing. These data need to be efficiently stored via the IO subsystem. However, limited IO performance and storage capacity may result in performance, and thus scientific discovery, bottlenecks. In comparison to traditional post-processing methods, in-situ techniques can reduce or avoid writing and reading the data through the IO subsystem, promising to be a solution to these problems. In this paper, we study the performance and resource usage of three in-situ use cases: data compression, image generation, and uncertainty quantification. We furthermore analyze three approaches when these in-situ tasks and the simulation are executed synchronously, asynchronously, or in a hybrid manner. In-situ compression can be used to reduce the IO time and storage requirements while maintaining data accuracy. Furthermore, in-situ visualization and analysis can save Terabytes of data from being routed through the IO subsystem to storage. However, the overall efficiency is crucially dependent on the characteristics of both, the in-situ task and the simulation. In some cases, the overhead introduced by the in-situ tasks can be substantial. Therefore, it is essential to choose the proper in-situ approach, synchronous, asynchronous, or hybrid, to minimize overhead and maximize the benefits of concurrent execution. △ Less

Submitted 30 July, 2024; originally announced July 2024.

First Measurement of Solar $^8$B Neutrino Flux through Coherent Elastic Neutrino-Nucleus Scattering in PandaX-4T

Authors: PandaX Collaboration, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Zhixing Gao, Lisheng Geng, Karl Giboni, Xunan Guo, Xuyuan Guo, Zichao Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Houqi Huang, Junting Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (77 additional authors not shown)

Abstract: The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (… ▽ More The PandaX-4T liquid xenon detector at the China Jinping Underground Laboratory is used to measure the solar $^8$B neutrino flux by detecting neutrinos through coherent scattering with xenon nuclei. Data samples requiring the coincidence of scintillation and ionization signals (paired), as well as unpaired ionization-only signals (US2), are selected with energy threshold of approximately 1.1 keV (0.33 keV) nuclear recoil energy. Combining the commissioning run and the first science run of PandaX-4T, a total exposure of 1.25 and 1.04 tonne$\cdot$year are collected for the paired and US2, respectively. After unblinding, 3 and 332 events are observed with an expectation of 2.8$\pm$0.5 and 251$\pm$32 background events, for the paired and US2 data, respectively. A combined analysis yields a best-fit $^8$B neutrino signal of 3.5 (75) events from the paired (US2) data sample, with $\sim$37\% uncertainty, and the background-only hypothesis is disfavored at 2.64$σ$ significance. This gives a solar $^8$B neutrino flux of ($8.4\pm3.1$)$\times$10$^6$ cm$^{-2}$s$^{-1}$, consistent with the standard solar model prediction. This is the first indication of solar $^8$B neutrino ``fog'' in a dark matter direct detection experiment. △ Less

Submitted 15 July, 2024; originally announced July 2024.

Twist angle driven electronic structure evolution of twisted bilayer graphene

Authors: Jiawei Yu, Guihao Jia, Qian Li, Yuyang Wang, Kebin Xiao, Yongkang Ju, Hongyun Zhang, Zhiqiang Hu, Yunkai Guo, Biao Lian, Peizhe Tang, Shuyun Zhou, Qi-Kun Xue, Wei Li

Abstract: In twisted bilayer graphene (TBG) devices, local strains often coexist and entangle with the twist-angle dependent moiré superlattice, both of which can significantly affect the electronic properties of TBG. Here, using low-temperature scanning tunneling microscopy, we investigate the fine evolution of the electronic structures of a TBG device with continuous variation of twist angles from 0.32° t… ▽ More In twisted bilayer graphene (TBG) devices, local strains often coexist and entangle with the twist-angle dependent moiré superlattice, both of which can significantly affect the electronic properties of TBG. Here, using low-temperature scanning tunneling microscopy, we investigate the fine evolution of the electronic structures of a TBG device with continuous variation of twist angles from 0.32° to 1.29°, spanning the first (1.1°), second (0.5°) and third (0.3°) magic angles. We reveal the exotic behavior of the flat bands and remote bands in both the energy space and real space near the magic angles. Interestingly, we observe an anomalous spectral weight transfer between the two flat band peaks in the tunneling spectra when approaching the first magic angle, suggesting strong inter-flat-bands interactions. The position of the remote band peak can be an index for the twist angle in TBG, since it positively correlates with the twist angle but is insensitive to the strain. Moreover, influences of the twist angle gradient on symmetry breaking of the flat bands are also studied. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Root Cause Analysis of Anomalies in 5G RAN Using Graph Neural Network and Transformer

Authors: Antor Hasan, Conrado Boeira, Khaleda Papry, Yue Ju, Zhongwen Zhu, Israat Haque

Abstract: The emergence of 5G technology marks a significant milestone in developing telecommunication networks, enabling exciting new applications such as augmented reality and self-driving vehicles. However, these improvements bring an increased management complexity and a special concern in dealing with failures, as the applications 5G intends to support heavily rely on high network performance and low l… ▽ More The emergence of 5G technology marks a significant milestone in developing telecommunication networks, enabling exciting new applications such as augmented reality and self-driving vehicles. However, these improvements bring an increased management complexity and a special concern in dealing with failures, as the applications 5G intends to support heavily rely on high network performance and low latency. Thus, automatic self-healing solutions have become effective in dealing with this requirement, allowing a learning-based system to automatically detect anomalies and perform Root Cause Analysis (RCA). However, there are inherent challenges to the implementation of such intelligent systems. First, there is a lack of suitable data for anomaly detection and RCA, as labelled data for failure scenarios is uncommon. Secondly, current intelligent solutions are tailored to LTE networks and do not fully capture the spatio-temporal characteristics present in the data. Considering this, we utilize a calibrated simulator, Simu5G, and generate open-source data for normal and failure scenarios. Using this data, we propose Simba, a state-of-the-art approach for anomaly detection and root cause analysis in 5G Radio Access Networks (RANs). We leverage Graph Neural Networks to capture spatial relationships while a Transformer model is used to learn the temporal dependencies of the data. We implement a prototype of Simba and evaluate it over multiple failures. The outcomes are compared against existing solutions to confirm the superiority of Simba. △ Less

Submitted 21 June, 2024; originally announced June 2024.

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms

Authors: Xingrun Xing, Zheng Zhang, Ziyi Ni, Shitao Xiao, Yiming Ju, Siqi Fan, Yequan Wang, Jiajun Zhang, Guoqi Li

Abstract: Towards energy-efficient artificial intelligence similar to the human brain, the bio-inspired spiking neural networks (SNNs) have advantages of biological plausibility, event-driven sparsity, and binary activation. Recently, large-scale language models exhibit promising generalization capability, making it a valuable issue to explore more general spike-driven models. However, the binary spikes in… ▽ More Towards energy-efficient artificial intelligence similar to the human brain, the bio-inspired spiking neural networks (SNNs) have advantages of biological plausibility, event-driven sparsity, and binary activation. Recently, large-scale language models exhibit promising generalization capability, making it a valuable issue to explore more general spike-driven models. However, the binary spikes in existing SNNs fail to encode adequate semantic information, placing technological challenges for generalization. This work proposes the first fully spiking mechanism for general language tasks, including both discriminative and generative ones. Different from previous spikes with {0,1} levels, we propose a more general spike formulation with bi-directional, elastic amplitude, and elastic frequency encoding, while still maintaining the addition nature of SNNs. In a single time step, the spike is enhanced by direction and amplitude information; in spike frequency, a strategy to control spike firing rate is well designed. We plug this elastic bi-spiking mechanism in language modeling, named SpikeLM. It is the first time to handle general language tasks with fully spike-driven models, which achieve much higher accuracy than previously possible. SpikeLM also greatly bridges the performance gap between SNNs and ANNs in language modeling. Our code is available at https://github.com/Xingrun-Xing/SpikeLM. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Constructing a Galaxy Cluster Catalog in IllustrisTNG-300 using the Mulguisin Algorithm

Authors: Lael Shin, Jubee Sohn, Young Ju, Inkyu Park, Cristiano G. Sabiu

Abstract: We present a new simulated galaxy cluster catalog based on the IllustrisTNG simulation. We use the Mulguisin (MGS) algorithm to identify galaxy overdensities. Our cluster identification differs from the previous FoF cluster identification in two aspects; 1) we identify cluster halos based on the galaxy subhalos instead of unobservable dark matter particles, and 2) we use the MGS algorithm that sep… ▽ More We present a new simulated galaxy cluster catalog based on the IllustrisTNG simulation. We use the Mulguisin (MGS) algorithm to identify galaxy overdensities. Our cluster identification differs from the previous FoF cluster identification in two aspects; 1) we identify cluster halos based on the galaxy subhalos instead of unobservable dark matter particles, and 2) we use the MGS algorithm that separates galaxy overdensities hosted by massive galaxies. Our approach provides a cluster catalog constructed similar to the observed cluster catalogs using spectroscopic surveys. The MGS cluster catalog lists 303 halos with M$_{200} > 10^{14}$ M$_{\odot}$, including $\sim 10\%$ more than the FoF. The MGS catalog includes more systems because we separate some independent massive MGS cluster halos that are bundled into a single FoF algorithm. These independent MGS halos are apparently distinguishable in galaxy spatial distribution and the phase-space diagram. Because we constructed a refined cluster catalog that identifies local galaxy overdensities, we evaluate the effect of MGS clusters on the evolution of galaxies better than using the FoF cluster catalog. The MGS halo identification also enables effective identifications of merging clusters by selecting systems with neighboring galaxy overdensities. We thus highlight that the MGS cluster catalog is a useful tool for studying clusters in cosmological simulations and for comparing with the observed cluster samples. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 15 pages, 9 figures, 2 tables, Submitted to ApJ

Ultrafast Carrier Relaxation Dynamics in a Nodal-Line Semimetal PtSn$_4$

Authors: Tianyun Lin, Yongkang Ju, Haoyuan Zhong, Xiangyu Zeng, Xue Dong, Changhua Bao, Hongyun Zhang, Tian-Long Xia, Peizhe Tang, Shuyun Zhou

Abstract: Topological Dirac nodal-line semimetals host topologically nontrivial electronic structure with nodal-line crossings around the Fermi level, which could affect the photocarrier dynamics and lead to novel relaxation mechanisms. Herein, by using time- and angle-resolved photoemission spectroscopy, we reveal the previously-inaccessible linear dispersions of the bulk conduction bands above the Fermi l… ▽ More Topological Dirac nodal-line semimetals host topologically nontrivial electronic structure with nodal-line crossings around the Fermi level, which could affect the photocarrier dynamics and lead to novel relaxation mechanisms. Herein, by using time- and angle-resolved photoemission spectroscopy, we reveal the previously-inaccessible linear dispersions of the bulk conduction bands above the Fermi level in a Dirac nodal-line semimetal PtSn$_4$, as well as the momentum and temporal evolution of the gapless nodal lines. A surprisingly ultrafast relaxation dynamics within a few hundred femtoseconds is revealed for photoexcited carriers in the nodal line. Theoretical calculations suggest that such ultrafast carrier relaxation is attributed to the multichannel scatterings among the complex metallic bands of PtSn$_4$ via electron-phonon coupling. In addition, a unique dynamic relaxation mechanism contributed by the highly anisotropic Dirac nodal-line electronic structure is also identified. Our work provides a comprehensive understanding of the ultrafast carrier dynamics in a Dirac nodal-line semimetal. △ Less

Submitted 20 May, 2024; originally announced May 2024.

Faces that Speak: Jointly Synthesising Talking Face and Speech from Text

Authors: Youngjoon Jang, Ji-Hoon Kim, Junseok Ahn, Doyeop Kwak, Hong-Sun Yang, Yoon-Cheol Ju, Il-Hwan Kim, Byeong-Yeol Kim, Joon Son Chung

Abstract: The goal of this work is to simultaneously generate natural talking faces and speech outputs from text. We achieve this by integrating Talking Face Generation (TFG) and Text-to-Speech (TTS) systems into a unified framework. We address the main challenges of each task: (1) generating a range of head poses representative of real-world scenarios, and (2) ensuring voice consistency despite variations… ▽ More The goal of this work is to simultaneously generate natural talking faces and speech outputs from text. We achieve this by integrating Talking Face Generation (TFG) and Text-to-Speech (TTS) systems into a unified framework. We address the main challenges of each task: (1) generating a range of head poses representative of real-world scenarios, and (2) ensuring voice consistency despite variations in facial motion for the same identity. To tackle these issues, we introduce a motion sampler based on conditional flow matching, which is capable of high-quality motion code generation in an efficient way. Moreover, we introduce a novel conditioning method for the TTS system, which utilises motion-removed features from the TFG model to yield uniform speech outputs. Our extensive experiments demonstrate that our method effectively creates natural-looking talking faces and speech that accurately match the input text. To our knowledge, this is the first effort to build a multimodal synthesis system that can generalise to unseen identities. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Comments: CVPR 2024

Secure Inference for Vertically Partitioned Data Using Multiparty Homomorphic Encryption

Authors: Shuangyi Chen, Yue Ju, Zhongwen Zhu, Ashish Khisti

Abstract: We propose a secure inference protocol for a distributed setting involving a single server node and multiple client nodes. We assume that the observed data vector is partitioned across multiple client nodes while the deep learning model is located at the server node. Each client node is required to encrypt its portion of the data vector and transmit the resulting ciphertext to the server node. The… ▽ More We propose a secure inference protocol for a distributed setting involving a single server node and multiple client nodes. We assume that the observed data vector is partitioned across multiple client nodes while the deep learning model is located at the server node. Each client node is required to encrypt its portion of the data vector and transmit the resulting ciphertext to the server node. The server node is required to collect the ciphertexts and perform inference in the encrypted domain. We demonstrate an application of multi-party homomorphic encryption (MPHE) to satisfy these requirements. We propose a packing scheme, that enables the server to form the ciphertext of the complete data by aggregating the ciphertext of data subsets encrypted using MPHE. While our proposed protocol builds upon prior horizontal federated training protocol~\cite{sav2020poseidon}, we focus on the inference for vertically partitioned data and avoid the transmission of (encrypted) model weights from the server node to the client nodes. △ Less

Submitted 22 July, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

Comments: Accepted to ISIT 2024

Exposing Text-Image Inconsistency Using Diffusion Models

Authors: Mingzhen Huang, Shan Jia, Zhou Zhou, Yan Ju, Jialing Cai, Siwei Lyu

Abstract: In the battle against widespread online misinformation, a growing problem is text-image inconsistency, where images are misleadingly paired with texts with different intent or meaning. Existing classification-based methods for text-image inconsistency can identify contextual inconsistencies but fail to provide explainable justifications for their decisions that humans can understand. Although more… ▽ More In the battle against widespread online misinformation, a growing problem is text-image inconsistency, where images are misleadingly paired with texts with different intent or meaning. Existing classification-based methods for text-image inconsistency can identify contextual inconsistencies but fail to provide explainable justifications for their decisions that humans can understand. Although more nuanced, human evaluation is impractical at scale and susceptible to errors. To address these limitations, this study introduces D-TIIL (Diffusion-based Text-Image Inconsistency Localization), which employs text-to-image diffusion models to localize semantic inconsistencies in text and image pairs. These models, trained on large-scale datasets act as ``omniscient" agents that filter out irrelevant information and incorporate background knowledge to identify inconsistencies. In addition, D-TIIL uses text embeddings and modified image regions to visualize these inconsistencies. To evaluate D-TIIL's efficacy, we introduce a new TIIL dataset containing 14K consistent and inconsistent text-image pairs. Unlike existing datasets, TIIL enables assessment at the level of individual words and image regions and is carefully designed to represent various inconsistencies. D-TIIL offers a scalable and evidence-based approach to identifying and localizing text-image inconsistency, providing a robust framework for future research combating misinformation. △ Less

Submitted 27 April, 2024; originally announced April 2024.

DeepFake-O-Meter v2.0: An Open Platform for DeepFake Detection

Authors: Yan Ju, Chengzhe Sun, Shan Jia, Shuwei Hou, Zhaofeng Si, Soumyya Kanti Datta, Lipeng Ke, Riky Zhou, Anita Nikolich, Siwei Lyu

Abstract: Deepfakes, as AI-generated media, have increasingly threatened media integrity and personal privacy with realistic yet fake digital content. In this work, we introduce an open-source and user-friendly online platform, DeepFake-O-Meter v2.0, that integrates state-of-the-art methods for detecting Deepfake images, videos, and audio. Built upon DeepFake-O-Meter v1.0, we have made significant upgrades… ▽ More Deepfakes, as AI-generated media, have increasingly threatened media integrity and personal privacy with realistic yet fake digital content. In this work, we introduce an open-source and user-friendly online platform, DeepFake-O-Meter v2.0, that integrates state-of-the-art methods for detecting Deepfake images, videos, and audio. Built upon DeepFake-O-Meter v1.0, we have made significant upgrades and improvements in platform architecture design, including user interaction, detector integration, job balancing, and security management. The platform aims to offer everyday users a convenient service for analyzing DeepFake media using multiple state-of-the-art detection algorithms. It ensures secure and private delivery of the analysis results. Furthermore, it serves as an evaluation and benchmarking platform for researchers in digital media forensics to compare the performance of multiple algorithms on the same input. We have also conducted detailed usage analysis based on the collected data to gain deeper insights into our platform's statistics. This involves analyzing two-month trends in user activity and evaluating the processing efficiency of each detector. △ Less

Submitted 27 June, 2024; v1 submitted 19 April, 2024; originally announced April 2024.

A Calibrated and Automated Simulator for Innovations in 5G

Authors: Conrado Boeira, Antor Hasan, Khaleda Papry, Yue Ju, Zhongwen Zhu, Israat Haque

Abstract: The rise of 5G deployments has created the environment for many emerging technologies to flourish. Self-driving vehicles, Augmented and Virtual Reality, and remote operations are examples of applications that leverage 5G networks' support for extremely low latency, high bandwidth, and increased throughput. However, the complex architecture of 5G hinders innovation due to the lack of accessibility… ▽ More The rise of 5G deployments has created the environment for many emerging technologies to flourish. Self-driving vehicles, Augmented and Virtual Reality, and remote operations are examples of applications that leverage 5G networks' support for extremely low latency, high bandwidth, and increased throughput. However, the complex architecture of 5G hinders innovation due to the lack of accessibility to testbeds or realistic simulators with adequate 5G functionalities. Also, configuring and managing simulators are complex and time consuming. Finally, the lack of adequate representative data hinders the data-driven designs in 5G campaigns. Thus, we calibrated a system-level open-source simulator, Simu5G, following 3GPP guidelines to enable faster innovation in the 5G domain. Furthermore, we developed an API for automatic simulator configuration without knowing the underlying architectural details. Finally, we demonstrate the usage of the calibrated and automated simulator by developing an ML-based anomaly detection in a 5G Radio Access Network (RAN). △ Less

Submitted 16 April, 2024; originally announced April 2024.

RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network

Authors: Kai Luo, Yakun Ju, Lin Qi, Kaixuan Wang, Junyu Dong

Abstract: Predicting accurate normal maps of objects from two-dimensional images in regions of complex structure and spatial material variations is challenging using photometric stereo methods due to the influence of surface reflection properties caused by variations in object geometry and surface materials. To address this issue, we propose a photometric stereo network called a RMAFF-PSN that uses residual… ▽ More Predicting accurate normal maps of objects from two-dimensional images in regions of complex structure and spatial material variations is challenging using photometric stereo methods due to the influence of surface reflection properties caused by variations in object geometry and surface materials. To address this issue, we propose a photometric stereo network called a RMAFF-PSN that uses residual multiscale attentional feature fusion to handle the ``difficult'' regions of the object. Unlike previous approaches that only use stacked convolutional layers to extract deep features from the input image, our method integrates feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions, through shallow-deep stage feature extraction, double branching enhancement, and attention optimization. To test the network structure under real-world conditions, we propose a new real dataset called Simple PS data, which contains multiple objects with varying structures and materials. Experimental results on a publicly available benchmark dataset demonstrate that our method outperforms most existing calibrated photometric stereo methods for the same number of input images, especially in the case of highly non-convex object structures. Our method also obtains good results under sparse lighting conditions. △ Less

Submitted 14 April, 2024; v1 submitted 11 April, 2024; originally announced April 2024.

Comments: 17 pages,12 figures

Journal ref: Photonics 2023,10(5),548

Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics

Authors: Shan Jia, Reilin Lyu, Kangran Zhao, Yize Chen, Zhiyuan Yan, Yan Ju, Chuanbo Hu, Xin Li, Baoyuan Wu, Siwei Lyu

Abstract: DeepFakes, which refer to AI-generated media content, have become an increasing concern due to their use as a means for disinformation. Detecting DeepFakes is currently solved with programmed machine learning algorithms. In this work, we investigate the capabilities of multimodal large language models (LLMs) in DeepFake detection. We conducted qualitative and quantitative experiments to demonstrat… ▽ More DeepFakes, which refer to AI-generated media content, have become an increasing concern due to their use as a means for disinformation. Detecting DeepFakes is currently solved with programmed machine learning algorithms. In this work, we investigate the capabilities of multimodal large language models (LLMs) in DeepFake detection. We conducted qualitative and quantitative experiments to demonstrate multimodal LLMs and show that they can expose AI-generated images through careful experimental design and prompt engineering. This is interesting, considering that LLMs are not inherently tailored for media forensic tasks, and the process does not require programming. We discuss the limitations of multimodal LLMs for these tasks and suggest possible improvements. △ Less

Submitted 11 June, 2024; v1 submitted 20 March, 2024; originally announced March 2024.

Search for cosmic-ray boosted sub-MeV dark matter-electron scatterings in PandaX-4T

Authors: Xiaofeng Shang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou, Xiangdong Ji, Yonglin Ju, Chenxiang Li , et al. (67 additional authors not shown)

Abstract: We report the first search for the elastic scatterings between cosmic-ray boosted sub-MeV dark matter and electrons in the PandaX-4T liquid xenon experiment. Sub-MeV dark matter particles can be accelerated by scattering with electrons in the cosmic rays and produce detectable electron recoil signals in the detector. Using the commissioning data from PandaX-4T of 0.63~tonne$\cdot$year exposure, we… ▽ More We report the first search for the elastic scatterings between cosmic-ray boosted sub-MeV dark matter and electrons in the PandaX-4T liquid xenon experiment. Sub-MeV dark matter particles can be accelerated by scattering with electrons in the cosmic rays and produce detectable electron recoil signals in the detector. Using the commissioning data from PandaX-4T of 0.63~tonne$\cdot$year exposure, we set new constraints on DM-electron scattering cross sections for DM masses ranging from 10~eV/$c^2$ to 3~keV/$c^2$. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Comments: 6 pages, 3 figures

Detecting Neutrinos from Supernova Bursts in PandaX-4T

Authors: Binyu Pang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Junting Huang, Zhou Huang, Ruquan Hou , et al. (71 additional authors not shown)

Abstract: Neutrinos from core-collapse supernovae are essential for the understanding of neutrino physics and stellar evolution. The dual-phase xenon dark matter detectors can provide a way to track explosions of galactic supernovae by detecting neutrinos through coherent elastic neutrino-nucleus scatterings. In this study, a variation of progenitor masses as well as explosion models are assumed to predict… ▽ More Neutrinos from core-collapse supernovae are essential for the understanding of neutrino physics and stellar evolution. The dual-phase xenon dark matter detectors can provide a way to track explosions of galactic supernovae by detecting neutrinos through coherent elastic neutrino-nucleus scatterings. In this study, a variation of progenitor masses as well as explosion models are assumed to predict the neutrino fluxes and spectra, which result in the number of expected neutrino events ranging from 6.6 to 13.7 at a distance of 10 kpc over a 10-second duration with negligible backgrounds at PandaX-4T. Two specialized triggering alarms for monitoring supernova burst neutrinos are built. The efficiency of detecting supernova explosions at various distances in the Milky Way is estimated. These alarms will be implemented in the real-time supernova monitoring system at PandaX-4T in the near future, providing the astronomical communities with supernova early warnings. △ Less

Submitted 10 March, 2024; originally announced March 2024.

Comments: 9 pages,6 figures

Signal Response Model in PandaX-4T

Authors: Yunyang Luo, Zihao Bo, Shibo Zhang, Abdusalam Abdukerim, Chen Cheng, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Zhou Huang , et al. (66 additional authors not shown)

Abstract: PandaX-4T experiment is a deep-underground dark matter direct search experiment that employs a dual-phase time projection chamber with a sensitive volume containing 3.7 tonne of liquid xenon. The detector of PandaX-4T is capable of simultaneously collecting the primary scintillation and ionization signals, utilizing their ratio to discriminate dark matter signals from background sources such as ga… ▽ More PandaX-4T experiment is a deep-underground dark matter direct search experiment that employs a dual-phase time projection chamber with a sensitive volume containing 3.7 tonne of liquid xenon. The detector of PandaX-4T is capable of simultaneously collecting the primary scintillation and ionization signals, utilizing their ratio to discriminate dark matter signals from background sources such as gamma rays and beta particles. The signal response model plays a crucial role in interpreting the data obtained by PandaX-4T. It describes the conversion from the deposited energy by dark matter interactions to the detectable signals within the detector. The signal response model is utilized in various PandaX-4T results. This work provides a comprehensive description of the procedures involved in constructing and parameter-fitting the signal response model for the energy range of approximately 1 keV to 25 keV for electronic recoils and 6 keV to 90 keV for nuclear recoils. It also covers the signal reconstruction, selection, and correction methods, which are crucial components integrated into the signal response model. △ Less

Submitted 14 June, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Preserving Fairness Generalization in Deepfake Detection

Authors: Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, Shu Hu

Abstract: Although effective deepfake detection models have been developed in recent years, recent studies have revealed that these models can result in unfair performance disparities among demographic groups, such as race and gender. This can lead to particular groups facing unfair targeting or exclusion from detection, potentially allowing misclassified deepfakes to manipulate public opinion and undermine… ▽ More Although effective deepfake detection models have been developed in recent years, recent studies have revealed that these models can result in unfair performance disparities among demographic groups, such as race and gender. This can lead to particular groups facing unfair targeting or exclusion from detection, potentially allowing misclassified deepfakes to manipulate public opinion and undermine trust in the model. The existing method for addressing this problem is providing a fair loss function. It shows good fairness performance for intra-domain evaluation but does not maintain fairness for cross-domain testing. This highlights the significance of fairness generalization in the fight against deepfakes. In this work, we propose the first method to address the fairness generalization problem in deepfake detection by simultaneously considering features, loss, and optimization aspects. Our method employs disentanglement learning to extract demographic and domain-agnostic forgery features, fusing them to encourage fair learning across a flattened loss landscape. Extensive experiments on prominent deepfake datasets demonstrate our method's effectiveness, surpassing state-of-the-art approaches in preserving fairness during cross-domain deepfake detection. The code is available at https://github.com/Purdue-M2/Fairness-Generalization △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: Accepted by The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)

Explaining generative diffusion models via visual analysis for interpretable decision-making process

Authors: Ji-Hoon Park, Yeong-Joon Ju, Seong-Whan Lee

Abstract: Diffusion models have demonstrated remarkable performance in generation tasks. Nevertheless, explaining the diffusion process remains challenging due to it being a sequence of denoising noisy images that are difficult for experts to interpret. To address this issue, we propose the three research questions to interpret the diffusion process from the perspective of the visual concepts generated by t… ▽ More Diffusion models have demonstrated remarkable performance in generation tasks. Nevertheless, explaining the diffusion process remains challenging due to it being a sequence of denoising noisy images that are difficult for experts to interpret. To address this issue, we propose the three research questions to interpret the diffusion process from the perspective of the visual concepts generated by the model and the region where the model attends in each time step. We devise tools for visualizing the diffusion process and answering the aforementioned research questions to render the diffusion process human-understandable. We show how the output is progressively generated in the diffusion process by explaining the level of denoising and highlighting relationships to foundational visual concepts at each time step through the results of experiments with various visual analyses using the tools. Throughout the training of the diffusion model, the model learns diverse visual concepts corresponding to each time-step, enabling the model to predict varying levels of visual concepts at different stages. We substantiate our tools using Area Under Cover (AUC) score, correlation quantification, and cross-attention mapping. Our findings provide insights into the diffusion process and pave the way for further research into explainable diffusion mechanisms. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Comments: 22 pages, published in Expert Systems with Applications

MSC Class: 68T01

Journal ref: Expert Systems with Applications 248 (2024) 123231

PandaX-xT: a Multi-ten-tonne Liquid Xenon Observatory at the China Jinping Underground Laboratory

Authors: PandaX Collaboration, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Linhui Gu, Xunan Guo, Xuyuan Guo, Zhichao Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou , et al. (68 additional authors not shown)

Abstract: We propose a major upgrade to the existing PandaX-4T experiment in the China Jinping Underground Laboratory. The new experiment, PandaX-xT, will be a multi-ten-tonne liquid xenon, ultra-low background, and general-purpose observatory. The full-scaled PandaX-xT contains a 43-tonne liquid xenon active target. Such an experiment will significantly advance our fundamental understanding of particle phy… ▽ More We propose a major upgrade to the existing PandaX-4T experiment in the China Jinping Underground Laboratory. The new experiment, PandaX-xT, will be a multi-ten-tonne liquid xenon, ultra-low background, and general-purpose observatory. The full-scaled PandaX-xT contains a 43-tonne liquid xenon active target. Such an experiment will significantly advance our fundamental understanding of particle physics and astrophysics. The sensitivity of dark matter direct detection will be improved by nearly two orders of magnitude compared to the current best limits, approaching the so-called "neutrino floor" for a dark matter mass above 10 GeV/$c^2$, providing a decisive test to the Weakly Interacting Massive Particle paradigm. By searching for the neutrinoless double beta decay of $^{136}$Xe isotope in the detector, the effective Majorana neutrino mass can be measured to a [10 -- 41] meV/$c^2$ sensitivity, providing a key test to the Dirac/Majorana nature of neutrino s. Astrophysical neutrinos and other ultra-rare interactions can also be measured and searched for with an unprecedented background level, opening up new windows of discovery. Depending on the findings, PandaX-xT will seek the next stage upgrade utilizing isotopic separation on natural xenon. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Multi-view MidiVAE: Fusing Track- and Bar-view Representations for Long Multi-track Symbolic Music Generation

Authors: Zhiwei Lin, Jun Chen, Boshi Tang, Binzhu Sha, Jing Yang, Yaolong Ju, Fan Fan, Shiyin Kang, Zhiyong Wu, Helen Meng

Abstract: Variational Autoencoders (VAEs) constitute a crucial component of neural symbolic music generation, among which some works have yielded outstanding results and attracted considerable attention. Nevertheless, previous VAEs still encounter issues with overly long feature sequences and generated results lack contextual coherence, thus the challenge of modeling long multi-track symbolic music still re… ▽ More Variational Autoencoders (VAEs) constitute a crucial component of neural symbolic music generation, among which some works have yielded outstanding results and attracted considerable attention. Nevertheless, previous VAEs still encounter issues with overly long feature sequences and generated results lack contextual coherence, thus the challenge of modeling long multi-track symbolic music still remains unaddressed. To this end, we propose Multi-view MidiVAE, as one of the pioneers in VAE methods that effectively model and generate long multi-track symbolic music. The Multi-view MidiVAE utilizes the two-dimensional (2-D) representation, OctupleMIDI, to capture relationships among notes while reducing the feature sequences length. Moreover, we focus on instrumental characteristics and harmony as well as global and local information about the musical composition by employing a hybrid variational encoding-decoding strategy to integrate both Track- and Bar-view MidiVAE features. Objective and subjective experimental results on the CocoChorales dataset demonstrate that, compared to the baseline, Multi-view MidiVAE exhibits significant improvements in terms of modeling long multi-track symbolic music. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Comments: Accepted by ICASSP 2024

Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation

Authors: Yuanchen Ju, Kaizhe Hu, Guowei Zhang, Gu Zhang, Mingrun Jiang, Huazhe Xu

Abstract: Enabling robotic manipulation that generalizes to out-of-distribution scenes is a crucial step toward open-world embodied intelligence. For human beings, this ability is rooted in the understanding of semantic correspondence among objects, which naturally transfers the interaction experience of familiar objects to novel ones. Although robots lack such a reservoir of interaction experience, the vas… ▽ More Enabling robotic manipulation that generalizes to out-of-distribution scenes is a crucial step toward open-world embodied intelligence. For human beings, this ability is rooted in the understanding of semantic correspondence among objects, which naturally transfers the interaction experience of familiar objects to novel ones. Although robots lack such a reservoir of interaction experience, the vast availability of human videos on the Internet may serve as a valuable resource, from which we extract an affordance memory including the contact points. Inspired by the natural way humans think, we propose Robo-ABC: when confronted with unfamiliar objects that require generalization, the robot can acquire affordance by retrieving objects that share visual or semantic similarities from the affordance memory. The next step is to map the contact points of the retrieved objects to the new object. While establishing this correspondence may present formidable challenges at first glance, recent research finds it naturally arises from pre-trained diffusion models, enabling affordance mapping even across disparate object categories. Through the Robo-ABC framework, robots may generalize to manipulate out-of-category objects in a zero-shot manner without any manual annotation, additional training, part segmentation, pre-coded knowledge, or viewpoint restrictions. Quantitatively, Robo-ABC significantly enhances the accuracy of visual affordance retrieval by a large margin of 31.6% compared to state-of-the-art (SOTA) end-to-end affordance models. We also conduct real-world experiments of cross-category object-grasping tasks. Robo-ABC achieved a success rate of 85.7%, proving its capacity for real-world tasks. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Measurement of Solar $pp$ Neutrino Flux using Electron Recoil Data from PandaX-4T Commissioning Run

Authors: PandaX Collaboration, Xiaoying Lu, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (67 additional authors not shown)

Abstract: The proton-proton ($pp$) fusion chain dominates the neutrino production from the Sun. The uncertainty of the predicted $pp$ neutrino flux is at the sub-percent level, whereas that of the best measurement is $\mathcal{O}(10\%)$. In this paper, we present the first result to measure the solar $pp$ neutrinos in the electron recoil energy range from 24 to 144 keV, using the PandaX-4T commissioning dat… ▽ More The proton-proton ($pp$) fusion chain dominates the neutrino production from the Sun. The uncertainty of the predicted $pp$ neutrino flux is at the sub-percent level, whereas that of the best measurement is $\mathcal{O}(10\%)$. In this paper, we present the first result to measure the solar $pp$ neutrinos in the electron recoil energy range from 24 to 144 keV, using the PandaX-4T commissioning data with 0.63 tonne$\times$year exposure. The $pp$ neutrino flux is determined to be $(8.0 \pm 3.9 \,{\rm{(stat)}} \pm 10.0 \,{\rm{(syst)}} )\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$, consistent with Standard Solar Model and existing measurements, corresponding to a flux upper limit of $23.3\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$ at 90\% C.L.. △ Less

Submitted 2 July, 2024; v1 submitted 13 January, 2024; originally announced January 2024.

Comments: 6 pages, 5 figures

Radon Removal Commissioning of the PandaX-4T Cryogenic Distillation System

Authors: Xiangyi Cui, Zhou Wang, Jiafu Li, Shuaijie Li, Lin Si, Yonglin Ju, Wenbo Ma, Jianglai Liu, Li Zhao, Xiangdong Ji, Rui Yan, Haidong Sha, Peiyao Huang, Xiuli Wang, Huaxuan Liu

Abstract: The PandaX-4T distillation system, designed for the removal of krypton and radon from xenon, is evaluated for its radon removal efficiency using a $^{222}$Rn source during the online distillation process. The PandaX-4T dark matter detector is employed to monitor the temporal evolution of radon activity. To determine the radon reduction factor, the experimental data of radon atoms introduced into a… ▽ More The PandaX-4T distillation system, designed for the removal of krypton and radon from xenon, is evaluated for its radon removal efficiency using a $^{222}$Rn source during the online distillation process. The PandaX-4T dark matter detector is employed to monitor the temporal evolution of radon activity. To determine the radon reduction factor, the experimental data of radon atoms introduced into and bypassed the distillation system is compared. The results indicate that the PandaX-4T distillation system achieves a radon reduction factor exceeding 190 at the flow rate of 10 slpm and the reflux ratio of 1.44. Gas-only online distillation process of a flow rate of 20 slpm is also conducted without observing significant reduction of radon levels in the detector. This observation suggests that the migration flow of radon atoms from the liquid phase to the gas phase is limited, and the flow rate of gas circulation and duration of the process are insignificant compared to the total xenon mass of 5.6 tons in the detector. This study provides the experimental data to support the efficient removal of radon at $\sim$Bq level using the PandaX-4T distillation system, which is the prerequisite of the radon background control in the detector. The further operation with higher flow rate will be applied for the upcoming science run in PandaX-4T. △ Less

Submitted 19 April, 2024; v1 submitted 3 January, 2024; originally announced January 2024.

Comments: 14 pages, 9 figures

Searching for Two-Neutrino and Neutrinoless Double Beta Decay of $^{134}$Xe with the PandaX-4T Experiment

Authors: PandaX Collaboration, Xiyu Yan, Zhaokan Cheng, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Junting Huang, Zhou Huang , et al. (72 additional authors not shown)

Abstract: $^{134}$Xe is a candidate isotope for neutrinoless double beta decay~($0νββ$) search. In addition, the two-neutrino case ($2νββ$) allowed by the Standard Model of particle physics has not yet been observed. Utilizing the 10.4% of $^{134}$Xe in the natural xenon in the PandaX-4T detector and its first 94.9-day exposure, we have established the most stringent constraints on $2νββ$ and $0νββ$ of $^{1… ▽ More $^{134}$Xe is a candidate isotope for neutrinoless double beta decay~($0νββ$) search. In addition, the two-neutrino case ($2νββ$) allowed by the Standard Model of particle physics has not yet been observed. Utilizing the 10.4% of $^{134}$Xe in the natural xenon in the PandaX-4T detector and its first 94.9-day exposure, we have established the most stringent constraints on $2νββ$ and $0νββ$ of $^{134}$Xe half-lives, with limits of $2.8\times10^{22}$ yr and $3.0\times10^{23}$ yr at 90% confidence level, respectively. The $2νββ$ ($0νββ$) limit surpasses the previously reported best result by a factor of 32 (2.7), highlighting the potential of large monolithic natural xenon detectors. △ Less

Submitted 28 April, 2024; v1 submitted 25 December, 2023; originally announced December 2023.

Journal ref: Phys.Rev.Lett. 132 (2024) 15, 152502

Waveform Simulation in PandaX-4T

Authors: Jiafu Li, Abdusalam Abdukerim, Chen Cheng, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Zhou Huang, Ruquan Hou , et al. (66 additional authors not shown)

Abstract: Signal reconstruction through software processing is a crucial component of the background and signal models in the PandaX-4T experiment, which is a multi-tonne dark matter direct search experiment. The accuracy of signal reconstruction is influenced by various detector artifacts, including noise, dark count of photomultiplier, impurity photoionization in the detector, and other relevant considera… ▽ More Signal reconstruction through software processing is a crucial component of the background and signal models in the PandaX-4T experiment, which is a multi-tonne dark matter direct search experiment. The accuracy of signal reconstruction is influenced by various detector artifacts, including noise, dark count of photomultiplier, impurity photoionization in the detector, and other relevant considerations. In this study, we present a detailed description of a semi-data-driven approach designed to simulate the signal waveform. This work provides a reliable model for the efficiency and bias of the signal reconstruction in the data analysis of PandaX-4T. By comparing critical variables which relate to the temporal shape and hit pattern of the signals, we demonstrate a good agreement between the simulation and data. △ Less

Submitted 21 May, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

Journal ref: Chin. Phys. C 48, no.7,073001 (2024)

How to Do Machine Learning with Small Data? -- A Review from an Industrial Perspective

Authors: Ivan Kraljevski, Yong Chul Ju, Dmitrij Ivanov, Constanze Tschöpe, Matthias Wolff

Abstract: Artificial intelligence experienced a technological breakthrough in science, industry, and everyday life in the recent few decades. The advancements can be credited to the ever-increasing availability and miniaturization of computational resources that resulted in exponential data growth. However, because of the insufficient amount of data in some cases, employing machine learning in solving compl… ▽ More Artificial intelligence experienced a technological breakthrough in science, industry, and everyday life in the recent few decades. The advancements can be credited to the ever-increasing availability and miniaturization of computational resources that resulted in exponential data growth. However, because of the insufficient amount of data in some cases, employing machine learning in solving complex tasks is not straightforward or even possible. As a result, machine learning with small data experiences rising importance in data science and application in several fields. The authors focus on interpreting the general term of "small data" and their engineering and industrial application role. They give a brief overview of the most important industrial applications of machine learning and small data. Small data is defined in terms of various characteristics compared to big data, and a machine learning formalism was introduced. Five critical challenges of machine learning with small data in industrial applications are presented: unlabeled data, imbalanced data, missing data, insufficient data, and rare events. Based on those definitions, an overview of the considerations in domain representation and data acquisition is given along with a taxonomy of machine learning approaches in the context of small data. △ Less

Submitted 13 November, 2023; originally announced November 2023.

A positivity-preserving numerical method for a thin liquid film on a vertical cylindrical fiber

Authors: Bohyun Kim, Hangjie Ji, Andrea L. Bertozzi, Abolfazl Sadeghpour, Y. Sungtaek Ju

Abstract: When a thin liquid film flows down on a vertical fiber, one can observe the complex and captivating interfacial dynamics of an unsteady flow. Such dynamics are applicable in various fluid experiments due to their high surface area-to-volume ratio. Recent studies verified that when the flow undergoes regime transitions, the magnitude of the film thickness changes dramatically, making numerical simu… ▽ More When a thin liquid film flows down on a vertical fiber, one can observe the complex and captivating interfacial dynamics of an unsteady flow. Such dynamics are applicable in various fluid experiments due to their high surface area-to-volume ratio. Recent studies verified that when the flow undergoes regime transitions, the magnitude of the film thickness changes dramatically, making numerical simulations challenging. In this paper, we present a computationally efficient numerical method that can maintain the positivity of the film thickness as well as conserve the volume of the fluid under the coarse mesh setting. A series of comparisons to laboratory experiments and previously proposed numerical methods supports the validity of our numerical method. We also prove that our method is second-order consistent in space and satisfies the entropy estimate. △ Less

Submitted 16 October, 2023; originally announced October 2023.

NeuroInspect: Interpretable Neuron-based Debugging Framework through Class-conditional Visualizations

Authors: Yeong-Joon Ju, Ji-Hoon Park, Seong-Whan Lee

Abstract: Despite deep learning (DL) has achieved remarkable progress in various domains, the DL models are still prone to making mistakes. This issue necessitates effective debugging tools for DL practitioners to interpret the decision-making process within the networks. However, existing debugging methods often demand extra data or adjustments to the decision process, limiting their applicability. To tack… ▽ More Despite deep learning (DL) has achieved remarkable progress in various domains, the DL models are still prone to making mistakes. This issue necessitates effective debugging tools for DL practitioners to interpret the decision-making process within the networks. However, existing debugging methods often demand extra data or adjustments to the decision process, limiting their applicability. To tackle this problem, we present NeuroInspect, an interpretable neuron-based debugging framework with three key stages: counterfactual explanations, feature visualizations, and false correlation mitigation. Our debugging framework first pinpoints neurons responsible for mistakes in the network and then visualizes features embedded in the neurons to be human-interpretable. To provide these explanations, we introduce CLIP-Illusion, a novel feature visualization method that generates images representing features conditioned on classes to examine the connection between neurons and the decision layer. We alleviate convoluted explanations of the conventional visualization approach by employing class information, thereby isolating mixed properties. This process offers more human-interpretable explanations for model errors without altering the trained network or requiring additional data. Furthermore, our framework mitigates false correlations learned from a dataset under a stochastic perspective, modifying decisions for the neurons considered as the main causes. We validate the effectiveness of our framework by addressing false correlations and improving inferences for classes with the worst performance in real-world settings. Moreover, we demonstrate that NeuroInspect helps debug the mistakes of DL models through evaluation for human understanding. The code is openly available at https://github.com/yeongjoonJu/NeuroInspect. △ Less

Submitted 17 October, 2023; v1 submitted 11 October, 2023; originally announced October 2023.

KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models

Authors: Yiming Ju, Xingrun Xing, Zhixiong Zeng

Abstract: Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches in changing factual knowledge stored in the Language models. However, there is a lack of research on whether present locating methods can pinpoint the exact parameters embedding the desired knowledge. Moreover, although many researchers have questioned the validity of locality hypothesis of factual knowledge, no method i… ▽ More Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches in changing factual knowledge stored in the Language models. However, there is a lack of research on whether present locating methods can pinpoint the exact parameters embedding the desired knowledge. Moreover, although many researchers have questioned the validity of locality hypothesis of factual knowledge, no method is provided to test the a hypothesis for more in-depth discussion and research. Therefore, we introduce KLoB, a benchmark examining three essential properties that a reliable knowledge locating method should satisfy. KLoB can serve as a benchmark for evaluating existing locating methods in language models, and can contributes a method to reassessing the validity of locality hypothesis of factual knowledge. KLoB is publicly available at an anonymous GitHub: \url{https://github.com/anon6662/KLoB}. △ Less

Submitted 26 August, 2024; v1 submitted 28 September, 2023; originally announced September 2023.

Exposing Image Splicing Traces in Scientific Publications via Uncertainty-guided Refinement

Authors: Xun Lin, Wenzhong Tang, Haoran Wang, Yizhong Liu, Yakun Ju, Shuai Wang, Zitong Yu

Abstract: Recently, a surge in scientific publications suspected of image manipulation has led to numerous retractions, bringing the issue of image integrity into sharp focus. Although research on forensic detectors for image plagiarism and image synthesis exists, the detection of image splicing traces in scientific publications remains unexplored. Compared to image duplication and synthesis, image splicing… ▽ More Recently, a surge in scientific publications suspected of image manipulation has led to numerous retractions, bringing the issue of image integrity into sharp focus. Although research on forensic detectors for image plagiarism and image synthesis exists, the detection of image splicing traces in scientific publications remains unexplored. Compared to image duplication and synthesis, image splicing detection is more challenging due to the lack of reference images and the typically small tampered areas. Furthermore, disruptive factors in scientific images, such as artifacts from digital compression, abnormal patterns, and noise from physical operations, present misleading features like splicing traces, significantly increasing the difficulty of this task. Moreover, the scarcity of high-quality datasets of spliced scientific images limits potential advancements. In this work, we propose an Uncertainty-guided Refinement Network (URN) to mitigate the impact of these disruptive factors. Our URN can explicitly suppress the propagation of unreliable information flow caused by disruptive factors between regions, thus obtaining robust splicing features. Additionally, the URN is designed to concentrate improvements in uncertain prediction areas during the decoding phase. We also construct a dataset for image splicing detection (SciSp) containing 1,290 spliced images. Compared to existing datasets, SciSp includes the largest number of spliced images and the most diverse sources. Comprehensive experiments conducted on three benchmark datasets demonstrate the superiority of our approach. We also validate the URN's generalisability in resisting cross-dataset domain shifts and its robustness against various post-processing techniques, including advanced deep-learning-based inpainting. △ Less

Submitted 18 April, 2024; v1 submitted 28 September, 2023; originally announced September 2023.

Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer

Authors: Anwei Luo, Rizhao Cai, Chenqi Kong, Yakun Ju, Xiangui Kang, Jiwu Huang, Alex C. Kot

Abstract: With the rapid progress of generative models, the current challenge in face forgery detection is how to effectively detect realistic manipulated faces from different unseen domains. Though previous studies show that pre-trained Vision Transformer (ViT) based models can achieve some promising results after fully fine-tuning on the Deepfake dataset, their generalization performances are still unsati… ▽ More With the rapid progress of generative models, the current challenge in face forgery detection is how to effectively detect realistic manipulated faces from different unseen domains. Though previous studies show that pre-trained Vision Transformer (ViT) based models can achieve some promising results after fully fine-tuning on the Deepfake dataset, their generalization performances are still unsatisfactory. One possible reason is that fully fine-tuned ViT-based models may disrupt the pre-trained features [1, 2] and overfit to some data-specific patterns [3]. To alleviate this issue, we present a \textbf{F}orgery-aware \textbf{A}daptive \textbf{Vi}sion \textbf{T}ransformer (FA-ViT) under the adaptive learning paradigm, where the parameters in the pre-trained ViT are kept fixed while the designed adaptive modules are optimized to capture forgery features. Specifically, a global adaptive module is designed to model long-range interactions among input tokens, which takes advantage of self-attention mechanism to mine global forgery clues. To further explore essential local forgery clues, a local adaptive module is proposed to expose local inconsistencies by enhancing the local contextual association. In addition, we introduce a fine-grained adaptive learning module that emphasizes the common compact representation of genuine faces through relationship learning in fine-grained pairs, driving these proposed adaptive modules to be aware of fine-grained forgery-aware information. Extensive experiments demonstrate that our FA-ViT achieves state-of-the-arts results in the cross-dataset evaluation, and enhances the robustness against unseen perturbations. Particularly, FA-ViT achieves 93.83\% and 78.32\% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation. The code and trained model have been released at: https://github.com/LoveSiameseCat/FAViT. △ Less

Submitted 21 August, 2024; v1 submitted 20 September, 2023; originally announced September 2023.

GAN-based Algorithm for Efficient Image Inpainting

Authors: Zhengyang Han, Zehao Jiang, Yuan Ju

Abstract: Global pandemic due to the spread of COVID-19 has post challenges in a new dimension on facial recognition, where people start to wear masks. Under such condition, the authors consider utilizing machine learning in image inpainting to tackle the problem, by complete the possible face that is originally covered in mask. In particular, autoencoder has great potential on retaining important, general… ▽ More Global pandemic due to the spread of COVID-19 has post challenges in a new dimension on facial recognition, where people start to wear masks. Under such condition, the authors consider utilizing machine learning in image inpainting to tackle the problem, by complete the possible face that is originally covered in mask. In particular, autoencoder has great potential on retaining important, general features of the image as well as the generative power of the generative adversarial network (GAN). The authors implement a combination of the two models, context encoders and explain how it combines the power of the two models and train the model with 50,000 images of influencers faces and yields a solid result that still contains space for improvements. Furthermore, the authors discuss some shortcomings with the model, their possible improvements, as well as some area of study for future investigation for applicative perspective, as well as directions to further enhance and refine the model. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 6 pages, 3 figures

MSC Class: 68U10

Journal ref: The 3rd International Conference on Artificial Intelligence and Computer Engineering(ICAICE 2022)

Unsupervised Text Style Transfer with Deep Generative Models

Authors: Zhongtao Jiang, Yuanzhe Zhang, Yiming Ju, Kang Liu

Abstract: We present a general framework for unsupervised text style transfer with deep generative models. The framework models each sentence-label pair in the non-parallel corpus as partially observed from a complete quadruplet which additionally contains two latent codes representing the content and style, respectively. These codes are learned by exploiting dependencies inside the observed data. Then a se… ▽ More We present a general framework for unsupervised text style transfer with deep generative models. The framework models each sentence-label pair in the non-parallel corpus as partially observed from a complete quadruplet which additionally contains two latent codes representing the content and style, respectively. These codes are learned by exploiting dependencies inside the observed data. Then a sentence is transferred by manipulating them. Our framework is able to unify previous embedding and prototype methods as two special forms. It also provides a principled perspective to explain previously proposed techniques in the field such as aligned encoder and adversarial training. We further conduct experiments on three benchmarks. Both automatic and human evaluation results show that our methods achieve better or competitive results compared to several strong baselines. △ Less

Submitted 31 August, 2023; originally announced August 2023.

Stripe charge order driven manipulation of Majorana bound states in 2M-WS2 topological superconductor

Authors: Xuemin Fan, Xiaoqi Sun, Penghao Zhu, Yuqiang Fang, Yongkang Ju, Yonghao Yuan, Fuqiang Huang, Taylor L. Hughes, Peizhe Tang, Qi-Kun Xue, Wei Li

Abstract: Majorana bound states (MBSs) are building blocks for topological quantum computing. They can be generated via the combination of electronic topology and superconductivity. To achieve logic operations via Majorana braiding, positional control of the MBS must be established. To this end, exotic co-existing phases or collective modes in an intrinsic topological superconductor can provide a tuning kno… ▽ More Majorana bound states (MBSs) are building blocks for topological quantum computing. They can be generated via the combination of electronic topology and superconductivity. To achieve logic operations via Majorana braiding, positional control of the MBS must be established. To this end, exotic co-existing phases or collective modes in an intrinsic topological superconductor can provide a tuning knob to manipulate the MBS. Here we report the observation of a striped surface charge order coexisting with superconductivity and its controllable tuning of the MBS in the topological superconductor 2M-WS2 using low-temperature scanning tunneling microscopy. By applying an out-of-plane magnetic field, we observe that MBS is absent in vortices in the region with strong stripe order. This is in contrast to adjacent underlaying layers without charge order where vortex-bound MBSs are observed. Via theoretical simulations, we show that the surface stripe order does not destroy the bulk topology, but it can effectively modify the spatial distribution of MBS, i.e., it pushes them downward away from the 2M-WS2 surface. Our findings demonstrate that the interplay of charge order and topological superconductivity can be used to manipulate the positions of the MBS, and to explore of new states of matter. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Comments: 20 pages, 4 figures

Search for Dark-Matter-Nucleon Interactions with a Dark Mediator in PandaX-4T

Authors: Di Huang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Yanlin Huang, Zhou Huang, Ruquan Hou, Xiangdong Ji , et al. (70 additional authors not shown)

Abstract: We report results of a search for dark-matter-nucleon interactions via a dark mediator using optimized low-energy data from the PandaX-4T liquid xenon experiment. With the ionization-signal-only data and utilizing the Migdal effect, we set the most stringent limits on the cross section for dark matter masses ranging from 30~$\rm{MeV/c^2}$ to 2~$\rm{GeV/c^2}$. Under the assumption that the dark med… ▽ More We report results of a search for dark-matter-nucleon interactions via a dark mediator using optimized low-energy data from the PandaX-4T liquid xenon experiment. With the ionization-signal-only data and utilizing the Migdal effect, we set the most stringent limits on the cross section for dark matter masses ranging from 30~$\rm{MeV/c^2}$ to 2~$\rm{GeV/c^2}$. Under the assumption that the dark mediator is a dark photon that decays into scalar dark matter pairs in the early Universe, we rule out significant parameter space of such thermal relic dark-matter model. △ Less

Submitted 18 December, 2023; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: 6 pages, 4 figures

Journal ref: Phys. Rev. Lett. 131, 191002 (2023)

EdgeMatrix: A Resource-Redefined Scheduling Framework for SLA-Guaranteed Multi-Tier Edge-Cloud Computing Systems

Authors: Shihao Shen, Yuanming Ren, Yanli Ju, Xiaofei Wang, Wenyu Wang, Victor C. M. Leung

Abstract: With the development of networking technology, the computing system has evolved towards the multi-tier paradigm gradually. However, challenges, such as multi-resource heterogeneity of devices, resource competition of services, and networked system dynamics, make it difficult to guarantee service-level agreement (SLA) for the applications. In this paper, we propose a multi-tier edge-cloud computing… ▽ More With the development of networking technology, the computing system has evolved towards the multi-tier paradigm gradually. However, challenges, such as multi-resource heterogeneity of devices, resource competition of services, and networked system dynamics, make it difficult to guarantee service-level agreement (SLA) for the applications. In this paper, we propose a multi-tier edge-cloud computing framework, EdgeMatrix, to maximize the throughput of the system while guaranteeing different SLA priorities. First, in order to reduce the impact of physical resource heterogeneity, EdgeMatrix introduces the Networked Multi-agent Actor-Critic (NMAC) algorithm to re-define physical resources with the same quality of service as logically isolated resource units and combinations, i.e., cells and channels. In addition, a multi-task mechanism is designed in EdgeMatrix to solve the problem of Joint Service Orchestration and Request Dispatch (JSORD) for matching the requests and services, which can significantly reduce the optimization runtime. For integrating above two algorithms, EdgeMatrix is designed with two time-scales, i.e., coordinating services and resources at the larger time-scale, and dispatching requests at the smaller time-scale. Realistic trace-based experiments proves that the overall throughput of EdgeMatrix is 36.7% better than that of the closest baseline, while the SLA priorities are guaranteed still. △ Less

Submitted 1 August, 2023; originally announced August 2023.

Comments: JSAC. arXiv admin note: substantial text overlap with arXiv:2203.10470

ArrayBot: Reinforcement Learning for Generalizable Distributed Manipulation through Touch

Authors: Zhengrong Xue, Han Zhang, Jingwen Cheng, Zhengmao He, Yuanchen Ju, Changyi Lin, Gu Zhang, Huazhe Xu

Abstract: We present ArrayBot, a distributed manipulation system consisting of a $16 \times 16$ array of vertically sliding pillars integrated with tactile sensors, which can simultaneously support, perceive, and manipulate the tabletop objects. Towards generalizable distributed manipulation, we leverage reinforcement learning (RL) algorithms for the automatic discovery of control policies. In the face of t… ▽ More We present ArrayBot, a distributed manipulation system consisting of a $16 \times 16$ array of vertically sliding pillars integrated with tactile sensors, which can simultaneously support, perceive, and manipulate the tabletop objects. Towards generalizable distributed manipulation, we leverage reinforcement learning (RL) algorithms for the automatic discovery of control policies. In the face of the massively redundant actions, we propose to reshape the action space by considering the spatially local action patch and the low-frequency actions in the frequency domain. With this reshaped action space, we train RL agents that can relocate diverse objects through tactile observations only. Surprisingly, we find that the discovered policy can not only generalize to unseen object shapes in the simulator but also transfer to the physical robot without any domain randomization. Leveraging the deployed policy, we present abundant real-world manipulation tasks, illustrating the vast potential of RL on ArrayBot for distributed manipulation. △ Less

Submitted 29 June, 2023; originally announced June 2023.

Improving Fairness in Deepfake Detection

Authors: Yan Ju, Shu Hu, Shan Jia, George H. Chen, Siwei Lyu

Abstract: Despite the development of effective deepfake detectors in recent years, recent studies have demonstrated that biases in the data used to train these detectors can lead to disparities in detection accuracy across different races and genders. This can result in different groups being unfairly targeted or excluded from detection, allowing undetected deepfakes to manipulate public opinion and erode t… ▽ More Despite the development of effective deepfake detectors in recent years, recent studies have demonstrated that biases in the data used to train these detectors can lead to disparities in detection accuracy across different races and genders. This can result in different groups being unfairly targeted or excluded from detection, allowing undetected deepfakes to manipulate public opinion and erode trust in a deepfake detection model. While existing studies have focused on evaluating fairness of deepfake detectors, to the best of our knowledge, no method has been developed to encourage fairness in deepfake detection at the algorithm level. In this work, we make the first attempt to improve deepfake detection fairness by proposing novel loss functions that handle both the setting where demographic information (eg, annotations of race and gender) is available as well as the case where this information is absent. Fundamentally, both approaches can be used to convert many existing deepfake detectors into ones that encourages fairness. Extensive experiments on four deepfake datasets and five deepfake detectors demonstrate the effectiveness and flexibility of our approach in improving deepfake detection fairness. Our code is available at https://github.com/littlejuyan/DF_Fairness. △ Less

Submitted 8 November, 2023; v1 submitted 28 June, 2023; originally announced June 2023.

MC-SpEx: Towards Effective Speaker Extraction with Multi-Scale Interfusion and Conditional Speaker Modulation

Authors: Jun Chen, Wei Rao, Zilin Wang, Jiuxin Lin, Yukai Ju, Shulin He, Yannan Wang, Zhiyong Wu

Abstract: The previous SpEx+ has yielded outstanding performance in speaker extraction and attracted much attention. However, it still encounters inadequate utilization of multi-scale information and speaker embedding. To this end, this paper proposes a new effective speaker extraction system with multi-scale interfusion and conditional speaker modulation (ConSM), which is called MC-SpEx. First of all, we d… ▽ More The previous SpEx+ has yielded outstanding performance in speaker extraction and attracted much attention. However, it still encounters inadequate utilization of multi-scale information and speaker embedding. To this end, this paper proposes a new effective speaker extraction system with multi-scale interfusion and conditional speaker modulation (ConSM), which is called MC-SpEx. First of all, we design the weight-share multi-scale fusers (ScaleFusers) for efficiently leveraging multi-scale information as well as ensuring consistency of the model's feature space. Then, to consider different scale information while generating masks, the multi-scale interactive mask generator (ScaleInterMG) is presented. Moreover, we introduce ConSM module to fully exploit speaker embedding in the speech extractor. Experimental results on the Libri2Mix dataset demonstrate the effectiveness of our improvements and the state-of-the-art performance of our proposed MC-SpEx. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Comments: Accepted by InterSpeech 2023

AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics

Authors: Shan Jia, Mingzhen Huang, Zhou Zhou, Yan Ju, Jialing Cai, Siwei Lyu

Abstract: Recent advancements in language-image models have led to the development of highly realistic images that can be generated from textual descriptions. However, the increased visual quality of these generated images poses a potential threat to the field of media forensics. This paper aims to investigate the level of challenge that language-image generation models pose to media forensics. To achieve t… ▽ More Recent advancements in language-image models have led to the development of highly realistic images that can be generated from textual descriptions. However, the increased visual quality of these generated images poses a potential threat to the field of media forensics. This paper aims to investigate the level of challenge that language-image generation models pose to media forensics. To achieve this, we propose a new approach that leverages the DALL-E2 language-image model to automatically generate and splice masked regions guided by a text prompt. To ensure the creation of realistic manipulations, we have designed an annotation platform with human checking to verify reasonable text prompts. This approach has resulted in the creation of a new image dataset called AutoSplice, containing 5,894 manipulated and authentic images. Specifically, we have generated a total of 3,621 images by locally or globally manipulating real-world image-caption pairs, which we believe will provide a valuable resource for developing generalized detection methods in this area. The dataset is evaluated under two media forensic tasks: forgery detection and localization. Our extensive experiments show that most media forensic models struggle to detect the AutoSplice dataset as an unseen manipulation. However, when fine-tuned models are used, they exhibit improved performance in both tasks. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Near Optimal Colourability on Hereditary Graph Families

Authors: Yiao Ju, Shenwei Huang

Abstract: In this paper, we initiate a systematic study on a new notion called near optimal colourability which is closely related to perfect graphs and the Lov{á}sz theta function. A graph family $\mathcal{G}$ is {\em near optimal colourable} if there is a constant number $c$ such that every graph $G\in\mathcal{G}$ satisfies $χ(G)\leq\max\{c, ω(G)\}$, where $χ(G)$ and $ω(G)$ are the chromatic number and cl… ▽ More In this paper, we initiate a systematic study on a new notion called near optimal colourability which is closely related to perfect graphs and the Lov{á}sz theta function. A graph family $\mathcal{G}$ is {\em near optimal colourable} if there is a constant number $c$ such that every graph $G\in\mathcal{G}$ satisfies $χ(G)\leq\max\{c, ω(G)\}$, where $χ(G)$ and $ω(G)$ are the chromatic number and clique number of $G$, respectively. The near optimal colourable graph families together with the Lov{á}sz theta function are useful for the study of the chromatic number problems for hereditary graph families. We investigate the near optimal colourability for ($H_1,H_2$)-free graphs. Our main result is an almost complete characterization for the near optimal colourability for ($H_1,H_2$)-free graphs with two exceptional cases, one of which is the celebrated Gy{á}rf{á}s conjecture. As an application of our results, we show that the chromatic number problem for ($2K_2,P_4\vee K_n$)-free graphs is polynomial time solvable, which solves an open problem in [K.~K.~Dabrowski and D.~Paulusma. On colouring ($2P_2$, $H$)-free and ($P_5$, $H$)-free graphs. Information Processing Letters, 134:35-41, 2018]. △ Less

Submitted 6 May, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

Comments: 12 pages, 1 figure

Complete Suppression of Phase Segregation in Mixed-Halide Perovskite Nanocrystals under Periodic Heating

Authors: Shengnan Feng, Rentong Duan, Yu Ju, Shuyi Li, Chunfeng Zhang, Shuxia Tao, Min Xiao, Xiaoyong Wang

Abstract: Under continuous light illumination, it is known that localized domains with segregated halide compositions form in semiconducting mixed-halide perovskites, thus severely limiting their optoelectronic applications due to the negative changes in bandgap energies and charge-carrier characteristics. Here we deposit mixed-halide perovskite CsPbBr1.2I1.8 nanocrystals onto an indium tin oxide substrate,… ▽ More Under continuous light illumination, it is known that localized domains with segregated halide compositions form in semiconducting mixed-halide perovskites, thus severely limiting their optoelectronic applications due to the negative changes in bandgap energies and charge-carrier characteristics. Here we deposit mixed-halide perovskite CsPbBr1.2I1.8 nanocrystals onto an indium tin oxide substrate, whose temperature can be rapidly changed by ~10 degree in a few seconds by applying or removing an external voltage. Such a sudden temperature change induces a temporary transition of CsPbBr1.2I1.8 nanocrystals from the segregated phase to the mixed phase, the latter of which can be permanently maintained when the light illumination is coupled with periodic heating cycles. These findings mark the emergence of a practical solution to the detrimental phase-segregation problem, given that a small temperature modulation is readily available in various fundamental studies and practical devices using mixed-halide perovskites. △ Less

Submitted 20 March, 2023; originally announced March 2023.

Comments: 25 pages, 4 figures

TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 DNS Challenge

Authors: Yukai Ju, Jun Chen, Shimin Zhang, Shulin He, Wei Rao, Weixin Zhu, Yannan Wang, Tao Yu, Shidong Shang

Abstract: This paper introduces the Unbeatable Team's submission to the ICASSP 2023 Deep Noise Suppression (DNS) Challenge. We expand our previous work, TEA-PSE, to its upgraded version -- TEA-PSE 3.0. Specifically, TEA-PSE 3.0 incorporates a residual LSTM after squeezed temporal convolution network (S-TCN) to enhance sequence modeling capabilities. Additionally, the local-global representation (LGR) struct… ▽ More This paper introduces the Unbeatable Team's submission to the ICASSP 2023 Deep Noise Suppression (DNS) Challenge. We expand our previous work, TEA-PSE, to its upgraded version -- TEA-PSE 3.0. Specifically, TEA-PSE 3.0 incorporates a residual LSTM after squeezed temporal convolution network (S-TCN) to enhance sequence modeling capabilities. Additionally, the local-global representation (LGR) structure is introduced to boost speaker information extraction, and multi-STFT resolution loss is used to effectively capture the time-frequency characteristics of the speech signals. Moreover, retraining methods are employed based on the freeze training strategy to fine-tune the system. According to the official results, TEA-PSE 3.0 ranks 1st in both ICASSP 2023 DNS-Challenge track 1 and track 2. △ Less

Submitted 14 March, 2023; originally announced March 2023.

Comments: Accepted by ICASSP 2023