Showing 1–50 of 177 results for author: Si, L

On the Causal Sufficiency and Necessity of Multi-Modal Representation Learning

Authors: Jingyao Wang, Wenwen Qiang, Jiangmeng Li, Lingyu Si, Changwen Zheng, Bing Su

Abstract: An effective paradigm of multi-modal learning (MML) is to learn unified representations among modalities. From a causal perspective, constraining the consistency between different modalities can mine causal representations that convey primary events. However, such simple consistency may face the risk of learning insufficient or unnecessary information: a necessary but insufficient cause is invaria… ▽ More An effective paradigm of multi-modal learning (MML) is to learn unified representations among modalities. From a causal perspective, constraining the consistency between different modalities can mine causal representations that convey primary events. However, such simple consistency may face the risk of learning insufficient or unnecessary information: a necessary but insufficient cause is invariant across modalities but may not have the required accuracy; a sufficient but unnecessary cause tends to adapt well to specific modalities but may be hard to adapt to new data. To address this issue, in this paper, we aim to learn representations that are both causal sufficient and necessary, i.e., Causal Complete Cause ($C^3$), for MML. Firstly, we define the concept of $C^3$ for MML, which reflects the probability of being causal sufficiency and necessity. We also propose the identifiability and measurement of $C^3$, i.e., $C^3$ risk, to ensure calculating the learned representations' $C^3$ scores in practice. Then, we theoretically prove the effectiveness of $C^3$ risk by establishing the performance guarantee of MML with a tight generalization bound. Based on these theoretical results, we propose a plug-and-play method, namely Causal Complete Cause Regularization ($C^3$R), to learn causal complete representations by constraining the $C^3$ risk bound. Extensive experiments conducted on various benchmark datasets empirically demonstrate the effectiveness of $C^3$R. △ Less

Submitted 19 July, 2024; originally announced July 2024.

A Physical Model-Guided Framework for Underwater Image Enhancement and Depth Estimation

Authors: Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Jianwei Niu, Fuchun Sun

Abstract: Due to the selective absorption and scattering of light by diverse aquatic media, underwater images usually suffer from various visual degradations. Existing underwater image enhancement (UIE) approaches that combine underwater physical imaging models with neural networks often fail to accurately estimate imaging model parameters such as depth and veiling light, resulting in poor performance in ce… ▽ More Due to the selective absorption and scattering of light by diverse aquatic media, underwater images usually suffer from various visual degradations. Existing underwater image enhancement (UIE) approaches that combine underwater physical imaging models with neural networks often fail to accurately estimate imaging model parameters such as depth and veiling light, resulting in poor performance in certain scenarios. To address this issue, we propose a physical model-guided framework for jointly training a Deep Degradation Model (DDM) with any advanced UIE model. DDM includes three well-designed sub-networks to accurately estimate various imaging parameters: a veiling light estimation sub-network, a factors estimation sub-network, and a depth estimation sub-network. Based on the estimated parameters and the underwater physical imaging model, we impose physical constraints on the enhancement process by modeling the relationship between underwater images and desired clean images, i.e., outputs of the UIE model. Moreover, while our framework is compatible with any UIE model, we design a simple yet effective fully convolutional UIE model, termed UIEConv. UIEConv utilizes both global and local features for image enhancement through a dual-branch structure. UIEConv trained within our framework achieves remarkable enhancement results across diverse underwater scenes. Furthermore, as a byproduct of UIE, the trained depth estimation sub-network enables accurate underwater scene depth estimation. Extensive experiments conducted in various real underwater imaging scenarios, including deep-sea environments with artificial light sources, validate the effectiveness of our framework and the UIEConv model. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Imaging of single barium atoms in a second matrix site in solid xenon for barium tagging in a $^{136}$Xe double beta decay experiment

Authors: M. Yvaine, D. Fairbank, J. Soderstrom, C. Taylor, J. Stanley, T. Walton, C. Chambers, A. Iverson, W. Fairbank, S. Al Kharusi, A. Amy, E. Angelico, A. Anker, I. J. Arnquist, A. Atencio, J. Bane, V. Belov, E. P. Bernard, T. Bhatta, A. Bolotnikov, J. Breslin, P. A. Breur, J. P. Brodsky, E. Brown, T. Brunner , et al. (112 additional authors not shown)

Abstract: Neutrinoless double beta decay is one of the most sensitive probes for new physics beyond the Standard Model of particle physics. One of the isotopes under investigation is $^{136}$Xe, which would double beta decay into $^{136}$Ba. Detecting the single $^{136}$Ba daughter provides a sort of ultimate tool in the discrimination against backgrounds. Previous work demonstrated the ability to perform s… ▽ More Neutrinoless double beta decay is one of the most sensitive probes for new physics beyond the Standard Model of particle physics. One of the isotopes under investigation is $^{136}$Xe, which would double beta decay into $^{136}$Ba. Detecting the single $^{136}$Ba daughter provides a sort of ultimate tool in the discrimination against backgrounds. Previous work demonstrated the ability to perform single atom imaging of Ba atoms in a single-vacancy site of a solid xenon matrix. In this paper, the effort to identify signal from individual barium atoms is extended to Ba atoms in a hexa-vacancy site in the matrix and is achieved despite increased photobleaching in this site. Abrupt fluorescence turn-off of a single Ba atom is also observed. Significant recovery of fluorescence signal lost through photobleaching is demonstrated upon annealing of Ba deposits in the Xe ice. Following annealing, it is observed that Ba atoms in the hexa-vacancy site exhibit antibleaching while Ba atoms in the tetra-vacancy site exhibit bleaching. This may be evidence for a matrix site transfer upon laser excitation. Our findings offer a path of continued research toward tagging of Ba daughters in all significant sites in solid xenon. △ Less

Submitted 28 June, 2024; originally announced July 2024.

Comments: 9 pages, 8 figures

Impact of rotational symmetry breaking on $d$-wave superconductivity in Hubbard models for cuprate and nickelate superconductors

Authors: Hongdao Zhuge, Liang Si, Mi Jiang

Abstract: Recent experiments have revealed the substantial impact of broken rotational symmetry on the superconductivity. In the pursuit of understanding the role played by this symmetry breaking particularly in cuprate and nickelate superconductors on their superconductivity, we investigated two characteristic symmetry breaking mechanisms arising from (1) structurally orthogonal distortions from $C_4$ to… ▽ More Recent experiments have revealed the substantial impact of broken rotational symmetry on the superconductivity. In the pursuit of understanding the role played by this symmetry breaking particularly in cuprate and nickelate superconductors on their superconductivity, we investigated two characteristic symmetry breaking mechanisms arising from (1) structurally orthogonal distortions from $C_4$ to $C_2$ symmetry and (2) anisotropic hybridization between $d_{x^2-y^2}$ orbital and an additional metallic band within the framework of the Hubbard model by employing dynamic cluster quantum Monte Carlo calculations. We discovered that the anisotropy is generically detrimental to the $d$-wave pairing so that the experimental findings of much lower superconducting $T_c$ of infinite-layer nickelates compared with the cuprates may be connected to the intrinsic anisotropy. Our exploration sheds light on the fundamental anisotropy factors governing superconductivity in nickelates and cuprates and offer insights contributing to the broader understanding of unconventional superconductors in anisotropic environment. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Comments: 6 pages, 6 figures

The Legendre Transform of Convex Lattice Sets

Authors: Tingting He, Lin Si

Abstract: The goal of this paper is to study convex lattice sets by the discrete Legendre transform. The definition of the polar of convex lattice sets in $\mathbb{Z}^n$ is provided. It is worth mentioning that the polar of convex lattice sets have the self-dual property similar to that of convex bodies. Some properties of convex lattice sets are established, for instance, the inclusion relation, the union… ▽ More The goal of this paper is to study convex lattice sets by the discrete Legendre transform. The definition of the polar of convex lattice sets in $\mathbb{Z}^n$ is provided. It is worth mentioning that the polar of convex lattice sets have the self-dual property similar to that of convex bodies. Some properties of convex lattice sets are established, for instance, the inclusion relation, the union and intersection on the polar of convex lattice sets. In addition, we discuss the relationship between the cross-polytope and the discrete Mahler product. It states that a convex lattice set is the cross-polytope if and only if its discrete Mahler product is the smallest. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Comments: 21 pages,5 figures

MSC Class: Primary 52C07; Secondary 11H06; 52B20

Data Augmentation for Text-based Person Retrieval Using Large Language Models

Authors: Zheng Li, Lijia Si, Caili Guo, Yang Yang, Qiushi Cao

Abstract: Text-based Person Retrieval (TPR) aims to retrieve person images that match the description given a text query. The performance improvement of the TPR model relies on high-quality data for supervised training. However, it is difficult to construct a large-scale, high-quality TPR dataset due to expensive annotation and privacy protection. Recently, Large Language Models (LLMs) have approached or ev… ▽ More Text-based Person Retrieval (TPR) aims to retrieve person images that match the description given a text query. The performance improvement of the TPR model relies on high-quality data for supervised training. However, it is difficult to construct a large-scale, high-quality TPR dataset due to expensive annotation and privacy protection. Recently, Large Language Models (LLMs) have approached or even surpassed human performance on many NLP tasks, creating the possibility to expand high-quality TPR datasets. This paper proposes an LLM-based Data Augmentation (LLM-DA) method for TPR. LLM-DA uses LLMs to rewrite the text in the current TPR dataset, achieving high-quality expansion of the dataset concisely and efficiently. These rewritten texts are able to increase the diversity of vocabulary and sentence structure while retaining the original key concepts and semantic information. In order to alleviate the hallucinations of LLMs, LLM-DA introduces a Text Faithfulness Filter (TFF) to filter out unfaithful rewritten text. To balance the contributions of original text and augmented text, a Balanced Sampling Strategy (BSS) is proposed to control the proportion of original text and augmented text used for training. LLM-DA is a plug-and-play method that can be easily integrated into various TPR models. Comprehensive experiments on three TPR benchmarks show that LLM-DA can improve the retrieval performance of current TPR models. △ Less

Submitted 20 May, 2024; originally announced May 2024.

Exploring Text-Guided Single Image Editing for Remote Sensing Images

Authors: Fangzhou Han, Lingyu Si, Hongwei Dong, Lamei Zhang, Hao Chen, Bo Du

Abstract: Artificial Intelligence Generative Content (AIGC) technologies have significantly influenced the remote sensing domain, particularly in the realm of image generation. However, remote sensing image editing, an equally vital research area, has not garnered sufficient attention. Different from text-guided editing in natural images, which relies on extensive text-image paired data for semantic correla… ▽ More Artificial Intelligence Generative Content (AIGC) technologies have significantly influenced the remote sensing domain, particularly in the realm of image generation. However, remote sensing image editing, an equally vital research area, has not garnered sufficient attention. Different from text-guided editing in natural images, which relies on extensive text-image paired data for semantic correlation, the application scenarios of remote sensing image editing are often extreme, such as forest on fire, so it is difficult to obtain sufficient paired samples. At the same time, the lack of remote sensing semantics and the ambiguity of text also restrict the further application of image editing in remote sensing field. To solve above problems, this letter proposes a diffusion based method to fulfill stable and controllable remote sensing image editing with text guidance. Our method avoids the use of a large number of paired image, and can achieve good image editing results using only a single image. The quantitative evaluation system including CLIP score and subjective evaluation metrics shows that our method has better editing effect on remote sensing images than the existing image editing model. △ Less

Submitted 9 May, 2024; originally announced May 2024.

Explicitly Modeling Universality into Self-Supervised Learning

Authors: Jingyao Wang, Wenwen Qiang, Zeen Song, Lingyu Si, Jiangmeng Li, Changwen Zheng, Bing Su

Abstract: The goal of universality in self-supervised learning (SSL) is to learn universal representations from unlabeled data and achieve excellent performance on all samples and tasks. However, these methods lack explicit modeling of the universality in the learning objective, and the related theoretical understanding remains limited. This may cause models to overfit in data-scarce situations and generali… ▽ More The goal of universality in self-supervised learning (SSL) is to learn universal representations from unlabeled data and achieve excellent performance on all samples and tasks. However, these methods lack explicit modeling of the universality in the learning objective, and the related theoretical understanding remains limited. This may cause models to overfit in data-scarce situations and generalize poorly in real life. To address these issues, we provide a theoretical definition of universality in SSL, which constrains both the learning and evaluation universality of the SSL models from the perspective of discriminability, transferability, and generalization. Then, we propose a $σ$-measurement to help quantify the score of one SSL model's universality. Based on the definition and measurement, we propose a general SSL framework, called GeSSL, to explicitly model universality into SSL. It introduces a self-motivated target based on $σ$-measurement, which enables the model to find the optimal update direction towards universality. Extensive theoretical and empirical evaluations demonstrate the superior performance of GeSSL. △ Less

Submitted 23 May, 2024; v1 submitted 2 May, 2024; originally announced May 2024.

Comments: 28 pages, submitted to ICML24 with 7766

A dataset of primary nasopharyngeal carcinoma MRI with multi-modalities segmentation

Authors: Yin Li, Qi Chen, Kai Wang, Meige Li, Liping Si, Yingwei Guo, Yu Xiong, Qixing Wang, Yang Qin, Ling Xu, Patrick van der Smagt, Jun Tang, Nutan Chen

Abstract: Multi-modality magnetic resonance imaging data with various sequences facilitate the early diagnosis, tumor segmentation, and disease staging in the management of nasopharyngeal carcinoma (NPC). The lack of publicly available, comprehensive datasets limits advancements in diagnosis, treatment planning, and the development of machine learning algorithms for NPC. Addressing this critical need, we in… ▽ More Multi-modality magnetic resonance imaging data with various sequences facilitate the early diagnosis, tumor segmentation, and disease staging in the management of nasopharyngeal carcinoma (NPC). The lack of publicly available, comprehensive datasets limits advancements in diagnosis, treatment planning, and the development of machine learning algorithms for NPC. Addressing this critical need, we introduce the first comprehensive NPC MRI dataset, encompassing MR axial imaging of 277 primary NPC patients. This dataset includes T1-weighted, T2-weighted, and contrast-enhanced T1-weighted sequences, totaling 831 scans. In addition to the corresponding clinical data, manually annotated and labeled segmentations by experienced radiologists offer high-quality data resources from untreated primary NPC. △ Less

Submitted 4 April, 2024; originally announced April 2024.

End-To-End Underwater Video Enhancement: Dataset and Model

Authors: Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Jianwei Niu

Abstract: Underwater video enhancement (UVE) aims to improve the visibility and frame quality of underwater videos, which has significant implications for marine research and exploration. However, existing methods primarily focus on developing image enhancement algorithms to enhance each frame independently. There is a lack of supervised datasets and models specifically tailored for UVE tasks. To fill this… ▽ More Underwater video enhancement (UVE) aims to improve the visibility and frame quality of underwater videos, which has significant implications for marine research and exploration. However, existing methods primarily focus on developing image enhancement algorithms to enhance each frame independently. There is a lack of supervised datasets and models specifically tailored for UVE tasks. To fill this gap, we construct the Synthetic Underwater Video Enhancement (SUVE) dataset, comprising 840 diverse underwater-style videos paired with ground-truth reference videos. Based on this dataset, we train a novel underwater video enhancement model, UVENet, which utilizes inter-frame relationships to achieve better enhancement performance. Through extensive experiments on both synthetic and real underwater videos, we demonstrate the effectiveness of our approach. This study represents the first comprehensive exploration of UVE to our knowledge. The code is available at https://anonymous.4open.science/r/UVENet. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Search for cosmic-ray boosted sub-MeV dark matter-electron scatterings in PandaX-4T

Authors: Xiaofeng Shang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou, Xiangdong Ji, Yonglin Ju, Chenxiang Li , et al. (67 additional authors not shown)

Abstract: We report the first search for the elastic scatterings between cosmic-ray boosted sub-MeV dark matter and electrons in the PandaX-4T liquid xenon experiment. Sub-MeV dark matter particles can be accelerated by scattering with electrons in the cosmic rays and produce detectable electron recoil signals in the detector. Using the commissioning data from PandaX-4T of 0.63~tonne$\cdot$year exposure, we… ▽ More We report the first search for the elastic scatterings between cosmic-ray boosted sub-MeV dark matter and electrons in the PandaX-4T liquid xenon experiment. Sub-MeV dark matter particles can be accelerated by scattering with electrons in the cosmic rays and produce detectable electron recoil signals in the detector. Using the commissioning data from PandaX-4T of 0.63~tonne$\cdot$year exposure, we set new constraints on DM-electron scattering cross sections for DM masses ranging from 10~eV/$c^2$ to 3~keV/$c^2$. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Comments: 6 pages, 3 figures

Detecting Neutrinos from Supernova Bursts in PandaX-4T

Authors: Binyu Pang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Junting Huang, Zhou Huang, Ruquan Hou , et al. (71 additional authors not shown)

Abstract: Neutrinos from core-collapse supernovae are essential for the understanding of neutrino physics and stellar evolution. The dual-phase xenon dark matter detectors can provide a way to track explosions of galactic supernovae by detecting neutrinos through coherent elastic neutrino-nucleus scatterings. In this study, a variation of progenitor masses as well as explosion models are assumed to predict… ▽ More Neutrinos from core-collapse supernovae are essential for the understanding of neutrino physics and stellar evolution. The dual-phase xenon dark matter detectors can provide a way to track explosions of galactic supernovae by detecting neutrinos through coherent elastic neutrino-nucleus scatterings. In this study, a variation of progenitor masses as well as explosion models are assumed to predict the neutrino fluxes and spectra, which result in the number of expected neutrino events ranging from 6.6 to 13.7 at a distance of 10 kpc over a 10-second duration with negligible backgrounds at PandaX-4T. Two specialized triggering alarms for monitoring supernova burst neutrinos are built. The efficiency of detecting supernova explosions at various distances in the Milky Way is estimated. These alarms will be implemented in the real-time supernova monitoring system at PandaX-4T in the near future, providing the astronomical communities with supernova early warnings. △ Less

Submitted 10 March, 2024; originally announced March 2024.

Comments: 9 pages,6 figures

Signal Response Model in PandaX-4T

Authors: Yunyang Luo, Zihao Bo, Shibo Zhang, Abdusalam Abdukerim, Chen Cheng, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Zhou Huang , et al. (66 additional authors not shown)

Abstract: PandaX-4T experiment is a deep-underground dark matter direct search experiment that employs a dual-phase time projection chamber with a sensitive volume containing 3.7 tonne of liquid xenon. The detector of PandaX-4T is capable of simultaneously collecting the primary scintillation and ionization signals, utilizing their ratio to discriminate dark matter signals from background sources such as ga… ▽ More PandaX-4T experiment is a deep-underground dark matter direct search experiment that employs a dual-phase time projection chamber with a sensitive volume containing 3.7 tonne of liquid xenon. The detector of PandaX-4T is capable of simultaneously collecting the primary scintillation and ionization signals, utilizing their ratio to discriminate dark matter signals from background sources such as gamma rays and beta particles. The signal response model plays a crucial role in interpreting the data obtained by PandaX-4T. It describes the conversion from the deposited energy by dark matter interactions to the detectable signals within the detector. The signal response model is utilized in various PandaX-4T results. This work provides a comprehensive description of the procedures involved in constructing and parameter-fitting the signal response model for the energy range of approximately 1 keV to 25 keV for electronic recoils and 6 keV to 90 keV for nuclear recoils. It also covers the signal reconstruction, selection, and correction methods, which are crucial components integrated into the signal response model. △ Less

Submitted 14 June, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Authors: Huijie Guo, Ying Ba, Jie Hu, Lingyu Si, Wenwen Qiang, Lei Shi

Abstract: Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To add… ▽ More Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and instance segmentation tasks on multiple benchmark datasets. △ Less

Submitted 3 March, 2024; originally announced March 2024.

Hybrid Base Complex: Extract and Visualize Structure of Hex-dominant Meshes

Authors: Lei Si, Haowei Cao, Guoning Chen

Abstract: Hex-dominant mesh generation has received significant attention in recent research due to its superior robustness compared to pure hex-mesh generation techniques. In this work, we introduce the first structure for analyzing hex-dominant meshes. This structure builds on the base complex of pure hex-meshes but incorporates the non-hex elements for a more comprehensive and complete representation. We… ▽ More Hex-dominant mesh generation has received significant attention in recent research due to its superior robustness compared to pure hex-mesh generation techniques. In this work, we introduce the first structure for analyzing hex-dominant meshes. This structure builds on the base complex of pure hex-meshes but incorporates the non-hex elements for a more comprehensive and complete representation. We provide its definition and describe its construction steps. Based on this structure, we present an extraction and categorization of sheets using advanced graph matching techniques to handle the non-hex elements. This enables us to develop an enhanced visual analysis of the structure for any hex-dominant meshes.We apply this structure-based visual analysis to compare hex-dominant meshes generated by different methods to study their advantages and disadvantages. This complements the standard quality metric based on the non-hex element percentage for hex-dominant meshes. Moreover, we propose a strategy to extract a cleaned (optimized) valence-based singularity graph wireframe to analyze the structure for both mesh and sheets. Our results demonstrate that the proposed hybrid base complex provides a coarse representation for mesh element, and the proposed valence singularity graph wireframe provides a better internal visualization of hex-dominant meshes. △ Less

Submitted 1 March, 2024; originally announced March 2024.

Comments: accepted by IEEE Transactions on Visualization and Computer Graphics

PandaX-xT: a Multi-ten-tonne Liquid Xenon Observatory at the China Jinping Underground Laboratory

Authors: PandaX Collaboration, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Linhui Gu, Xunan Guo, Xuyuan Guo, Zhichao Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou , et al. (68 additional authors not shown)

Abstract: We propose a major upgrade to the existing PandaX-4T experiment in the China Jinping Underground Laboratory. The new experiment, PandaX-xT, will be a multi-ten-tonne liquid xenon, ultra-low background, and general-purpose observatory. The full-scaled PandaX-xT contains a 43-tonne liquid xenon active target. Such an experiment will significantly advance our fundamental understanding of particle phy… ▽ More We propose a major upgrade to the existing PandaX-4T experiment in the China Jinping Underground Laboratory. The new experiment, PandaX-xT, will be a multi-ten-tonne liquid xenon, ultra-low background, and general-purpose observatory. The full-scaled PandaX-xT contains a 43-tonne liquid xenon active target. Such an experiment will significantly advance our fundamental understanding of particle physics and astrophysics. The sensitivity of dark matter direct detection will be improved by nearly two orders of magnitude compared to the current best limits, approaching the so-called "neutrino floor" for a dark matter mass above 10 GeV/$c^2$, providing a decisive test to the Weakly Interacting Massive Particle paradigm. By searching for the neutrinoless double beta decay of $^{136}$Xe isotope in the detector, the effective Majorana neutrino mass can be measured to a [10 -- 41] meV/$c^2$ sensitivity, providing a key test to the Dirac/Majorana nature of neutrino s. Astrophysical neutrinos and other ultra-rare interactions can also be measured and searched for with an unprecedented background level, opening up new windows of discovery. Depending on the findings, PandaX-xT will seek the next stage upgrade utilizing isotopic separation on natural xenon. △ Less

Submitted 5 February, 2024; originally announced February 2024.

FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models

Authors: Feihong He, Gang Li, Mengyuan Zhang, Leilei Yan, Lingyu Si, Fanzhang Li, Li Shen

Abstract: The rapid development of generative diffusion models has significantly advanced the field of style transfer. However, most current style transfer methods based on diffusion models typically involve a slow iterative optimization process, e.g., model fine-tuning and textual inversion of style concept. In this paper, we introduce FreeStyle, an innovative style transfer method built upon a pre-trained… ▽ More The rapid development of generative diffusion models has significantly advanced the field of style transfer. However, most current style transfer methods based on diffusion models typically involve a slow iterative optimization process, e.g., model fine-tuning and textual inversion of style concept. In this paper, we introduce FreeStyle, an innovative style transfer method built upon a pre-trained large diffusion model, requiring no further optimization. Besides, our method enables style transfer only through a text description of the desired style, eliminating the necessity of style images. Specifically, we propose a dual-stream encoder and single-stream decoder architecture, replacing the conventional U-Net in diffusion models. In the dual-stream encoder, two distinct branches take the content image and style text prompt as inputs, achieving content and style decoupling. In the decoder, we further modulate features from the dual streams based on a given content image and the corresponding style text prompt for precise style transfer. Our experimental results demonstrate high-quality synthesis and fidelity of our method across various content images and style text prompts. Compared with state-of-the-art methods that require training, our FreeStyle approach notably reduces the computational burden by thousands of iterations, while achieving comparable or superior performance across multiple evaluation metrics including CLIP Aesthetic Score, CLIP Score, and Preference. We have released the code anonymously at: \href{https://anonymous.4open.science/r/FreeStyleAnonymous-0F9B} △ Less

Submitted 18 July, 2024; v1 submitted 28 January, 2024; originally announced January 2024.

Sequential Model for Predicting Patient Adherence in Subcutaneous Immunotherapy for Allergic Rhinitis

Authors: Yin Li, Yu Xiong, Wenxin Fan, Kai Wang, Qingqing Yu, Liping Si, Patrick van der Smagt, Jun Tang, Nutan Chen

Abstract: Objective: Subcutaneous Immunotherapy (SCIT) is the long-lasting causal treatment of allergic rhinitis (AR). How to enhance the adherence of patients to maximize the benefit of allergen immunotherapy (AIT) plays a crucial role in the management of AIT. This study aims to leverage novel machine learning models to precisely predict the risk of non-adherence of AR patients and related local symptom s… ▽ More Objective: Subcutaneous Immunotherapy (SCIT) is the long-lasting causal treatment of allergic rhinitis (AR). How to enhance the adherence of patients to maximize the benefit of allergen immunotherapy (AIT) plays a crucial role in the management of AIT. This study aims to leverage novel machine learning models to precisely predict the risk of non-adherence of AR patients and related local symptom scores in three years SCIT. Methods: The research develops and analyzes two models, sequential latent-variable model (SLVM) of Stochastic Latent Actor-Critic (SLAC) and Long Short-Term Memory (LSTM) evaluating them based on scoring and adherence prediction capabilities. Results: Excluding the biased samples at the first time step, the predictive adherence accuracy of the SLAC models is from 60\% to 72\%, and for LSTM models, it is 66\% to 84\%, varying according to the time steps. The range of Root Mean Square Error (RMSE) for SLAC models is between 0.93 and 2.22, while for LSTM models it is between 1.09 and 1.77. Notably, these RMSEs are significantly lower than the random prediction error of 4.55. Conclusion: We creatively apply sequential models in the long-term management of SCIT with promising accuracy in the prediction of SCIT nonadherence in AR patients. While LSTM outperforms SLAC in adherence prediction, SLAC excels in score prediction for patients undergoing SCIT for AR. The state-action-based SLAC adds flexibility, presenting a novel and effective approach for managing long-term AIT. △ Less

Submitted 19 July, 2024; v1 submitted 21 January, 2024; originally announced January 2024.

Comments: Frontiers in Pharmacology, research topic: Methods and Metrics to Measure Medication Adherence

Measurement of Solar $pp$ Neutrino Flux using Electron Recoil Data from PandaX-4T Commissioning Run

Authors: PandaX Collaboration, Xiaoying Lu, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Lisheng Geng, Karl Giboni, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Junting Huang, Zhou Huang, Ruquan Hou, Yu Hou, Xiangdong Ji , et al. (67 additional authors not shown)

Abstract: The proton-proton ($pp$) fusion chain dominates the neutrino production from the Sun. The uncertainty of the predicted $pp$ neutrino flux is at the sub-percent level, whereas that of the best measurement is $\mathcal{O}(10\%)$. In this paper, we present the first result to measure the solar $pp$ neutrinos in the electron recoil energy range from 24 to 144 keV, using the PandaX-4T commissioning dat… ▽ More The proton-proton ($pp$) fusion chain dominates the neutrino production from the Sun. The uncertainty of the predicted $pp$ neutrino flux is at the sub-percent level, whereas that of the best measurement is $\mathcal{O}(10\%)$. In this paper, we present the first result to measure the solar $pp$ neutrinos in the electron recoil energy range from 24 to 144 keV, using the PandaX-4T commissioning data with 0.63 tonne$\times$year exposure. The $pp$ neutrino flux is determined to be $(8.0 \pm 3.9 \,{\rm{(stat)}} \pm 10.0 \,{\rm{(syst)}} )\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$, consistent with Standard Solar Model and existing measurements, corresponding to a flux upper limit of $23.3\times 10^{10}\, $$\rm{s}^{-1} \rm{cm}^{-2}$ at 90\% C.L.. △ Less

Submitted 2 July, 2024; v1 submitted 13 January, 2024; originally announced January 2024.

Comments: 6 pages, 5 figures

Radon Removal Commissioning of the PandaX-4T Cryogenic Distillation System

Authors: Xiangyi Cui, Zhou Wang, Jiafu Li, Shuaijie Li, Lin Si, Yonglin Ju, Wenbo Ma, Jianglai Liu, Li Zhao, Xiangdong Ji, Rui Yan, Haidong Sha, Peiyao Huang, Xiuli Wang, Huaxuan Liu

Abstract: The PandaX-4T distillation system, designed for the removal of krypton and radon from xenon, is evaluated for its radon removal efficiency using a $^{222}$Rn source during the online distillation process. The PandaX-4T dark matter detector is employed to monitor the temporal evolution of radon activity. To determine the radon reduction factor, the experimental data of radon atoms introduced into a… ▽ More The PandaX-4T distillation system, designed for the removal of krypton and radon from xenon, is evaluated for its radon removal efficiency using a $^{222}$Rn source during the online distillation process. The PandaX-4T dark matter detector is employed to monitor the temporal evolution of radon activity. To determine the radon reduction factor, the experimental data of radon atoms introduced into and bypassed the distillation system is compared. The results indicate that the PandaX-4T distillation system achieves a radon reduction factor exceeding 190 at the flow rate of 10 slpm and the reflux ratio of 1.44. Gas-only online distillation process of a flow rate of 20 slpm is also conducted without observing significant reduction of radon levels in the detector. This observation suggests that the migration flow of radon atoms from the liquid phase to the gas phase is limited, and the flow rate of gas circulation and duration of the process are insignificant compared to the total xenon mass of 5.6 tons in the detector. This study provides the experimental data to support the efficient removal of radon at $\sim$Bq level using the PandaX-4T distillation system, which is the prerequisite of the radon background control in the detector. The further operation with higher flow rate will be applied for the upcoming science run in PandaX-4T. △ Less

Submitted 19 April, 2024; v1 submitted 3 January, 2024; originally announced January 2024.

Comments: 14 pages, 9 figures

Searching for Two-Neutrino and Neutrinoless Double Beta Decay of $^{134}$Xe with the PandaX-4T Experiment

Authors: PandaX Collaboration, Xiyu Yan, Zhaokan Cheng, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Chen Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Junting Huang, Zhou Huang , et al. (72 additional authors not shown)

Abstract: $^{134}$Xe is a candidate isotope for neutrinoless double beta decay~($0νββ$) search. In addition, the two-neutrino case ($2νββ$) allowed by the Standard Model of particle physics has not yet been observed. Utilizing the 10.4% of $^{134}$Xe in the natural xenon in the PandaX-4T detector and its first 94.9-day exposure, we have established the most stringent constraints on $2νββ$ and $0νββ$ of $^{1… ▽ More $^{134}$Xe is a candidate isotope for neutrinoless double beta decay~($0νββ$) search. In addition, the two-neutrino case ($2νββ$) allowed by the Standard Model of particle physics has not yet been observed. Utilizing the 10.4% of $^{134}$Xe in the natural xenon in the PandaX-4T detector and its first 94.9-day exposure, we have established the most stringent constraints on $2νββ$ and $0νββ$ of $^{134}$Xe half-lives, with limits of $2.8\times10^{22}$ yr and $3.0\times10^{23}$ yr at 90% confidence level, respectively. The $2νββ$ ($0νββ$) limit surpasses the previously reported best result by a factor of 32 (2.7), highlighting the potential of large monolithic natural xenon detectors. △ Less

Submitted 28 April, 2024; v1 submitted 25 December, 2023; originally announced December 2023.

Journal ref: Phys.Rev.Lett. 132 (2024) 15, 152502

Waveform Simulation in PandaX-4T

Authors: Jiafu Li, Abdusalam Abdukerim, Chen Cheng, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Zhou Huang, Ruquan Hou , et al. (66 additional authors not shown)

Abstract: Signal reconstruction through software processing is a crucial component of the background and signal models in the PandaX-4T experiment, which is a multi-tonne dark matter direct search experiment. The accuracy of signal reconstruction is influenced by various detector artifacts, including noise, dark count of photomultiplier, impurity photoionization in the detector, and other relevant considera… ▽ More Signal reconstruction through software processing is a crucial component of the background and signal models in the PandaX-4T experiment, which is a multi-tonne dark matter direct search experiment. The accuracy of signal reconstruction is influenced by various detector artifacts, including noise, dark count of photomultiplier, impurity photoionization in the detector, and other relevant considerations. In this study, we present a detailed description of a semi-data-driven approach designed to simulate the signal waveform. This work provides a reliable model for the efficiency and bias of the signal reconstruction in the data analysis of PandaX-4T. By comparing critical variables which relate to the temporal shape and hit pattern of the signals, we demonstrate a good agreement between the simulation and data. △ Less

Submitted 21 May, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

Journal ref: Chin. Phys. C 48, no.7,073001 (2024)

Rethinking Causal Relationships Learning in Graph Neural Networks

Authors: Hang Gao, Chengyu Yao, Jiangmeng Li, Lingyu Si, Yifan Jin, Fengge Wu, Changwen Zheng, Huaping Liu

Abstract: Graph Neural Networks (GNNs) demonstrate their significance by effectively modeling complex interrelationships within graph-structured data. To enhance the credibility and robustness of GNNs, it becomes exceptionally crucial to bolster their ability to capture causal relationships. However, despite recent advancements that have indeed strengthened GNNs from a causal learning perspective, conductin… ▽ More Graph Neural Networks (GNNs) demonstrate their significance by effectively modeling complex interrelationships within graph-structured data. To enhance the credibility and robustness of GNNs, it becomes exceptionally crucial to bolster their ability to capture causal relationships. However, despite recent advancements that have indeed strengthened GNNs from a causal learning perspective, conducting an in-depth analysis specifically targeting the causal modeling prowess of GNNs remains an unresolved issue. In order to comprehensively analyze various GNN models from a causal learning perspective, we constructed an artificially synthesized dataset with known and controllable causal relationships between data and labels. The rationality of the generated data is further ensured through theoretical foundations. Drawing insights from analyses conducted using our dataset, we introduce a lightweight and highly adaptable GNN module designed to strengthen GNNs' causal learning capabilities across a diverse range of tasks. Through a series of experiments conducted on both synthetic datasets and other real-world datasets, we empirically validate the effectiveness of the proposed module. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Spin fluctuations sufficient to mediate superconductivity in nickelates

Authors: Paul Worm, Qisi Wang, Motoharu Kitatani, Izabela Biało, Qiang Gao, Xiaolin Ren, Jaewon Choi, Diana Csontosová, Ke-Jin Zhou, Xingjiang Zhou, Zhihai Zhu, Liang Si, Johan Chang, Jan M. Tomczak, Karsten Held

Abstract: Infinite-layer nickelates show high-temperature superconductivity, and the experimental phase diagram agrees well with the one simulated within the dynamical vertex approximation (D$Γ$A). Here, we compare the spin-fluctuation spectrum behind these calculations to resonant inelastic X-ray scattering experiments. The overall agreement is good. This independent cross-validation of the strength of spi… ▽ More Infinite-layer nickelates show high-temperature superconductivity, and the experimental phase diagram agrees well with the one simulated within the dynamical vertex approximation (D$Γ$A). Here, we compare the spin-fluctuation spectrum behind these calculations to resonant inelastic X-ray scattering experiments. The overall agreement is good. This independent cross-validation of the strength of spin fluctuations strongly supports the scenario, advanced by D$Γ$A, that spin-fluctuations are the mediator of the superconductivity observed in nickelates. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Comments: 13 pages, 8 figures

UIEDP:Underwater Image Enhancement with Diffusion Prior

Authors: Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Jianwei Niu, Fuchun Sun

Abstract: Underwater image enhancement (UIE) aims to generate clear images from low-quality underwater images. Due to the unavailability of clear reference images, researchers often synthesize them to construct paired datasets for training deep models. However, these synthesized images may sometimes lack quality, adversely affecting training outcomes. To address this issue, we propose UIE with Diffusion Pri… ▽ More Underwater image enhancement (UIE) aims to generate clear images from low-quality underwater images. Due to the unavailability of clear reference images, researchers often synthesize them to construct paired datasets for training deep models. However, these synthesized images may sometimes lack quality, adversely affecting training outcomes. To address this issue, we propose UIE with Diffusion Prior (UIEDP), a novel framework treating UIE as a posterior distribution sampling process of clear images conditioned on degraded underwater inputs. Specifically, UIEDP combines a pre-trained diffusion model capturing natural image priors with any existing UIE algorithm, leveraging the latter to guide conditional generation. The diffusion prior mitigates the drawbacks of inferior synthetic images, resulting in higher-quality image generation. Extensive experiments have demonstrated that our UIEDP yields significant improvements across various metrics, especially no-reference image quality assessment. And the generated enhanced images also exhibit a more natural appearance. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Strain mediated phase crossover in Ruddlesden Popper nickelates

Authors: Ting Cui, Songhee Choi, Ting Lin, Chen Liu, Gang Wang, Ningning Wang, Shengru Chen, Haitao Hong, Dongke Rong, Qianying Wang, Qiao Jin, Jia-Ou Wang, Lin Gu, Chen Ge, Can Wang, Jin Guang Cheng, Qinghua Zhang, Liang Si, Kui-juan Jin, Er-Jia Guo

Abstract: Recent progress on the signatures of pressure-induced high temperature superconductivity in Ruddlesden Popper (RP) nickelates (Lan+1NinO3n+1) has attracted growing interest in both theoretical calculations and experimental efforts. The fabrication of high-quality single crystalline RP nickelate thin films is critical for possible reducing the superconducting transition pressure and advancing appli… ▽ More Recent progress on the signatures of pressure-induced high temperature superconductivity in Ruddlesden Popper (RP) nickelates (Lan+1NinO3n+1) has attracted growing interest in both theoretical calculations and experimental efforts. The fabrication of high-quality single crystalline RP nickelate thin films is critical for possible reducing the superconducting transition pressure and advancing applications in microelectronics in the future. In this study, we report the observations of an active phase transition in RP nickelate films induced by misfit strain. We found that RP nickelate films favor the perovskite structure (n = infinite) under tensile strains, while compressive strains stabilize the La3Ni2O7 (n = 2) phase. The selection of distinct phases is governed by the strain dependent formation energy and electronic configuration. In compressively strained La3Ni2O7, we experimentally determined splitting energy is ~0.2 eV and electrons prefer to occupy in-plane orbitals. First principles calculations unveil a robust coupling between strain effects and the valence state of Ni ions in RP nickelates, suggesting a dual driving force for the inevitable phase co-existence transition in RP nickelates. Our work underscores the sensitivity of RP nickelate formation to epitaxial strain, presenting a significant challenge in fabricating pure-phase RP nickelate films. Therefore, special attention to stacking defects and grain boundaries between different RP phases is essential when discussing the pressure-induced superconductivity in RP nickelates. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: 29 pages, 5 figures, one supplementary materials

Unconventional superconductivity without doping: infinite-layer nickelates under pressure

Authors: Simone Di Cataldo, Paul Worm, Jan Tomczak, Liang Si, Karsten Held

Abstract: High-temperature unconventional superconductivity quite generically emerges from doping a strongly correlated parent compound, often (close to) an antiferromagnetic insulator. The recently developed dynamical vertex approximation is a state-of-the-art technique that has quantitatively predicted the superconducting dome of nickelates. Here, we apply it to study the effect of pressure in the infinit… ▽ More High-temperature unconventional superconductivity quite generically emerges from doping a strongly correlated parent compound, often (close to) an antiferromagnetic insulator. The recently developed dynamical vertex approximation is a state-of-the-art technique that has quantitatively predicted the superconducting dome of nickelates. Here, we apply it to study the effect of pressure in the infinite-layer nickelate Sr$_x$Pr$_ {1-x}$NiO$_2$. We reproduce the increase of the critical temperature ($T_c$) under pressure found in experiment up to 12 GPa. According to our results, $T_c$ can be further increased with higher pressures. Even without Sr-doping the parent compound, PrNiO$_2$, will become a high-temperature superconductor thanks to a strongly enhanced self-doping of the \nidxsqysq{} orbital under pressure. With a maximal \Tc{} of 100\,K around 100\,GPa, nickelate superconductors can reach that of the best cuprates. △ Less

Submitted 10 November, 2023; originally announced November 2023.

Comments: Main text: 6 pages, 4 figures. Supplementary information: 18 pages, 16 figures

Journal ref: Nature Communications 5, 3952 (2024)

Weyl points and spin-orbit coupling in copper-substituted lead phosphate apatite

Authors: Martin Braß, Liang Si, Karten Held

Abstract: We study the impact of spin-orbit coupling on the topological band-properties of copper-substituted lead phosphate apatite using a combination of group-theoretical analysis and full-relativistic density-functional theory calculations. We characterize Weyl points at time-reversal invariant momenta and find that a band-inversion due to spin-orbit coupling leads to additional Weyl points close to the… ▽ More We study the impact of spin-orbit coupling on the topological band-properties of copper-substituted lead phosphate apatite using a combination of group-theoretical analysis and full-relativistic density-functional theory calculations. We characterize Weyl points at time-reversal invariant momenta and find that a band-inversion due to spin-orbit coupling leads to additional Weyl points close to the Fermi-edge at general momenta. To determine the position of the altogether 66 Weyl points in the Brilouin-zone, we develop an algorithm that follows a Berry-curvature-derived vector field to its monopole: the Weyl point. The emerging surface Fermi-arcs and their spin-polarization reveal avoided crossings and a Fermi-loop detached from the Weyl points. △ Less

Submitted 12 October, 2023; originally announced October 2023.

PrototypeFormer: Learning to Explore Prototype Relationships for Few-shot Image Classification

Authors: Feihong He, Gang Li, Lingyu Si, Leilei Yan, Fanzhang Li, Fuchun Sun

Abstract: Few-shot image classification has received considerable attention for addressing the challenge of poor classification performance with limited samples in novel classes. However, numerous studies have employed sophisticated learning strategies and diversified feature extraction methods to address this issue. In this paper, we propose our method called PrototypeFormer, which aims to significantly ad… ▽ More Few-shot image classification has received considerable attention for addressing the challenge of poor classification performance with limited samples in novel classes. However, numerous studies have employed sophisticated learning strategies and diversified feature extraction methods to address this issue. In this paper, we propose our method called PrototypeFormer, which aims to significantly advance traditional few-shot image classification approaches by exploring prototype relationships. Specifically, we utilize a transformer architecture to build a prototype extraction module, aiming to extract class representations that are more discriminative for few-shot classification. Additionally, during the model training process, we propose a contrastive learning-based optimization approach to optimize prototype features in few-shot learning scenarios. Despite its simplicity, the method performs remarkably well, with no bells and whistles. We have experimented with our approach on several popular few-shot image classification benchmark datasets, which shows that our method outperforms all current state-of-the-art methods. In particular, our method achieves 97.07% and 90.88% on 5-way 5-shot and 5-way 1-shot tasks of miniImageNet, which surpasses the state-of-the-art results with accuracy of 7.27% and 8.72%, respectively. The code will be released later. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Comments: Submitted to AAAI2024

Cartoondiff: Training-free Cartoon Image Generation with Diffusion Transformer Models

Authors: Feihong He, Gang Li, Lingyu Si, Leilei Yan, Shimeng Hou, Hongwei Dong, Fanzhang Li

Abstract: Image cartoonization has attracted significant interest in the field of image generation. However, most of the existing image cartoonization techniques require re-training models using images of cartoon style. In this paper, we present CartoonDiff, a novel training-free sampling approach which generates image cartoonization using diffusion transformer models. Specifically, we decompose the reverse… ▽ More Image cartoonization has attracted significant interest in the field of image generation. However, most of the existing image cartoonization techniques require re-training models using images of cartoon style. In this paper, we present CartoonDiff, a novel training-free sampling approach which generates image cartoonization using diffusion transformer models. Specifically, we decompose the reverse process of diffusion models into the semantic generation phase and the detail generation phase. Furthermore, we implement the image cartoonization process by normalizing high-frequency signal of the noisy image in specific denoising steps. CartoonDiff doesn't require any additional reference images, complex model designs, or the tedious adjustment of multiple parameters. Extensive experimental results show the powerful ability of our CartoonDiff. The project page is available at: https://cartoondiff.github.io/ △ Less

Submitted 15 September, 2023; originally announced September 2023.

Comments: 5 pages,5 figures

Background Debiased SAR Target Recognition via Causal Interventional Regularizer

Authors: Hongwei Dong, Fangzhou Han, Lingyu Si, Wenwen Qiang, Lamei Zhang

Abstract: Recent studies have utilized deep learning (DL) techniques to automatically extract features from synthetic aperture radar (SAR) images, which shows great promise for enhancing the performance of SAR automatic target recognition (ATR). However, our research reveals a previously overlooked issue: SAR images to be recognized include not only the foreground (i.e., the target), but also a certain size… ▽ More Recent studies have utilized deep learning (DL) techniques to automatically extract features from synthetic aperture radar (SAR) images, which shows great promise for enhancing the performance of SAR automatic target recognition (ATR). However, our research reveals a previously overlooked issue: SAR images to be recognized include not only the foreground (i.e., the target), but also a certain size of the background area. When a DL-model is trained exclusively on foreground data, its recognition performance is significantly superior to a model trained on original data that includes both foreground and background. This suggests that the presence of background impedes the ability of the DL-model to learn additional semantic information about the target. To address this issue, we construct a structural causal model (SCM) that incorporates the background as a confounder. Based on the constructed SCM, we propose a causal intervention based regularization method to eliminate the negative impact of background on feature semantic learning and achieve background debiased SAR-ATR. The proposed causal interventional regularizer can be integrated into any existing DL-based SAR-ATR models to mitigate the impact of background interference on the feature extraction and recognition accuracy. Experimental results on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset indicate that the proposed method can enhance the efficiency of existing DL-based methods in a plug-and-play manner. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Comments: 38 pages, 8 figures

A Visualization System for Hexahedral Mesh Quality Study

Authors: Lei Si, Guoning Chen

Abstract: In this paper, we introduce a new 3D hex mesh visual analysis system that emphasizes poor-quality areas with an aggregated glyph, highlights overlapping elements, and provides detailed boundary error inspection in three forms. By supporting multi-level analysis through multiple views, our system effectively evaluates various mesh models and compares the performance of mesh generation and optimizat… ▽ More In this paper, we introduce a new 3D hex mesh visual analysis system that emphasizes poor-quality areas with an aggregated glyph, highlights overlapping elements, and provides detailed boundary error inspection in three forms. By supporting multi-level analysis through multiple views, our system effectively evaluates various mesh models and compares the performance of mesh generation and optimization algorithms for hexahedral meshes. △ Less

Submitted 24 August, 2023; v1 submitted 23 August, 2023; originally announced August 2023.

Comments: Accepted by IEEE VIS 2023 Short Papers and will be published on IEEE Xplore. Paper contains 4 pages, and 1 reference page. Supplemental includes 4 pages

ACM Class: I.3.0

No superconductivity in Pb$_9$Cu$_1$(PO$_4$)$_6$O found in orbital and spin fluctuation exchange calculations

Authors: Niklas Witt, Liang Si, Jan M. Tomczak, Karsten Held, Tim O. Wehling

Abstract: Finding a material that turns superconducting under ambient conditions has been the goal of over a century of research, and recently Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O aka LK-99 has been put forward as a possible contestant. In this work, we study the possibility of electronically driven superconductivity in LK-99 also allowing for electron or hole doping. We use an $\textit{ab initio}$ derived two-ba… ▽ More Finding a material that turns superconducting under ambient conditions has been the goal of over a century of research, and recently Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O aka LK-99 has been put forward as a possible contestant. In this work, we study the possibility of electronically driven superconductivity in LK-99 also allowing for electron or hole doping. We use an $\textit{ab initio}$ derived two-band model of the Cu $e_g$ orbitals for which we determine interaction values from the constrained random phase approximation (cRPA). For this two-band model we perform calculations in the fluctuation exchange (FLEX) approach to assess the strength of orbital and spin fluctuations. We scan over a broad range of parameters and enforce no magnetic or orbital symmetry breaking. Even under optimized conditions for superconductivity, spin and orbital fluctuations turn out to be too weak for superconductivity anywhere near to room-temperature. We contrast this finding to non-self-consistent RPA, where it is possible to induce spin-singlet $d$-wave superconductivity at $T_{\mathrm{c}}\geq300$ K if the system is put close enough to a magnetic instability. △ Less

Submitted 26 October, 2023; v1 submitted 14 August, 2023; originally announced August 2023.

Comments: 14 pages, 3 figures; revised submission to SciPost Physics

Journal ref: SciPost Phys. 15, 197 (2023)

Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O: a Mott or charge transfer insulator in need of further doping for (super)conductivity

Authors: Liang Si, Markus Wallerberger, Andriy Smolyanyuk, Simone di Cataldo, Jan M. Tomczak, Karsten Held

Abstract: We briefly review the status quo of research on the putative superconductor Pb$_9$Cu(PO$_4$)$_6$O also known as LK-99. Further, we provide {\em ab initio} derived tight-binding parameters for a two- and five-band model, and solve these in dynamical-mean-field theory. The ratio interaction-to-bandwidth makes LK-99 a Mott or charge transfer insulator. Electron or hole doping (which is different from… ▽ More We briefly review the status quo of research on the putative superconductor Pb$_9$Cu(PO$_4$)$_6$O also known as LK-99. Further, we provide {\em ab initio} derived tight-binding parameters for a two- and five-band model, and solve these in dynamical-mean-field theory. The ratio interaction-to-bandwidth makes LK-99 a Mott or charge transfer insulator. Electron or hole doping (which is different from substituting Pb by Cu and thus differs from LK-99) is required to make it metallic and potentially superconducting. △ Less

Submitted 9 August, 2023; v1 submitted 8 August, 2023; originally announced August 2023.

Comments: 7 figures, 9 pages, 2 tables. Version 2 is the same as version 1 except for additional DMFT and DFT+U results in Secs. IV and V

Search for Dark-Matter-Nucleon Interactions with a Dark Mediator in PandaX-4T

Authors: Di Huang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Yanlin Huang, Zhou Huang, Ruquan Hou, Xiangdong Ji , et al. (70 additional authors not shown)

Abstract: We report results of a search for dark-matter-nucleon interactions via a dark mediator using optimized low-energy data from the PandaX-4T liquid xenon experiment. With the ionization-signal-only data and utilizing the Migdal effect, we set the most stringent limits on the cross section for dark matter masses ranging from 30~$\rm{MeV/c^2}$ to 2~$\rm{GeV/c^2}$. Under the assumption that the dark med… ▽ More We report results of a search for dark-matter-nucleon interactions via a dark mediator using optimized low-energy data from the PandaX-4T liquid xenon experiment. With the ionization-signal-only data and utilizing the Migdal effect, we set the most stringent limits on the cross section for dark matter masses ranging from 30~$\rm{MeV/c^2}$ to 2~$\rm{GeV/c^2}$. Under the assumption that the dark mediator is a dark photon that decays into scalar dark matter pairs in the early Universe, we rule out significant parameter space of such thermal relic dark-matter model. △ Less

Submitted 18 December, 2023; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: 6 pages, 4 figures

Journal ref: Phys. Rev. Lett. 131, 191002 (2023)

Electronic structure of the putative room-temperature superconductor Pb$_9$Cu(PO$_4$)$_6$O

Authors: Liang Si, Karsten Held

Abstract: A recent paper [Lee {\em et al.}, J. Korean Cryt. Growth Cryst. Techn. {\bf 33}, 61 (2023)] provides some experimental indications that Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O with $x\approx 1$, coined LK-99, might be a room-temperature superconductor at ambient pressure. Our density-functional theory calculations show lattice parameters and a volume contraction with $x$ -- very similar to experiment. The… ▽ More A recent paper [Lee {\em et al.}, J. Korean Cryt. Growth Cryst. Techn. {\bf 33}, 61 (2023)] provides some experimental indications that Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O with $x\approx 1$, coined LK-99, might be a room-temperature superconductor at ambient pressure. Our density-functional theory calculations show lattice parameters and a volume contraction with $x$ -- very similar to experiment. The DFT electronic structure shows Cu$^{2+}$ in a $3d^9$ configuration with two flat Cu bands crossing the Fermi energy. This puts Pb$_{9}$Cu(PO$_4$)$_6$O in an ultra-correlated regime and suggests that, without doping, it is a Mott or charge transfer insulator. If doped such an electronic structure might support flat-band superconductivity or an correlation-enhanced electron-phonon mechanism, whereas a diamagnet without superconductivity appears to be rather at odds with our results. △ Less

Submitted 25 September, 2023; v1 submitted 1 August, 2023; originally announced August 2023.

Comments: 10 pages, 7 figures and 4 tables including supplementary materials

Journal ref: Physical Review B 108, L121110 (2023)

Super-tetragonal Sr4Al2O7: a versatile sacrificial layer for high-integrity freestanding oxide membranes

Authors: Jinfeng Zhang, Ting Lin, Ao Wang, Xiaochao Wang, Qingyu He, Huan Ye, Jingdi Lu, Qing Wang, Zhengguo Liang, Feng Jin, Shengru Chen, Minghui Fan, Er-Jia Guo, Qinghua Zhang, Lin Gu, Zhenlin Luo, Liang Si, Wenbin Wu, Lingfei Wang

Abstract: Releasing the epitaxial oxide heterostructures from substrate constraints leads to the emergence of various correlated electronic phases and paves the way for integrations with advanced semiconductor technologies. Identifying a suitable water-soluble sacrificial layer, compatible with the high-quality epitaxial growth of oxide heterostructures, is currently the key to the development of large-scal… ▽ More Releasing the epitaxial oxide heterostructures from substrate constraints leads to the emergence of various correlated electronic phases and paves the way for integrations with advanced semiconductor technologies. Identifying a suitable water-soluble sacrificial layer, compatible with the high-quality epitaxial growth of oxide heterostructures, is currently the key to the development of large-scale freestanding oxide membranes. In this study, we unveil the super-tetragonal Sr4Al2O7 (SAOT) as a promising water-soluble sacrificial layer. The distinct low-symmetric crystal structure of SAOT enables a superior capability to sustain epitaxial strain, thus allowing for broad tunability in lattice constants. The resultant structural coherency and defect-free interface in perovskite ABO3/SAOT heterostructures effectively restrain crack formations during the water-assisted release of freestanding oxide membranes. For a variety of non-ferroelectric oxide membranes, the crack-free areas can span up to a few millimeters in length scale. These compelling features, combined with the inherent high-water solubility, make SAOT a versatile and feasible sacrificial layer for producing high-quality freestanding oxide membranes, thereby boosting their potential for innovative oxide electronics and flexible device designs. △ Less

Submitted 6 October, 2023; v1 submitted 27 July, 2023; originally announced July 2023.

Comments: 5 figures and SI, it is the second version of this manuscript

Ru doping induced spin frustration and enhancement of the room-temperature anomalous Hall effect in La2/3Sr1/3MnO3 films

Authors: Enda Hua, Liang Si, Kunjie Dai, Qing Wang, Huan Ye, Kuan Liu, Jinfeng Zhang, Jingdi Lu, Kai Chen, Feng Jin, Lingfei Wang, Wenbin Wu

Abstract: In transition-metal-oxide heterostructures, the anomalous Hall effect (AHE) is a powerful tool for detecting the magnetic state and revealing intriguing interfacial magnetic orderings. However, achieving a larger AHE at room temperature in oxide heterostructures is still challenging due to the dilemma of mutually strong spin-orbit coupling and magnetic exchange interactions. Here, we exploit the R… ▽ More In transition-metal-oxide heterostructures, the anomalous Hall effect (AHE) is a powerful tool for detecting the magnetic state and revealing intriguing interfacial magnetic orderings. However, achieving a larger AHE at room temperature in oxide heterostructures is still challenging due to the dilemma of mutually strong spin-orbit coupling and magnetic exchange interactions. Here, we exploit the Ru doping-enhanced AHE in LSMRO epitaxial films. As the B-site Ru doping level increases up to 20 percent, the anomalous Hall resistivity at room temperature can be enhanced from nOhmcm to uOhmcm scale. Ru doping leads to strong competition between ferromagnetic double-exchange interaction and antiferromagnetic super-exchange interaction. The resultant spin frustration and spin-glass state facilitate a strong skew-scattering process, thus significantly enhancing the extrinsic AHE. Our findings could pave a feasible approach for boosting the controllability and reliability of oxide-based spintronic devices. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Journal ref: Advanced Materials 34, 2206685 (2022)

Effects of different concentrations of topotactic hydrogen impurities on the electronic structure of nickelate superconductors

Authors: Chenye Qin, Mi Jiang, Liang Si

Abstract: Infinite-layer nickelate superconductors have recently been discovered to share both similarities and differences with cuprate superconductors. Notably, the incorporation of hydrogen (H) through topotactic reduction has been found to play a critical role in their electronic structure and, consequently, their superconductivity. In this study, we utilized a theoretical approach combining density-fun… ▽ More Infinite-layer nickelate superconductors have recently been discovered to share both similarities and differences with cuprate superconductors. Notably, the incorporation of hydrogen (H) through topotactic reduction has been found to play a critical role in their electronic structure and, consequently, their superconductivity. In this study, we utilized a theoretical approach combining density-functional theory and impurity approximation to design three characteristic multi-orbital Hubbard models representing low, moderate, and high concentrations of topotactic-hydrogen. Consistent with experimental findings, our simulations revealed that both low and high concentrations of topotactic-hydrogen induce high-spin states ($S$=1) that are composed by holes at $d_{x^2-y^2}$ and $d_{z^2}$ orbitals and consequently the emergent inter-site hopping between $d_{z^2}$ to $d_{x^2-y^2}$ is unfavorable for superconductivity. Conversely, an optimal concentration of 25\% H aligns with the single Ni-$d_{x^2-y^2}$ band picture of superconductivity in infinite-layer nickelates, demonstrating its beneficial effect on promoting superconducting behavior. △ Less

Submitted 22 July, 2023; originally announced July 2023.

Comments: 9 pages, 6 figures

A Dimensional Structure based Knowledge Distillation Method for Cross-Modal Learning

Authors: Lingyu Si, Hongwei Dong, Wenwen Qiang, Junzhi Yu, Wenlong Zhai, Changwen Zheng, Fanjiang Xu, Fuchun Sun

Abstract: Due to limitations in data quality, some essential visual tasks are difficult to perform independently. Introducing previously unavailable information to transfer informative dark knowledge has been a common way to solve such hard tasks. However, research on why transferred knowledge works has not been extensively explored. To address this issue, in this paper, we discover the correlation between… ▽ More Due to limitations in data quality, some essential visual tasks are difficult to perform independently. Introducing previously unavailable information to transfer informative dark knowledge has been a common way to solve such hard tasks. However, research on why transferred knowledge works has not been extensively explored. To address this issue, in this paper, we discover the correlation between feature discriminability and dimensional structure (DS) by analyzing and observing features extracted from simple and hard tasks. On this basis, we express DS using deep channel-wise correlation and intermediate spatial distribution, and propose a novel cross-modal knowledge distillation (CMKD) method for better supervised cross-modal learning (CML) performance. The proposed method enforces output features to be channel-wise independent and intermediate ones to be uniformly distributed, thereby learning semantically irrelevant features from the hard task to boost its accuracy. This is especially useful in specific applications where the performance gap between dual modalities is relatively large. Furthermore, we collect a real-world CML dataset to promote community development. The dataset contains more than 10,000 paired optical and radar images and is continuously being updated. Experimental results on real-world and benchmark datasets validate the effectiveness of the proposed method. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Chiral magnetism and ordering of oxygen vacancies in SrTiO$_{2.5}$

Authors: Liang Si, Xiaochao Wang, Paul Worm, Wei Peng, Minjae Kim, Lingfei Wang, Karsten Held

Abstract: Oxygen vacancies in the perovskite insulator SrTiO$_3$ free electrons that couple with other physical degrees of freedom such as lattice, orbital, and spin. This leads to the emergence of exotic quantum states such as superconductivity and unusual ferromagnetism. We perform density-functional theory and dynamical mean-field theory calculations and demonstrate that the orientation and ordering of t… ▽ More Oxygen vacancies in the perovskite insulator SrTiO$_3$ free electrons that couple with other physical degrees of freedom such as lattice, orbital, and spin. This leads to the emergence of exotic quantum states such as superconductivity and unusual ferromagnetism. We perform density-functional theory and dynamical mean-field theory calculations and demonstrate that the orientation and ordering of the TiO$_5$ pentahedra plays a crucial role. Specifically, for vacancy-rich SrTiO$_{3-δ}$ ($δ\sim$0.5), we find a chiral ordering of the TiO$_5$ pentahedra in a sixfold superlattice. This chiral structure is accompanied by a chiral magnetic state with a net moment in the (111) direction at room temperature, which can explain several experimental observations. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: 10 pages, 8 figures including supplementary materials

Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering

Authors: Qianglong Chen, Guohai Xu, Ming Yan, Ji Zhang, Fei Huang, Luo Si, Yin Zhang

Abstract: Existing knowledge-enhanced methods have achieved remarkable results in certain QA tasks via obtaining diverse knowledge from different knowledge bases. However, limited by the properties of retrieved knowledge, they still have trouble benefiting from both the knowledge relevance and distinguishment simultaneously. To address the challenge, we propose CPACE, a Concept-centric Prompt-bAsed Contrast… ▽ More Existing knowledge-enhanced methods have achieved remarkable results in certain QA tasks via obtaining diverse knowledge from different knowledge bases. However, limited by the properties of retrieved knowledge, they still have trouble benefiting from both the knowledge relevance and distinguishment simultaneously. To address the challenge, we propose CPACE, a Concept-centric Prompt-bAsed Contrastive Explanation Generation model, which aims to convert obtained symbolic knowledge into a contrastive explanation for better distinguishing the differences among given candidates. Firstly, following previous works, we retrieve different types of symbolic knowledge with a concept-centric knowledge extraction module. After that, we generate corresponding contrastive explanations using acquired symbolic knowledge and explanation prompts as guidance for better modeling the knowledge distinguishment and interpretability. Finally, we regard the generated contrastive explanation as external knowledge for downstream task enhancement. We conduct a series of experiments on three widely-used question-answering datasets: CSQA, QASC, and OBQA. Experimental results demonstrate that with the help of generated contrastive explanation, our CPACE model achieves new SOTA on CSQA (89.8% on the testing set, 0.9% higher than human performance), and gains impressive improvement on QASC and OBQA (4.2% and 3.5%, respectively). △ Less

Submitted 21 May, 2023; v1 submitted 14 May, 2023; originally announced May 2023.

Comments: Accepted to ACL2023(Findings). The Camera-ready Version

Absence of electron-phonon-mediated superconductivity in hydrogen-intercalated nickelates

Authors: Simone Di Cataldo, Paul Worm, Liang Si, Karsten Held

Abstract: A recent experiment [X. Ding et al., Nature 615, 50 (2023)] indicates that superconductivity in nickelates is restricted to a narrow window of hydrogen concentration: 0.22 < x < 0.28 in Nd$_{0.8}$Sr$_{0.2}$NiO$_{2}$H$_{x}$. This reported necessity of hydrogen suggests that it plays a crucial role for superconductivity, as it does in the vast field of hydride superconductors. Using density-function… ▽ More A recent experiment [X. Ding et al., Nature 615, 50 (2023)] indicates that superconductivity in nickelates is restricted to a narrow window of hydrogen concentration: 0.22 < x < 0.28 in Nd$_{0.8}$Sr$_{0.2}$NiO$_{2}$H$_{x}$. This reported necessity of hydrogen suggests that it plays a crucial role for superconductivity, as it does in the vast field of hydride superconductors. Using density-functional theory and its extensions, we explore the effect of topotactic hydrogen on the electronic structure and phonon-mediated superconductivity in nickelate superconductors. Our calculations show that the electron-phonon coupling in hydrogen-intercalated nickelates is not strong enough to drive the electron pairing, and thus cannot explain the reported superconductivity. △ Less

Submitted 7 April, 2023; originally announced April 2023.

Comments: 3 figures, 1 table

Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillation

Authors: Xiaoxiao He, Chaowei Tan, Bo Liu, Liping Si, Weiwu Yao, Liang Zhao, Di Liu, Qilong Zhangli, Qi Chang, Kang Li, Dimitris N. Metaxas

Abstract: Federated Learning has gained popularity among medical institutions since it enables collaborative training between clients (e.g., hospitals) without aggregating data. However, due to the high cost associated with creating annotations, especially for large 3D image datasets, clinical institutions do not have enough supervised data for training locally. Thus, the performance of the collaborative mo… ▽ More Federated Learning has gained popularity among medical institutions since it enables collaborative training between clients (e.g., hospitals) without aggregating data. However, due to the high cost associated with creating annotations, especially for large 3D image datasets, clinical institutions do not have enough supervised data for training locally. Thus, the performance of the collaborative model is subpar under limited supervision. On the other hand, large institutions have the resources to compile data repositories with high-resolution images and labels. Therefore, individual clients can utilize the knowledge acquired in the public data repositories to mitigate the shortage of private annotated images. In this paper, we propose a federated few-shot learning method with dual knowledge distillation. This method allows joint training with limited annotations across clients without jeopardizing privacy. The supervised learning of the proposed method extracts features from limited labeled data in each client, while the unsupervised data is used to distill both feature and response-based knowledge from a national data repository to further improve the accuracy of the collaborative model and reduce the communication cost. Extensive evaluations are conducted on 3D magnetic resonance knee images from a private clinical dataset. Our proposed method shows superior performance and less training time than other semi-supervised federated learning methods. Codes and additional visualization results are available at https://github.com/hexiaoxiao-cs/fedml-knee. △ Less

Submitted 17 April, 2023; v1 submitted 25 March, 2023; originally announced March 2023.

Introducing Expertise Logic into Graph Representation Learning from A Causal Perspective

Authors: Hang Gao, Jiangmeng Li, Wenwen Qiang, Lingyu Si, Xingzhe Su, Fengge Wu, Changwen Zheng, Fuchun Sun

Abstract: Benefiting from the injection of human prior knowledge, graphs, as derived discrete data, are semantically dense so that models can efficiently learn the semantic information from such data. Accordingly, graph neural networks (GNNs) indeed achieve impressive success in various fields. Revisiting the GNN learning paradigms, we discover that the relationship between human expertise and the knowledge… ▽ More Benefiting from the injection of human prior knowledge, graphs, as derived discrete data, are semantically dense so that models can efficiently learn the semantic information from such data. Accordingly, graph neural networks (GNNs) indeed achieve impressive success in various fields. Revisiting the GNN learning paradigms, we discover that the relationship between human expertise and the knowledge modeled by GNNs still confuses researchers. To this end, we introduce motivating experiments and derive an empirical observation that the GNNs gradually learn human expertise in general domains. By further observing the ramifications of introducing expertise logic into graph representation learning, we conclude that leading the GNNs to learn human expertise can improve the model performance. Hence, we propose a novel graph representation learning method to incorporate human expert knowledge into GNN models. The proposed method ensures that the GNN model can not only acquire the expertise held by human experts but also engage in end-to-end learning from datasets. Plentiful experiments on the crafted and real-world domains support the consistent effectiveness of the proposed method. △ Less

Submitted 23 May, 2023; v1 submitted 20 January, 2023; originally announced January 2023.

Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing

Authors: Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, Yongbin Li

Abstract: The task of text-to-SQL parsing, which aims at converting natural language questions into executable SQL queries, has garnered increasing attention in recent years, as it can assist end users in efficiently extracting vital information from databases without the need for technical background. One of the major challenges in text-to-SQL parsing is domain generalization, i.e., how to generalize well… ▽ More The task of text-to-SQL parsing, which aims at converting natural language questions into executable SQL queries, has garnered increasing attention in recent years, as it can assist end users in efficiently extracting vital information from databases without the need for technical background. One of the major challenges in text-to-SQL parsing is domain generalization, i.e., how to generalize well to unseen databases. Recently, the pre-trained text-to-text transformer model, namely T5, though not specialized for text-to-SQL parsing, has achieved state-of-the-art performance on standard benchmarks targeting domain generalization. In this work, we explore ways to further augment the pre-trained T5 model with specialized components for text-to-SQL parsing. Such components are expected to introduce structural inductive bias into text-to-SQL parsers thus improving model's capacity on (potentially multi-hop) reasoning, which is critical for generating structure-rich SQLs. To this end, we propose a new architecture GRAPHIX-T5, a mixed model with the standard pre-trained transformer model augmented by some specially-designed graph-aware layers. Extensive experiments and analysis demonstrate the effectiveness of GRAPHIX-T5 across four text-to-SQL benchmarks: SPIDER, SYN, REALISTIC and DK. GRAPHIX-T5 surpass all other T5-based parsers with a significant margin, achieving new state-of-the-art performance. Notably, GRAPHIX-T5-large reach performance superior to the original T5-large by 5.7% on exact match (EM) accuracy and 6.6% on execution accuracy (EX). This even outperforms the T5-3B by 1.2% on EM and 1.5% on EX. △ Less

Submitted 18 January, 2023; originally announced January 2023.

Comments: Accepted to AAAI 2023 main conference (oral)

Search for light dark matter from atmosphere in PandaX-4T

Authors: Xuyang Ning, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, Changbo Fu, Mengting Fu, Lisheng Geng, Karl Giboni, Linhui Gu, Xuyuan Guo, Chencheng Han, Ke Han, Changda He, Jinrong He, Di Huang, Yanlin Huang, Zhou Huang, Ruquan Hou , et al. (70 additional authors not shown)

Abstract: We report a search for light dark matter produced through the cascading decay of $η$ mesons, which are created as a result of inelastic collisions between cosmic rays and Earth's atmosphere. We introduce a new and general framework, publicly accessible, designed to address boosted dark matter specifically, with which a full and dedicated simulation including both elastic and quasi-elastic processe… ▽ More We report a search for light dark matter produced through the cascading decay of $η$ mesons, which are created as a result of inelastic collisions between cosmic rays and Earth's atmosphere. We introduce a new and general framework, publicly accessible, designed to address boosted dark matter specifically, with which a full and dedicated simulation including both elastic and quasi-elastic processes of Earth attenuation effect on the dark matter particles arriving at the detector is performed. In the PandaX-4T commissioning data of 0.63 tonne$\cdot$year exposure, no significant excess over background is observed. The first constraints on the interaction between light dark matter generated in the atmosphere and nucleus through a light scalar mediator are obtained. The lowest excluded cross-section is set at $5.9 \times 10^{-37}{\rm cm^2}$ for dark matter mass of $0.1$ MeV$/c^2$ and mediator mass of 300 MeV$/c^2$. The lowest upper limit of $η$ to dark matter decay branching ratio is $1.6 \times 10^{-7}$. △ Less

Submitted 25 July, 2023; v1 submitted 8 January, 2023; originally announced January 2023.

Comments: 6 pages, 3 figures

Timestamp-Supervised Action Segmentation from the Perspective of Clustering

Authors: Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Fuchun Sun

Abstract: Video action segmentation under timestamp supervision has recently received much attention due to lower annotation costs. Most existing methods generate pseudo-labels for all frames in each video to train the segmentation model. However, these methods suffer from incorrect pseudo-labels, especially for the semantically unclear frames in the transition region between two consecutive actions, which… ▽ More Video action segmentation under timestamp supervision has recently received much attention due to lower annotation costs. Most existing methods generate pseudo-labels for all frames in each video to train the segmentation model. However, these methods suffer from incorrect pseudo-labels, especially for the semantically unclear frames in the transition region between two consecutive actions, which we call ambiguous intervals. To address this issue, we propose a novel framework from the perspective of clustering, which includes the following two parts. First, pseudo-label ensembling generates incomplete but high-quality pseudo-label sequences, where the frames in ambiguous intervals have no pseudo-labels. Second, iterative clustering iteratively propagates the pseudo-labels to the ambiguous intervals by clustering, and thus updates the pseudo-label sequences to train the model. We further introduce a clustering loss, which encourages the features of frames within the same action segment more compact. Extensive experiments show the effectiveness of our method. △ Less

Submitted 22 April, 2023; v1 submitted 22 December, 2022; originally announced December 2022.

Comments: Accepted as a conference paper to the 32nd International Joint Conference on Artificial Intelligence (IJCAI-23)

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Authors: Weiwen Xu, Xin Li, Wenxuan Zhang, Meng Zhou, Wai Lam, Luo Si, Lidong Bing

Abstract: We present Pre-trained Machine Reader (PMR), a novel method for retrofitting pre-trained masked language models (MLMs) to pre-trained machine reading comprehension (MRC) models without acquiring labeled data. PMR can resolve the discrepancy between model pre-training and downstream fine-tuning of existing MLMs. To build the proposed PMR, we constructed a large volume of general-purpose and high-qu… ▽ More We present Pre-trained Machine Reader (PMR), a novel method for retrofitting pre-trained masked language models (MLMs) to pre-trained machine reading comprehension (MRC) models without acquiring labeled data. PMR can resolve the discrepancy between model pre-training and downstream fine-tuning of existing MLMs. To build the proposed PMR, we constructed a large volume of general-purpose and high-quality MRC-style training data by using Wikipedia hyperlinks and designed a Wiki Anchor Extraction task to guide the MRC-style pre-training. Apart from its simplicity, PMR effectively solves extraction tasks, such as Extractive Question Answering and Named Entity Recognition. PMR shows tremendous improvements over existing approaches, especially in low-resource scenarios. When applied to the sequence classification task in the MRC formulation, PMR enables the extraction of high-quality rationales to explain the classification process, thereby providing greater prediction explainability. PMR also has the potential to serve as a unified model for tackling various extraction and classification tasks in the MRC formulation. △ Less

Submitted 16 October, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: Accepted to NeurIPS 2023

Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?

Authors: Pei Zhang, Baosong Yang, Haoran Wei, Dayiheng Liu, Kai Fan, Luo Si, Jun Xie

Abstract: Neural machine translation (NMT) is often criticized for failures that happen without awareness. The lack of competency awareness makes NMT untrustworthy. This is in sharp contrast to human translators who give feedback or conduct further investigations whenever they are in doubt about predictions. To fill this gap, we propose a novel competency-aware NMT by extending conventional NMT with a self-… ▽ More Neural machine translation (NMT) is often criticized for failures that happen without awareness. The lack of competency awareness makes NMT untrustworthy. This is in sharp contrast to human translators who give feedback or conduct further investigations whenever they are in doubt about predictions. To fill this gap, we propose a novel competency-aware NMT by extending conventional NMT with a self-estimator, offering abilities to translate a source sentence and estimate its competency. The self-estimator encodes the information of the decoding procedure and then examines whether it can reconstruct the original semantics of the source sentence. Experimental results on four translation tasks demonstrate that the proposed method not only carries out translation tasks intact but also delivers outstanding performance on quality estimation. Without depending on any reference or annotated data typically required by state-of-the-art metric and quality estimation methods, our model yields an even higher correlation with human quality judgments than a variety of aforementioned methods, such as BLEURT, COMET, and BERTScore. Quantitative and qualitative analyses show better robustness of competency awareness in our model. △ Less

Submitted 24 November, 2022; originally announced November 2022.

Comments: accepted to EMNLP 2022