Showing 1–50 of 131 results for author: Shi, E

Merging Multiple Datasets for Improved Appearance-Based Gaze Estimation

Authors: Liang Wu, Bertram E. Shi

Abstract: Multiple datasets have been created for training and testing appearance-based gaze estimators. Intuitively, more data should lead to better performance. However, combining datasets to train a single esti-mator rarely improves gaze estimation performance. One reason may be differences in the experimental protocols used to obtain the gaze sam-ples, resulting in differences in the distributions of he… ▽ More Multiple datasets have been created for training and testing appearance-based gaze estimators. Intuitively, more data should lead to better performance. However, combining datasets to train a single esti-mator rarely improves gaze estimation performance. One reason may be differences in the experimental protocols used to obtain the gaze sam-ples, resulting in differences in the distributions of head poses, gaze an-gles, illumination, etc. Another reason may be the inconsistency between methods used to define gaze angles (label mismatch). We propose two innovations to improve the performance of gaze estimation by leveraging multiple datasets, a change in the estimator architecture and the intro-duction of a gaze adaptation module. Most state-of-the-art estimators merge information extracted from images of the two eyes and the entire face either in parallel or combine information from the eyes first then with the face. Our proposed Two-stage Transformer-based Gaze-feature Fusion (TTGF) method uses transformers to merge information from each eye and the face separately and then merge across the two eyes. We argue that this improves head pose invariance since changes in head pose affect left and right eye images in different ways. Our proposed Gaze Adaptation Module (GAM) method handles annotation inconsis-tency by applying a Gaze Adaption Module for each dataset to correct gaze estimates from a single shared estimator. This enables us to combine information across datasets despite differences in labeling. Our experi-ments show that these innovations improve gaze estimation performance over the SOTA both individually and collectively (by 10% - 20%). Our code is available at https://github.com/HKUST-NISL/GazeSetMerge. △ Less

Submitted 1 September, 2024; originally announced September 2024.

Comments: 14 pages

Enhancing Depression Diagnosis with Chain-of-Thought Prompting

Authors: Elysia Shi, Adithri Manda, London Chowdhury, Runeema Arun, Kevin Zhu, Michael Lam

Abstract: When using AI to detect signs of depressive disorder, AI models habitually draw preemptive conclusions. We theorize that using chain-of-thought (CoT) prompting to evaluate Patient Health Questionnaire-8 (PHQ-8) scores will improve the accuracy of the scores determined by AI models. In our findings, when the models reasoned with CoT, the estimated PHQ-8 scores were consistently closer on average to… ▽ More When using AI to detect signs of depressive disorder, AI models habitually draw preemptive conclusions. We theorize that using chain-of-thought (CoT) prompting to evaluate Patient Health Questionnaire-8 (PHQ-8) scores will improve the accuracy of the scores determined by AI models. In our findings, when the models reasoned with CoT, the estimated PHQ-8 scores were consistently closer on average to the accepted true scores reported by each participant compared to when not using CoT. Our goal is to expand upon AI models' understanding of the intricacies of human conversation, allowing them to more effectively assess a patient's feelings and tone, therefore being able to more accurately discern mental disorder symptoms; ultimately, we hope to augment AI models' abilities, so that they can be widely accessible and used in the medical field. △ Less

Submitted 27 August, 2024; v1 submitted 26 August, 2024; originally announced August 2024.

Joint SIM Configuration and Power Allocation for Stacked Intelligent Metasurface-assisted MU-MISO Systems with TD3

Authors: Xiaolei Yang, Jiayi Zhang, Enyu Shi, Ziheng Liu, Jun Liu, Kang Zheng, Bo Ai

Abstract: The stacked intelligent metasurface (SIM) emerges as an innovative technology with the ability to directly manipulate electromagnetic (EM) wave signals, drawing parallels to the operational principles of artificial neural networks (ANN). Leveraging its structure for direct EM signal processing alongside its low-power consumption, SIM holds promise for enhancing system performance within wireless c… ▽ More The stacked intelligent metasurface (SIM) emerges as an innovative technology with the ability to directly manipulate electromagnetic (EM) wave signals, drawing parallels to the operational principles of artificial neural networks (ANN). Leveraging its structure for direct EM signal processing alongside its low-power consumption, SIM holds promise for enhancing system performance within wireless communication systems. In this paper, we focus on SIM-assisted multi-user multi-input and single-output (MU-MISO) system downlink scenarios in the transmitter. We proposed a joint optimization method for SIM phase shift configuration and antenna power allocation based on the twin delayed deep deterministic policy gradient (TD3) algorithm to efficiently improve the sum rate. The results show that the proposed algorithm outperforms both deep deterministic policy gradient (DDPG) and alternating optimization (AO) algorithms. Furthermore, increasing the number of meta-atoms per layer of the SIM is always beneficial. However, continuously increasing the number of layers of SIM does not lead to sustained performance improvement. △ Less

Submitted 11 August, 2024; originally announced August 2024.

Comments: accepted by IEEE GLOBECOM 2024

You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search

Authors: Yanlin Wang, Lianghong Guo, Ensheng Shi, Wenqing Chen, Jiachi Chen, Wanjun Zhong, Menghan Wang, Hui Li, Hongyu Zhang, Ziyu Lyu, Zibin Zheng

Abstract: Code search plays a crucial role in software development, enabling developers to retrieve and reuse code using natural language queries. While the performance of code search models improves with an increase in high-quality data, obtaining such data can be challenging and expensive. Recently, large language models (LLMs) such as ChatGPT have made remarkable progress in both natural and programming… ▽ More Code search plays a crucial role in software development, enabling developers to retrieve and reuse code using natural language queries. While the performance of code search models improves with an increase in high-quality data, obtaining such data can be challenging and expensive. Recently, large language models (LLMs) such as ChatGPT have made remarkable progress in both natural and programming language understanding and generation, offering user-friendly interaction via simple prompts. Inspired by these advancements, we propose a novel approach ChatDANCE, which utilizes high-quality and diverse augmented data generated by a large language model and leverages a filtering mechanism to eliminate low-quality augmentations. Specifically, we first propose a set of ChatGPT prompting rules that are specifically designed for source code and queries. Then, we leverage ChatGPT to rewrite code and queries based on the according prompts and then propose a filtering mechanism which trains a cross-encoder from the backbone model UniXcoder to filter out code and query pairs with low matching scores. Finally, we re-train the backbone model using the obtained high-quality augmented data. Experimental results show that ChatDANCE achieves state-of-the-art performance, improving the best baseline by 13.2% (R@1) and 7% (MRR). Surprisingly, we find that this augment-filter-retrain strategy enables the backbone model (UniXcoder) to self-grow. Moreover, extensive experiments show the effectiveness of each component and ChatDANCE has stable performance under different hyperparameter settings. In addition, we conduct qualitative and quantitative analyses to investigate why ChatDANCE works well and find that it learns a more uniform distribution of representations and effectively aligns the code and query spaces. △ Less

Submitted 17 August, 2024; v1 submitted 10 August, 2024; originally announced August 2024.

Comments: Accepted at ICSME 2023

Survey of Design Paradigms for Social Robots

Authors: Rita Frieske, Xiaoyu Mo, Yini Fang, Jay Nieles, Bertram E. Shi

Abstract: The demand for social robots in fields like healthcare, education, and entertainment increases due to their emotional adaptation features. These robots leverage multimodal communication, incorporating speech, facial expressions, and gestures to enhance user engagement and emotional support. The understanding of design paradigms of social robots is obstructed by the complexity of the system and the… ▽ More The demand for social robots in fields like healthcare, education, and entertainment increases due to their emotional adaptation features. These robots leverage multimodal communication, incorporating speech, facial expressions, and gestures to enhance user engagement and emotional support. The understanding of design paradigms of social robots is obstructed by the complexity of the system and the necessity to tune it to a specific task. This article provides a structured review of social robot design paradigms, categorizing them into cognitive architectures, role design models, linguistic models, communication flow, activity system models, and integrated design models. By breaking down the articles on social robot design and application based on these paradigms, we highlight the strengths and areas for improvement in current approaches. We further propose our original integrated design model that combines the most important aspects of the design of social robots. Our approach shows the importance of integrating operational, communicational, and emotional dimensions to create more adaptive and empathetic interactions between robots and humans. △ Less

Submitted 30 July, 2024; originally announced July 2024.

When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention

Authors: Lianghong Guo, Yanlin Wang, Ensheng Shi, Wanjun Zhong, Hongyu Zhang, Jiachi Chen, Ruikai Zhang, Yuchi Ma, Zibin Zheng

Abstract: Code generation aims to automatically generate code snippets that meet given natural language requirements and plays an important role in software development. Although Code LLMs have shown excellent performance in this domain, their long generation time poses a signification limitation in practice use. In this paper, we first conduct an in-depth preliminary study with different Code LLMs on code… ▽ More Code generation aims to automatically generate code snippets that meet given natural language requirements and plays an important role in software development. Although Code LLMs have shown excellent performance in this domain, their long generation time poses a signification limitation in practice use. In this paper, we first conduct an in-depth preliminary study with different Code LLMs on code generation tasks and identify a significant efficiency issue, i.e., continual generation of excess tokens. It harms the developer productivity and leads to huge computational wastes. To address it, we introduce CodeFast, an inference acceleration approach for Code LLMs on code generation. The key idea of CodeFast is to terminate the inference process in time when unnecessary excess tokens are detected. First, we propose an automatic data construction framework to obtain training data. Then, we train a unified lightweight model GenGuard applicable to multiple programming languages to predict whether to terminate inference at the current step. Finally, we enhance Code LLM with GenGuard to accelerate its inference in code generation tasks. We conduct extensive experiments with CodeFast on five representative Code LLMs across four widely used code generation datasets. Experimental results show that (1) CodeFast can significantly improve the inference speed of various Code LLMs in code generation, ranging form 34% to 452%, without compromising the quality of generated code. (2) CodeFast is stable across different parameter settings and can generalize to untrained datasets. Our code and data are available at https://github.com/DeepSoftwareAnalytics/CodeFast △ Less

Submitted 29 July, 2024; originally announced July 2024.

Comments: To appear at ISSTA 2024

ERIT Lightweight Multimodal Dataset for Elderly Emotion Recognition and Multimodal Fusion Evaluation

Authors: Rita Frieske, Bertram E. Shi

Abstract: ERIT is a novel multimodal dataset designed to facilitate research in a lightweight multimodal fusion. It contains text and image data collected from videos of elderly individuals reacting to various situations, as well as seven emotion labels for each data sample. Because of the use of labeled images of elderly users reacting emotionally, it is also facilitating research on emotion recognition in… ▽ More ERIT is a novel multimodal dataset designed to facilitate research in a lightweight multimodal fusion. It contains text and image data collected from videos of elderly individuals reacting to various situations, as well as seven emotion labels for each data sample. Because of the use of labeled images of elderly users reacting emotionally, it is also facilitating research on emotion recognition in an underrepresented age group in machine learning visual emotion recognition. The dataset is validated through comprehensive experiments indicating its importance in neural multimodal fusion research. △ Less

Submitted 25 July, 2024; originally announced July 2024.

$(ω, α, n)$-sensitivity and limit sets of zero entropy homeomorphisms on the square

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: For a homeomorphism $f$ of a compact metric space $X$ and a positive integer $n\geq 2$, we introduce the notion of $(ω, α, n)$-sensitivity of $f$, which describes such a kind of chaos: there is some $c>0$ such that for any $x\in X$ and any open neighborhood $U$ of $x$, there are points $\{x_i\}_{i=1}^n$ and $\{y_i\}_{i=1}^n$ in $U$ such that both the collection of $ω$-limit sets $ω(x_i, f)$ and th… ▽ More For a homeomorphism $f$ of a compact metric space $X$ and a positive integer $n\geq 2$, we introduce the notion of $(ω, α, n)$-sensitivity of $f$, which describes such a kind of chaos: there is some $c>0$ such that for any $x\in X$ and any open neighborhood $U$ of $x$, there are points $\{x_i\}_{i=1}^n$ and $\{y_i\}_{i=1}^n$ in $U$ such that both the collection of $ω$-limit sets $ω(x_i, f)$ and that of the $α$-limit sets $α(y_i, f)$ are pairwise $c$-separated. Then we construct a class of homeomorphisms of the square $[-1, 1]^2$ which are $(ω, α, n)$-sensitive for any $n\geq 2$ and have zero topological entropies. To investigate further the complexity of zero entropy homeomorphisms by using limit sets, we analyze in depth the limit sets of square homeomorphisms by the boundary permeating technique. Specially, we prove that for any given set of points $Y\equiv\{y_{n1}, y_{n2}:n\in\mathbb N\}$ in $(-1, 1)^2$ which satisfies some loosely technical conditions, and for any given family of pairwise disjoint countable dense subsets $\{W_n:n\in\mathbb N\}$ of $(-1, 1)^2-Y$, there is a zero entropy homeomorphism $f$ on the square $[-1, 1]^2$ such that $ω(x, f)=\{y_{n1}\}$ and $α(x, f)=\{y_{n2}\}$ for any $n$ and any $x\in W_n$. △ Less

Submitted 9 July, 2024; originally announced July 2024.

A new construction of counterexamples to the bounded orbit conjecture

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: The bounded orbit conjecture says that every homeomorphism on the plane with each of its orbits being bounded must have a fixed point. Brouwer's translation theorem asserts that the conjecture is true for orientation preserving homeomorphisms, but Boyles' counterexample shows that it is false for the orientation reversing case. In this paper, we give a more comprehensible construction of counterex… ▽ More The bounded orbit conjecture says that every homeomorphism on the plane with each of its orbits being bounded must have a fixed point. Brouwer's translation theorem asserts that the conjecture is true for orientation preserving homeomorphisms, but Boyles' counterexample shows that it is false for the orientation reversing case. In this paper, we give a more comprehensible construction of counterexamples to the conjecture. Roughly speaking, we construct an orientation reversing homeomorphisms $f$ on the square $J^2=[-1, 1]^2$ with $ω(x, f)=\{(-1. 1), (1, 1)\}$ and $α(x, f)=\{(-1. -1), (1, -1)\}$ for each $x\in (-1, 1)^2$. Then by a semi-conjugacy defined by pushing an appropriate part of $\partial J^2$ into $(-1, 1)^2$, $f$ induces a homeomorphism on the plane, which is a counterexample. △ Less

Submitted 24 June, 2024; originally announced June 2024.

The structure of periodic point free distal homeomorphisms on the annulus

Authors: Enhui Shi, Hui Xu, Ziqi YU

Abstract: Let $A$ be an annulus in the plane $\mathbb R^2$ and $g:A\rightarrow A$ be a boundary components preserving homeomorphism which is distal and has no periodic points. Then there is a continuous decomposition of $A$ into $g$-invariant circles such that all the restrictions of $g$ on them share a common irrational rotation number and all these circles are linearly ordered by the inclusion relation on… ▽ More Let $A$ be an annulus in the plane $\mathbb R^2$ and $g:A\rightarrow A$ be a boundary components preserving homeomorphism which is distal and has no periodic points. Then there is a continuous decomposition of $A$ into $g$-invariant circles such that all the restrictions of $g$ on them share a common irrational rotation number and all these circles are linearly ordered by the inclusion relation on the sets of bounded components of their complements in $\mathbb R^2$. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Comments: 18 pages. Comments are welcome

Towards more realistic evaluation of LLM-based code generation: an experimental study and beyond

Authors: Dewu Zheng, Yanlin Wang, Ensheng Shi, Ruikai Zhang, Yuchi Ma, Hongyu Zhang, Zibin Zheng

Abstract: To evaluate the code generation capabilities of Large Language Models (LLMs) in complex real-world software development scenarios, many evaluation approaches have been developed. They typically leverage contextual code from the latest version of a project to facilitate LLMs in accurately generating the desired function. However, such evaluation approaches fail to consider the dynamic evolution of… ▽ More To evaluate the code generation capabilities of Large Language Models (LLMs) in complex real-world software development scenarios, many evaluation approaches have been developed. They typically leverage contextual code from the latest version of a project to facilitate LLMs in accurately generating the desired function. However, such evaluation approaches fail to consider the dynamic evolution of software projects over time, which we refer to as evolving-ignored situation, leading to issues of future context leakage and useful context missing. This in turn results in inaccurate evaluation of LLMs' performance. In this paper, we conduct an empirical study to deeply understand LLMs' code generation performance within settings that reflect the evolving nature of software development. To achieve this, we first construct an evolving-aware repository-level code generation dataset, namely HumanEvo, equipped with an automated execution-based evaluation tool. Second, we manually categorize HumanEvo according to dependency levels to more comprehensively analyze the model's performance in generating functions with different dependency levels. Third, we conduct extensive experiments on HumanEvo with seven representative and diverse LLMs to verify the effectiveness of the proposed benchmark. We obtain many important findings through our experimental study. For example, we find that previous evolving-ignored evaluation approaches lead to inflated performance of the LLMs, ranging from 10.0% to 61.1%. Based on the findings, we give actionable suggestions on more realistic evaluation of LLMs on code generation. We also build a shared evolving-aware code generation toolbox to facilitate future research. Replication package including source code, datasets and appendix is available at https://github.com/DeepSoftwareAnalytics/EvoEval. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Erdős' problem and $(n, \frac{1}{3})$-separated sets

Authors: Enhui Shi, Hui Xu

Abstract: Inspired by the Erdős' problem in Ramsey theory, we propose a dynamical version of the problem and answer it positively for circle maps. Inspired by the Erdős' problem in Ramsey theory, we propose a dynamical version of the problem and answer it positively for circle maps. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Comments: Comments are welcome!

Performance Analysis of RIS-aided MISO Systems with EMI and Channel Aging

Authors: Taoyu Song, Enyu Shi, Yu Lu, Yiyang Zhu, Jiayi Zhang, Bo Ai

Abstract: In this paper, we investigate a reconfigurable intelligent surface (RIS)-aided multiple-input single-output (MISO) system in the presence of electromagnetic interference (EMI) and channel aging with a Rician fading channel model between the base station (BS) and user equipment (UE). Specifically, we derive the closed-form expression for downlink spectral efficiency (SE) with maximum ratio transmis… ▽ More In this paper, we investigate a reconfigurable intelligent surface (RIS)-aided multiple-input single-output (MISO) system in the presence of electromagnetic interference (EMI) and channel aging with a Rician fading channel model between the base station (BS) and user equipment (UE). Specifically, we derive the closed-form expression for downlink spectral efficiency (SE) with maximum ratio transmission (MRT) precoding. The Monte-Carlo simulation supports the theoretical results, demonstrating that amplifying the weight of the line-of-sight (LoS) component in Rician fading channels can boost SE, while EMI has a detrimental impact. Furthermore, continuously increasing the number of RIS elements is not an optimal choice when EMI exists. Nonetheless, RIS can be deployed to compensate for SE degradation caused by channel aging effects. Finally, enlarging the RIS elements size can significantly improve system performance. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Multi-agent Reinforcement Learning-based Joint Precoding and Phase Shift Optimization for RIS-aided Cell-Free Massive MIMO Systems

Authors: Yiyang Zhu, Enyu Shi, Ziheng Liu, Jiayi Zhang, Bo Ai

Abstract: Cell-free (CF) massive multiple-input multiple-output (mMIMO) is a promising technique for achieving high spectral efficiency (SE) using multiple distributed access points (APs). However, harsh propagation environments often lead to significant communication performance degradation due to high penetration loss. To overcome this issue, we introduce the reconfigurable intelligent surface (RIS) into… ▽ More Cell-free (CF) massive multiple-input multiple-output (mMIMO) is a promising technique for achieving high spectral efficiency (SE) using multiple distributed access points (APs). However, harsh propagation environments often lead to significant communication performance degradation due to high penetration loss. To overcome this issue, we introduce the reconfigurable intelligent surface (RIS) into the CF mMIMO system as a low-cost and power-efficient solution. In this paper, we focus on optimizing the joint precoding design of the RIS-aided CF mMIMO system to maximize the sum SE. This involves optimizing the precoding matrix at the APs and the reflection coefficients at the RIS. To tackle this problem, we propose a fully distributed multi-agent reinforcement learning (MARL) algorithm that incorporates fuzzy logic (FL). Unlike conventional approaches that rely on alternating optimization techniques, our FL-based MARL algorithm only requires local channel state information, which reduces the need for high backhaul capacity. Simulation results demonstrate that our proposed FL-MARL algorithm effectively reduces computational complexity while achieving similar performance as conventional MARL methods. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Can points of bounded orbits surround points of unbounded orbits ?

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: We show a somewhat surprising result: if $E$ is a disk in the plane $\mathbb R^2$, then there is a homeomorphism $h:\mathbb R^2\rightarrow\mathbb R^2$ such that, for every $x\in\partial E$, the orbit $O(x, h)$ is bounded, but for every $y\in {\rm Int}(E)$, the orbit $O(y, h)$ is doubly divergent. To prove this, we define a class of homeomorphisms on the square $[-1, 1]^2$, called normally rising h… ▽ More We show a somewhat surprising result: if $E$ is a disk in the plane $\mathbb R^2$, then there is a homeomorphism $h:\mathbb R^2\rightarrow\mathbb R^2$ such that, for every $x\in\partial E$, the orbit $O(x, h)$ is bounded, but for every $y\in {\rm Int}(E)$, the orbit $O(y, h)$ is doubly divergent. To prove this, we define a class of homeomorphisms on the square $[-1, 1]^2$, called normally rising homeomorphisms, and show that a normally rising homeomorphism can have very complex $ω$-limit sets and $α$-limt sets, though the homeomorphism itself looks very simple. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 18 pages. Comments are welcome

MSC Class: 37E30

Some extensions of the Brouwer fixed point theorem

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: We study the existence of fixed points for continuous maps $f$ from an $n$-ball $X$ in $\mathbb R^n$ to $\mathbb R^n$ with $n\geq 1$. We show that $f$ has a fixed point if, for some absolute retract $Y\subset\partial X$, $f(Y)\subset X$ and $\partial X-Y$ is an $(f, X)$-blockading set. For $n\geq 2$, let $D$ be an $n$-ball in $X$ and $Y$ be an $(n-1)$-ball in $\partial X$. Relying on the result ju… ▽ More We study the existence of fixed points for continuous maps $f$ from an $n$-ball $X$ in $\mathbb R^n$ to $\mathbb R^n$ with $n\geq 1$. We show that $f$ has a fixed point if, for some absolute retract $Y\subset\partial X$, $f(Y)\subset X$ and $\partial X-Y$ is an $(f, X)$-blockading set. For $n\geq 2$, let $D$ be an $n$-ball in $X$ and $Y$ be an $(n-1)$-ball in $\partial X$. Relying on the result just mentioned, we show the existence of a fixed point of $f$, if $D$ and $Y$ are well placed and behave well under $f$, and ${\rm deg}(f_D)=-{\rm deg}(f_{\partial Y})$, where $f_D=f|D: D \rightarrow \mathbb{R}^n$ and $f_{\partial Y}=f|\partial Y: \partial Y \rightarrow \partial Y$. The degree ${\rm deg}(f_D)$ of $f_D$ is explicitly defined and some elementary properties of which are investigated. These results extend the Brouwer fixed point theorem. △ Less

Submitted 8 April, 2024; originally announced April 2024.

MSC Class: 55M20; 55M25; 54H20

Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions

Authors: Ziheng Liu, Jiayi Zhang, Enyu Shi, Zhilong Liu, Dusit Niyato, Bo Ai, Xuemin, Shen

Abstract: Multi-agent reinforcement learning (MARL) has become a fundamental component of next-generation wireless communication systems. Theoretically, although MARL has the advantages of low computational complexity and fast convergence rate, there exist several challenges including partial observability, non-stationary, and scalability. In this article, we investigate a novel MARL with graph neural netwo… ▽ More Multi-agent reinforcement learning (MARL) has become a fundamental component of next-generation wireless communication systems. Theoretically, although MARL has the advantages of low computational complexity and fast convergence rate, there exist several challenges including partial observability, non-stationary, and scalability. In this article, we investigate a novel MARL with graph neural network-aided communication (GNNComm-MARL) to address the aforementioned challenges by making use of graph attention networks to effectively sample neighborhoods and selectively aggregate messages. Furthermore, we thoroughly study the architecture of GNNComm-MARL and present a systematic design solution. We then present the typical applications of GNNComm-MARL from two aspects: resource allocation and mobility management. The results obtained unveil that GNNComm-MARL can achieve better performance with lower communication overhead compared to conventional communication schemes. Finally, several important research directions regarding GNNComm-MARL are presented to facilitate further investigation. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Almost Instance-optimal Clipping for Summation Problems in the Shuffle Model of Differential Privacy

Authors: Wei Dong, Qiyao Luo, Giulia Fanti, Elaine Shi, Ke Yi

Abstract: Differentially private mechanisms achieving worst-case optimal error bounds (e.g., the classical Laplace mechanism) are well-studied in the literature. However, when typical data are far from the worst case, \emph{instance-specific} error bounds -- which depend on the largest value in the dataset -- are more meaningful. For example, consider the sum estimation problem, where each user has an integ… ▽ More Differentially private mechanisms achieving worst-case optimal error bounds (e.g., the classical Laplace mechanism) are well-studied in the literature. However, when typical data are far from the worst case, \emph{instance-specific} error bounds -- which depend on the largest value in the dataset -- are more meaningful. For example, consider the sum estimation problem, where each user has an integer $x_i$ from the domain $\{0,1,\dots,U\}$ and we wish to estimate $\sum_i x_i$. This has a worst-case optimal error of $O(U/\varepsilon)$, while recent work has shown that the clipping mechanism can achieve an instance-optimal error of $O(\max_i x_i \cdot \log\log U /\varepsilon)$. Under the shuffle model, known instance-optimal protocols are less communication-efficient. The clipping mechanism also works in the shuffle model, but requires two rounds: Round one finds the clipping threshold, and round two does the clipping and computes the noisy sum of the clipped data. In this paper, we show how these two seemingly sequential steps can be done simultaneously in one round using just $1+o(1)$ messages per user, while maintaining the instance-optimal error bound. We also extend our technique to the high-dimensional sum estimation problem and sparse vector aggregation (a.k.a. frequency estimation under user-level differential privacy). △ Less

Submitted 30 August, 2024; v1 submitted 15 March, 2024; originally announced March 2024.

Quasi-intermediate value theorem and outflanking arc theorem for plane maps

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: For a disk $D$ in the plane $\mathbb R^2$ and a plane map $f$, we give several conditions on the restriction of $f$ to the boundary $\partial D$ of $D$ which imply the existence of a fixed point of $f$ in some specified domain in $D$. These conditions are similar to those appeared in the intermediate value theorem for maps on the real line. As an application of the main results, we establish a fix… ▽ More For a disk $D$ in the plane $\mathbb R^2$ and a plane map $f$, we give several conditions on the restriction of $f$ to the boundary $\partial D$ of $D$ which imply the existence of a fixed point of $f$ in some specified domain in $D$. These conditions are similar to those appeared in the intermediate value theorem for maps on the real line. As an application of the main results, we establish a fixed point theorem for plane maps having an outflanking arc, which extends the famous theorem due to Brouwer: if $f$ is an orientation-preserving homeomorphism on the plane and has a periodic point, then it has a fixed point. △ Less

Submitted 25 February, 2024; originally announced February 2024.

Mechanism Design for Automated Market Makers

Authors: T-H. Hubert Chan, Ke Wu, Elaine Shi

Abstract: Blockchains have popularized automated market makers (AMMs). An AMM exchange is an application running on a blockchain which maintains a pool of crypto-assets and automatically trades assets with users governed by some pricing function that prices the assets based on their relative demand/supply. AMMs have created an important challenge commonly known as the Miner Extractable Value (MEV). In parti… ▽ More Blockchains have popularized automated market makers (AMMs). An AMM exchange is an application running on a blockchain which maintains a pool of crypto-assets and automatically trades assets with users governed by some pricing function that prices the assets based on their relative demand/supply. AMMs have created an important challenge commonly known as the Miner Extractable Value (MEV). In particular, the miners who control the contents and ordering of transactions in a block can extract value by front-running and back-running users' transactions, leading to arbitrage opportunities that guarantee them risk-free returns. In this paper, we consider how to design AMM mechanisms that eliminate MEV opportunities. Specifically, we propose a new AMM mechanism that processes all transactions contained within a block in a batch. We show that our new mechanism satisfies two tiers of guarantees. First, for legacy blockchains where each block is proposed by a single (possibly rotating) miner, we prove that our mechanism satisfies arbitrage resilience, i.e., a miner cannot gain risk-free profit. Moreover, we also guarantee fair treatment among all transactions within the same block, such that the miner is unable to sell off favorable positions in the block to users or arbitragers. Second, for blockchains where the block proposal process is decentralized and offers sequencing-fairness, we prove a stronger notion called incentive compatibility -- roughly speaking, we guarantee that any individual user's best response is to follow the honest strategy. △ Less

Submitted 21 April, 2024; v1 submitted 14 February, 2024; originally announced February 2024.

Comments: 1 title page and 23 pages for the main body

Collusion-Resilience in Transaction Fee Mechanism Design

Authors: Hao Chung, Tim Roughgarden, Elaine Shi

Abstract: Users bid in a transaction fee mechanism (TFM) to get their transactions included and confirmed by a blockchain protocol. Roughgarden (EC'21) initiated the formal treatment of TFMs and proposed three requirements: user incentive compatibility (UIC), miner incentive compatibility (MIC), and a form of collusion-resilience called OCA-proofness. Ethereum's EIP-1559 mechanism satisfies all three proper… ▽ More Users bid in a transaction fee mechanism (TFM) to get their transactions included and confirmed by a blockchain protocol. Roughgarden (EC'21) initiated the formal treatment of TFMs and proposed three requirements: user incentive compatibility (UIC), miner incentive compatibility (MIC), and a form of collusion-resilience called OCA-proofness. Ethereum's EIP-1559 mechanism satisfies all three properties simultaneously when there is no contention between transactions, but loses the UIC property when there are too many eligible transactions to fit in a single block. Chung and Shi (SODA'23) considered an alternative notion of collusion-resilience, called c-side-contract-proofness (c-SCP), and showed that, when there is contention between transactions, no TFM can satisfy UIC, MIC, and c-SCP for any c at least 1. OCA-proofness asserts that the users and a miner should not be able to "steal from the protocol." On the other hand, the c-SCP condition requires that a coalition of a miner and a subset of users should not be able to profit through strategic deviations (whether at the expense of the protocol or of the users outside the coalition). Our main result is the first proof that, when there is contention between transactions, no (possibly randomized) TFM in which users are expected to bid truthfully satisfies UIC, MIC, and OCA-proofness. This result resolves the main open question in Roughgarden (EC'21). We also suggest several relaxations of the basic model that allow our impossibility result to be circumvented. △ Less

Submitted 19 June, 2024; v1 submitted 14 February, 2024; originally announced February 2024.

Large Language Models for Robotics: Opportunities, Challenges, and Perspectives

Authors: Jiaqi Wang, Zihao Wu, Yiwei Li, Hanqi Jiang, Peng Shu, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Huaqin Zhao, Zhengliang Liu, Haixing Dai, Lin Zhao, Bao Ge, Xiang Li, Tianming Liu, Shu Zhang

Abstract: Large language models (LLMs) have undergone significant expansion and have been increasingly integrated across various domains. Notably, in the realm of robot task planning, LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions. However, for embodied tasks, where robots interact with comp… ▽ More Large language models (LLMs) have undergone significant expansion and have been increasingly integrated across various domains. Notably, in the realm of robot task planning, LLMs harness their advanced reasoning and language comprehension capabilities to formulate precise and efficient action plans based on natural language instructions. However, for embodied tasks, where robots interact with complex environments, text-only LLMs often face challenges due to a lack of compatibility with robotic visual perception. This study provides a comprehensive overview of the emerging integration of LLMs and multimodal LLMs into various robotic tasks. Additionally, we propose a framework that utilizes multimodal GPT-4V to enhance embodied task planning through the combination of natural language instructions and robot visual perceptions. Our results, based on diverse datasets, indicate that GPT-4V effectively enhances robot performance in embodied tasks. This extensive survey and evaluation of LLMs and multimodal LLMs across a variety of robotic tasks enriches the understanding of LLM-centric embodied intelligence and provides forward-looking insights toward bridging the gap in Human-Robot-Environment interaction. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models

Authors: Rita Frieske, Bertram E. Shi

Abstract: Hallucinations are a type of output error produced by deep neural networks. While this has been studied in natural language processing, they have not been researched previously in automatic speech recognition. Here, we define hallucinations in ASR as transcriptions generated by a model that are semantically unrelated to the source utterance, yet still fluent and coherent. The similarity of halluci… ▽ More Hallucinations are a type of output error produced by deep neural networks. While this has been studied in natural language processing, they have not been researched previously in automatic speech recognition. Here, we define hallucinations in ASR as transcriptions generated by a model that are semantically unrelated to the source utterance, yet still fluent and coherent. The similarity of hallucinations to probable natural language outputs of the model creates a danger of deception and impacts the credibility of the system. We show that commonly used metrics, such as word error rates, cannot differentiate between hallucinatory and non-hallucinatory models. To address this, we propose a perturbation-based method for assessing the susceptibility of an automatic speech recognition (ASR) model to hallucination at test time, which does not require access to the training dataset. We demonstrate that this method helps to distinguish between hallucinatory and non-hallucinatory models that have similar baseline word error rates. We further explore the relationship between the types of ASR errors and the types of dataset noise to determine what types of noise are most likely to create hallucinatory outputs. We devise a framework for identifying hallucinations by analysing their semantic connection with the ground truth and their fluency. Finally, we discover how to induce hallucinations with a random noise injection to the utterance. △ Less

Submitted 3 January, 2024; originally announced January 2024.

Connected Components in Linear Work and Near-Optimal Time

Authors: Alireza Farhadi, S. Cliff Liu, Elaine Shi

Abstract: Computing the connected components of a graph is a fundamental problem in algorithmic graph theory. A major question in this area is whether we can compute connected components in $o(\log n)$ parallel time. Recent works showed an affirmative answer in the Massively Parallel Computation (MPC) model for a wide class of graphs. Specifically, Behnezhad et al. (FOCS'19) showed that connected components… ▽ More Computing the connected components of a graph is a fundamental problem in algorithmic graph theory. A major question in this area is whether we can compute connected components in $o(\log n)$ parallel time. Recent works showed an affirmative answer in the Massively Parallel Computation (MPC) model for a wide class of graphs. Specifically, Behnezhad et al. (FOCS'19) showed that connected components can be computed in $O(\log d + \log \log n)$ rounds in the MPC model. More recently, Liu et al. (SPAA'20) showed that the same result can be achieved in the standard PRAM model but their result incurs $Θ((m+n) \cdot (\log d + \log \log n))$ work which is sub-optimal. In this paper, we show that for graphs that contain \emph{well-connected} components, we can compute connected components on a PRAM in sub-logarithmic parallel time with \emph{optimal}, i.e., $O(m+n)$ total work. Specifically, our algorithm achieves $O(\log(1/λ) + \log \log n)$ parallel time with high probability, where $λ$ is the minimum spectral gap of any connected component in the input graph. The algorithm requires no prior knowledge on $λ$. Additionally, based on the \textsc{2-Cycle} Conjecture we provide a time lower bound of $Ω(\log(1/λ))$ for solving connected components on a PRAM with $O(m+n)$ total memory when $λ\le (1/\log n)^c$, giving conditional optimality to the running time of our algorithm as a parameter of $λ$. △ Less

Submitted 20 May, 2024; v1 submitted 4 December, 2023; originally announced December 2023.

Structures of $R(f)-\overline{P(f)}$ for graph maps $f$

Authors: Jiehua Mai, Enhui Shi, Kesong Yan, Fanping Zeng

Abstract: Let $G$ be a graph and $f: G\rightarrow G$ be a continuous map. We establish a structure theorem which describes the structures of the set $R(f)-\overline{P(f)}$, where $R(f)$ and $P(f)$ are the recurrent point set and the periodic point set of $f$ respectively. Roughly speaking, the set $R(f)-\overline{P(f)}$ is covered by finitely many pairwise disjoint $f$-invariant open sets… ▽ More Let $G$ be a graph and $f: G\rightarrow G$ be a continuous map. We establish a structure theorem which describes the structures of the set $R(f)-\overline{P(f)}$, where $R(f)$ and $P(f)$ are the recurrent point set and the periodic point set of $f$ respectively. Roughly speaking, the set $R(f)-\overline{P(f)}$ is covered by finitely many pairwise disjoint $f$-invariant open sets $U_{1\,},\,\cdots,\,U_{n\,}$; each $U_i$ contains a unique minimal set $W_i$ which absorbs each point of $U_i$; each $W_i$ lies in finitely many pairwise disjoint circles each of which is contained in a connected closed set; all of these connected closed sets are contained in $U_i$ and permutated cyclically by $f$. As applications of the structure theorem, several known results are improved or reproved. △ Less

Submitted 29 October, 2023; originally announced October 2023.

MSC Class: 37E25 (Primary); 37B20; 37C25; 54H20 (Secondary)

RIS-Aided Cell-Free Massive MIMO Systems for 6G: Fundamentals, System Design, and Applications

Authors: Enyu Shi, Jiayi Zhang, Hongyang Du, Bo Ai, Chau Yuen, Dusit Niyato, Khaled B. Letaief, Xuemin Shen

Abstract: An introduction of intelligent interconnectivity for people and things has posed higher demands and more challenges for sixth-generation (6G) networks, such as high spectral efficiency and energy efficiency, ultra-low latency, and ultra-high reliability. Cell-free (CF) massive multiple-input multiple-output (mMIMO) and reconfigurable intelligent surface (RIS), also called intelligent reflecting su… ▽ More An introduction of intelligent interconnectivity for people and things has posed higher demands and more challenges for sixth-generation (6G) networks, such as high spectral efficiency and energy efficiency, ultra-low latency, and ultra-high reliability. Cell-free (CF) massive multiple-input multiple-output (mMIMO) and reconfigurable intelligent surface (RIS), also called intelligent reflecting surface (IRS), are two promising technologies for coping with these unprecedented demands. Given their distinct capabilities, integrating the two technologies to further enhance wireless network performances has received great research and development attention. In this paper, we provide a comprehensive survey of research on RIS-aided CF mMIMO wireless communication systems. We first introduce system models focusing on system architecture and application scenarios, channel models, and communication protocols. Subsequently, we summarize the relevant studies on system operation and resource allocation, providing in-depth analyses and discussions. Following this, we present practical challenges faced by RIS-aided CF mMIMO systems, particularly those introduced by RIS, such as hardware impairments and electromagnetic interference. We summarize corresponding analyses and solutions to further facilitate the implementation of RIS-aided CF mMIMO systems. Furthermore, we explore an interplay between RIS-aided CF mMIMO and other emerging 6G technologies, such as next-generation multiple-access (NGMA), simultaneous wireless information and power transfer (SWIPT), and millimeter wave (mmWave). Finally, we outline several research directions for future RIS-aided CF mMIMO systems. △ Less

Submitted 22 May, 2024; v1 submitted 30 September, 2023; originally announced October 2023.

Comments: Proceedings of the IEEE, Accept, 2024

SoTaNa: The Open-Source Software Development Assistant

Authors: Ensheng Shi, Fengji Zhang, Yanlin Wang, Bei Chen, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Hongbin Sun

Abstract: Software development plays a crucial role in driving innovation and efficiency across modern societies. To meet the demands of this dynamic field, there is a growing need for an effective software development assistant. However, existing large language models represented by ChatGPT suffer from limited accessibility, including training data and model weights. Although other large open-source models… ▽ More Software development plays a crucial role in driving innovation and efficiency across modern societies. To meet the demands of this dynamic field, there is a growing need for an effective software development assistant. However, existing large language models represented by ChatGPT suffer from limited accessibility, including training data and model weights. Although other large open-source models like LLaMA have shown promise, they still struggle with understanding human intent. In this paper, we present SoTaNa, an open-source software development assistant. SoTaNa utilizes ChatGPT to generate high-quality instruction-based data for the domain of software engineering and employs a parameter-efficient fine-tuning approach to enhance the open-source foundation model, LLaMA. We evaluate the effectiveness of \our{} in answering Stack Overflow questions and demonstrate its capabilities. Additionally, we discuss its capabilities in code summarization and generation, as well as the impact of varying the volume of generated data on model performance. Notably, SoTaNa can run on a single GPU, making it accessible to a broader range of researchers. Our code, model weights, and data are public at \url{https://github.com/DeepSoftwareAnalytics/SoTaNa}. △ Less

Submitted 25 August, 2023; originally announced August 2023.

SAMAug: Point Prompt Augmentation for Segment Anything Model

Authors: Haixing Dai, Chong Ma, Zhiling Yan, Zhengliang Liu, Enze Shi, Yiwei Li, Peng Shu, Xiaozheng Wei, Lin Zhao, Zihao Wu, Fang Zeng, Dajiang Zhu, Wei Liu, Quanzheng Li, Lichao Sun, Shu Zhang Tianming Liu, Xiang Li

Abstract: This paper introduces SAMAug, a novel visual point augmentation method for the Segment Anything Model (SAM) that enhances interactive image segmentation performance. SAMAug generates augmented point prompts to provide more information about the user's intention to SAM. Starting with an initial point prompt, SAM produces an initial mask, which is then fed into our proposed SAMAug to generate augmen… ▽ More This paper introduces SAMAug, a novel visual point augmentation method for the Segment Anything Model (SAM) that enhances interactive image segmentation performance. SAMAug generates augmented point prompts to provide more information about the user's intention to SAM. Starting with an initial point prompt, SAM produces an initial mask, which is then fed into our proposed SAMAug to generate augmented point prompts. By incorporating these extra points, SAM can generate augmented segmentation masks based on both the augmented point prompts and the initial prompt, resulting in improved segmentation performance. We conducted evaluations using four different point augmentation strategies: random sampling, sampling based on maximum difference entropy, maximum distance, and saliency. Experiment results on the COCO, Fundus, COVID QUEx, and ISIC2018 datasets show that SAMAug can boost SAM's segmentation results, especially using the maximum distance and saliency. SAMAug demonstrates the potential of visual prompt augmentation for computer vision. Codes of SAMAug are available at github.com/yhydhx/SAMAug △ Less

Submitted 19 March, 2024; v1 submitted 3 July, 2023; originally announced July 2023.

Review of Large Vision Models and Visual Prompt Engineering

Authors: Jiaqi Wang, Zhengliang Liu, Lin Zhao, Zihao Wu, Chong Ma, Sigang Yu, Haixing Dai, Qiushi Yang, Yiheng Liu, Songyao Zhang, Enze Shi, Yi Pan, Tuo Zhang, Dajiang Zhu, Xiang Li, Xi Jiang, Bao Ge, Yixuan Yuan, Dinggang Shen, Tianming Liu, Shu Zhang

Abstract: Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilities. As the development of large vision models progresses, the importance of prompt engineering becomes increasingly evident. Designing suitable prompts for specific visual tasks has emerged as a meaningful research dire… ▽ More Visual prompt engineering is a fundamental technology in the field of visual and image Artificial General Intelligence, serving as a key component for achieving zero-shot capabilities. As the development of large vision models progresses, the importance of prompt engineering becomes increasingly evident. Designing suitable prompts for specific visual tasks has emerged as a meaningful research direction. This review aims to summarize the methods employed in the computer vision domain for large vision models and visual prompt engineering, exploring the latest advancements in visual prompt engineering. We present influential large models in the visual domain and a range of prompt engineering methods employed on these models. It is our hope that this review provides a comprehensive and systematic description of prompt engineering methods based on large visual models, offering valuable insights for future researchers in their exploration of this field. △ Less

Submitted 3 July, 2023; originally announced July 2023.

Uplink Performance of RIS-aided Cell-Free Massive MIMO System with Electromagnetic Interference

Authors: Enyu Shi, Jiayi Zhang, Derrick Wing Kwan Ng, Bo Ai

Abstract: Cell-free (CF) massive multiple-input multiple-output (MIMO) and reconfigurable intelligent surface (RIS) are two promising technologies for realizing future beyond-fifth generation (B5G) networks. In this paper, we consider a practical spatially correlated RIS-aided CF massive MIMO system with multi-antenna access points (APs) over spatially correlated fading channels. Different from previous wor… ▽ More Cell-free (CF) massive multiple-input multiple-output (MIMO) and reconfigurable intelligent surface (RIS) are two promising technologies for realizing future beyond-fifth generation (B5G) networks. In this paper, we consider a practical spatially correlated RIS-aided CF massive MIMO system with multi-antenna access points (APs) over spatially correlated fading channels. Different from previous work, the electromagnetic interference (EMI) at RIS is considered to further characterize the system performance of the actual environment. Then, we derive the closed-form expression for the system spectral efficiency (SE) with the maximum ratio (MR) combining at the APs and the large-scale fading decoding (LSFD) at the central processing unit (CPU). Moreover, to counteract the near-far effect and EMI, we propose practical fractional power control (FPC) and max-min power control algorithms to further improve the system performance. We unveil the impact of EMI, channel correlations, and different signal processing methods on the uplink SE of user equipments (UEs). The accuracy of our derived analytical results is verified by extensive Monte-Carlo simulations. Our results show that the EMI can substantially degrade the SE, especially for those UEs with unsatisfactory channel conditions. Besides, increasing the number of RIS elements is always beneficial in terms of the SE, but with diminishing returns when the number of RIS elements is sufficiently large. Furthermore, the existence of spatial correlations among RIS elements can deteriorate the system performance when RIS is impaired by EMI. △ Less

Submitted 14 June, 2023; originally announced June 2023.

Comments: to appear in IEEE Journal on Selected Areas in Communications

Prompt Engineering for Healthcare: Methodologies and Applications

Authors: Jiaqi Wang, Enze Shi, Sigang Yu, Zihao Wu, Chong Ma, Haixing Dai, Qiushi Yang, Yanqing Kang, Jinru Wu, Huawen Hu, Chenxi Yue, Haiyang Zhang, Yiheng Liu, Yi Pan, Zhengliang Liu, Lichao Sun, Xiang Li, Bao Ge, Xi Jiang, Dajiang Zhu, Yixuan Yuan, Dinggang Shen, Tianming Liu, Shu Zhang

Abstract: Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important… ▽ More Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important in the healthcare domain. However, there is a lack of comprehensive reviews specifically focusing on prompt engineering in the medical field. This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. First, we will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications such as question-answering systems, text summarization, and machine translation. With the continuous improvement of general large language models, the importance of prompt engineering in the healthcare domain is becoming increasingly prominent. The aim of this article is to provide useful resources and bridges for healthcare natural language processing researchers to better explore the application of prompt engineering in this field. We hope that this review can provide new ideas and inspire for research and application in medical natural language processing. △ Less

Submitted 23 March, 2024; v1 submitted 28 April, 2023; originally announced April 2023.

Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond

Authors: Ensheng Shi, Yanlin Wang, Hongyu Zhang, Lun Du, Shi Han, Dongmei Zhang, Hongbin Sun

Abstract: Recently, fine-tuning pre-trained code models such as CodeBERT on downstream tasks has achieved great success in many software testing and analysis tasks. While effective and prevalent, fine-tuning the pre-trained parameters incurs a large computational cost. In this paper, we conduct an extensive experimental study to explore what happens to layer-wise pre-trained representations and their encode… ▽ More Recently, fine-tuning pre-trained code models such as CodeBERT on downstream tasks has achieved great success in many software testing and analysis tasks. While effective and prevalent, fine-tuning the pre-trained parameters incurs a large computational cost. In this paper, we conduct an extensive experimental study to explore what happens to layer-wise pre-trained representations and their encoded code knowledge during fine-tuning. We then propose efficient alternatives to fine-tune the large pre-trained code model based on the above findings. Our experimental study shows that (1) lexical, syntactic and structural properties of source code are encoded in the lower, intermediate, and higher layers, respectively, while the semantic property spans across the entire model. (2) The process of fine-tuning preserves most of the code properties. Specifically, the basic code properties captured by lower and intermediate layers are still preserved during fine-tuning. Furthermore, we find that only the representations of the top two layers change most during fine-tuning for various downstream tasks. (3) Based on the above findings, we propose Telly to efficiently fine-tune pre-trained code models via layer freezing. The extensive experimental results on five various downstream tasks demonstrate that training parameters and the corresponding time cost are greatly reduced, while performances are similar or better. Replication package including source code, datasets, and online Appendix is available at: \url{https://github.com/DeepSoftwareAnalytics/Telly}. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Comments: Accepted by ISSTA 2023 (The 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis)

Giant room-temperature nonlinearities from a monolayer Janus topological semiconductor

Authors: Jiaojian Shi, Haowei Xu, Christian Heide, Changan HuangFu, Chenyi Xia, Felipe de Quesada, Hongzhi Shen, Tianyi Zhang, Leo Yu, Amalya Johnson, Fang Liu, Enzheng Shi, Liying Jiao, Tony Heinz, Shambhu Ghimire, Ju Li, Jing Kong, Yunfan Guo, Aaron M. Lindenberg

Abstract: Nonlinear optical materials possess wide applications, ranging from terahertz and mid-infrared detection to energy harvesting. Recently, the correlations between nonlinear optical responses and topological properties, such as Berry curvature and the quantum metric tensor, have stimulated great interest. Here, we report giant room-temperature nonlinearities in an emergent non-centrosymmetric two-di… ▽ More Nonlinear optical materials possess wide applications, ranging from terahertz and mid-infrared detection to energy harvesting. Recently, the correlations between nonlinear optical responses and topological properties, such as Berry curvature and the quantum metric tensor, have stimulated great interest. Here, we report giant room-temperature nonlinearities in an emergent non-centrosymmetric two-dimensional topological material, the Janus transition metal dichalcogenides in the 1T' phase, which are synthesized by an advanced atomic-layer substitution method. High harmonic generation, terahertz emission spectroscopy, and second harmonic generation measurements consistently reveal orders-of-the-magnitude enhancement in terahertz-frequency nonlinearities of 1T' MoSSe (e.g., > 50 times higher than 2H MoS$_2$ for 18th order harmonic generation; > 20 times higher than 2H MoS$_2$ for terahertz emission). It is elucidated that such colossal nonlinear optical responses come from topological band mixing and strong inversion symmetry breaking due to the Janus structure. Our work defines general protocols for designing materials with large nonlinearities and preludes the applications of topological materials in optoelectronics down to the monolayer limit. This two-dimensional form of topological materials also constitute a unique platform for examining origin of the anomalous high-harmonic generation, with potential applications as building blocks for scalable attosecond sources. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: 22 pages, 4 figures

Maximizing Miner Revenue in Transaction Fee Mechanism Design

Authors: Ke Wu, Elaine Shi, Hao Chung

Abstract: Transaction fee mechanism design is a new decentralized mechanism design problem where users bid for space on the blockchain. Several recent works showed that the transaction fee mechanism design fundamentally departs from classical mechanism design. They then systematically explored the mathematical landscape of this new decentralized mechanism design problem in two settings: in the plain setting… ▽ More Transaction fee mechanism design is a new decentralized mechanism design problem where users bid for space on the blockchain. Several recent works showed that the transaction fee mechanism design fundamentally departs from classical mechanism design. They then systematically explored the mathematical landscape of this new decentralized mechanism design problem in two settings: in the plain setting where no cryptography is employed, and in a cryptography-assisted setting where the rules of the mechanism are enforced by a multi-party computation protocol. Unfortunately, in both settings, prior works showed that if we want the mechanism to incentivize honest behavior for both users as well as miners (possibly colluding with users), then the miner revenue has to be zero. Although adopting a relaxed, approximate notion of incentive compatibility gets around this zero miner-revenue limitation, the scaling of the miner revenue is nonetheless poor. In this paper, we show that if we make a mildly stronger reasonable-world assumption than prior works, we can circumvent the known limitations on miner revenue, and design auctions that generate optimal miner revenue. We also systematically explore the mathematical landscape of transaction fee mechanism design under the new reasonable-world and demonstrate how such assumptions can alter the feasibility and infeasibility landscape. △ Less

Submitted 21 April, 2024; v1 submitted 24 February, 2023; originally announced February 2023.

Scalable inference in functional linear regression with streaming data

Authors: Jinhan Xie, Enze Shi, Peijun Sang, Zuofeng Shang, Bei Jiang, Linglong Kong

Abstract: Traditional static functional data analysis is facing new challenges due to streaming data, where data constantly flow in. A major challenge is that storing such an ever-increasing amount of data in memory is nearly impossible. In addition, existing inferential tools in online learning are mainly developed for finite-dimensional problems, while inference methods for functional data are focused on… ▽ More Traditional static functional data analysis is facing new challenges due to streaming data, where data constantly flow in. A major challenge is that storing such an ever-increasing amount of data in memory is nearly impossible. In addition, existing inferential tools in online learning are mainly developed for finite-dimensional problems, while inference methods for functional data are focused on the batch learning setting. In this paper, we tackle these issues by developing functional stochastic gradient descent algorithms and proposing an online bootstrap resampling procedure to systematically study the inference problem for functional linear regression. In particular, the proposed estimation and inference procedures use only one pass over the data; thus they are easy to implement and suitable to the situation where data arrive in a streaming manner. Furthermore, we establish the convergence rate as well as the asymptotic distribution of the proposed estimator. Meanwhile, the proposed perturbed estimator from the bootstrap procedure is shown to enjoy the same theoretical properties, which provide the theoretical justification for our online inference tool. As far as we know, this is the first inference result on the functional linear regression model with streaming data. Simulation studies are conducted to investigate the finite-sample performance of the proposed procedure. An application is illustrated with the Beijing multi-site air-quality data. △ Less

Submitted 10 October, 2023; v1 submitted 5 February, 2023; originally announced February 2023.

Comments: Due to the request of one of the co-authors, we tentatively withdrew the manuscript

XCRYPT: Accelerating Lattice Based Cryptography with Memristor Crossbar Arrays

Authors: Sarabjeet Singh, Xiong Fan, Ananth Krishna Prasad, Lin Jia, Anirban Nag, Rajeev Balasubramonian, Mahdi Nazm Bojnordi, Elaine Shi

Abstract: This paper makes a case for accelerating lattice-based post quantum cryptography (PQC) with memristor based crossbars, and shows that these inherently error-tolerant algorithms are a good fit for noisy analog MAC operations in crossbars. We compare different NIST round-3 lattice-based candidates for PQC, and identify that SABER is not only a front-runner when executing on traditional systems, but… ▽ More This paper makes a case for accelerating lattice-based post quantum cryptography (PQC) with memristor based crossbars, and shows that these inherently error-tolerant algorithms are a good fit for noisy analog MAC operations in crossbars. We compare different NIST round-3 lattice-based candidates for PQC, and identify that SABER is not only a front-runner when executing on traditional systems, but it is also amenable to acceleration with crossbars. SABER is a module-LWR based approach, which performs modular polynomial multiplications with rounding. We map the polynomial multiplications in SABER on crossbars and show that analog dot-products can yield a $1.7-32.5\times$ performance and energy efficiency improvement, compared to recent hardware proposals. This initial design combines the innovations in multiple state-of-the-art works -- the algorithm in SABER and the memristive acceleration principles proposed in ISAAC (for deep neural network acceleration). We then identify the bottlenecks in this initial design and introduce several additional techniques to improve its efficiency. These techniques are synergistic and especially benefit from SABER's power-of-two modulo operation. First, we show that some of the software techniques used in SABER, that are effective on CPU platforms, are unhelpful in crossbar-based accelerators. Relying on simpler algorithms further improves our efficiencies by $1.3-3.6\times$. Second, we exploit the nature of SABER's computations to stagger the operations in crossbars and share a few variable precision ADCs, resulting in up to $1.8\times$ higher efficiency. Third, to further reduce ADC pressure, we propose a simple analog Shift-and-Add technique, which results in a $1.3-6.3\times$ increase in the efficiency. Overall, our designs achieve $3-15\times$ higher efficiency over initial design, and $3-51\times$ higher than prior work. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Groups acting distally and minimally on $\mathbb S^2$ and $\mathbb {RP}^2$

Authors: Enhui Shi, Hui Xu

Abstract: Let $X$ be the $2$-sphere $\mathbb S^2$ or the real projective plane $\mathbb {RP}^2$. We show that if $Γ$ is a finitely generated group acting minimally and distally on $X$, then $Γ$ contains a nonabelian free subgroup. Let $X$ be the $2$-sphere $\mathbb S^2$ or the real projective plane $\mathbb {RP}^2$. We show that if $Γ$ is a finitely generated group acting minimally and distally on $X$, then $Γ$ contains a nonabelian free subgroup. △ Less

Submitted 13 January, 2023; originally announced January 2023.

Adore: Differentially Oblivious Relational Database Operators

Authors: Lianke Qin, Rajesh Jayaram, Elaine Shi, Zhao Song, Danyang Zhuo, Shumo Chu

Abstract: There has been a recent effort in applying differential privacy on memory access patterns to enhance data privacy. This is called differential obliviousness. Differential obliviousness is a promising direction because it provides a principled trade-off between performance and desired level of privacy. To date, it is still an open question whether differential obliviousness can speed up database pr… ▽ More There has been a recent effort in applying differential privacy on memory access patterns to enhance data privacy. This is called differential obliviousness. Differential obliviousness is a promising direction because it provides a principled trade-off between performance and desired level of privacy. To date, it is still an open question whether differential obliviousness can speed up database processing with respect to full obliviousness. In this paper, we present the design and implementation of three new major database operators: selection with projection, grouping with aggregation, and foreign key join. We prove that they satisfy the notion of differential obliviousness. Our differentially oblivious operators have reduced cache complexity, runtime complexity, and output size compared to their state-of-the-art fully oblivious counterparts. We also demonstrate that our implementation of these differentially oblivious operators can outperform their state-of-the-art fully oblivious counterparts by up to $7.4\times$. △ Less

Submitted 29 September, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: VLDB 2023

The structures of pointwise recurrent quasi-graph maps

Authors: Ziqi Yu, Suhua Wang, Enhui Shi

Abstract: We show that a continuous map $f$ from a quasi-graph $G$ to itself is pointwise recurrent if and only if one of the following two statements holds: (1) $X$ is a simple closed curve and $f$ is topologically conjugate to an irrational rotation on the unit circle $\mathbb S^1$; (2) $f$ is a perodic homeomorphism. We show that a continuous map $f$ from a quasi-graph $G$ to itself is pointwise recurrent if and only if one of the following two statements holds: (1) $X$ is a simple closed curve and $f$ is topologically conjugate to an irrational rotation on the unit circle $\mathbb S^1$; (2) $f$ is a perodic homeomorphism. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Efficient Privacy-Preserving Machine Learning with Lightweight Trusted Hardware

Authors: Pengzhi Huang, Thang Hoang, Yueying Li, Elaine Shi, G. Edward Suh

Abstract: In this paper, we propose a new secure machine learning inference platform assisted by a small dedicated security processor, which will be easier to protect and deploy compared to today's TEEs integrated into high-performance processors. Our platform provides three main advantages over the state-of-the-art: (i) We achieve significant performance improvements compared to state-of-the-art distribu… ▽ More In this paper, we propose a new secure machine learning inference platform assisted by a small dedicated security processor, which will be easier to protect and deploy compared to today's TEEs integrated into high-performance processors. Our platform provides three main advantages over the state-of-the-art: (i) We achieve significant performance improvements compared to state-of-the-art distributed Privacy-Preserving Machine Learning (PPML) protocols, with only a small security processor that is comparable to a discrete security chip such as the Trusted Platform Module (TPM) or on-chip security subsystems in SoCs similar to the Apple enclave processor. In the semi-honest setting with WAN/GPU, our scheme is 4X-63X faster than Falcon (PoPETs'21) and AriaNN (PoPETs'22) and 3.8X-12X more communication efficient. We achieve even higher performance improvements in the malicious setting. (ii) Our platform guarantees security with abort against malicious adversaries under honest majority assumption. (iii) Our technique is not limited by the size of secure memory in a TEE and can support high-capacity modern neural networks like ResNet18 and Transformer. While previous work investigated the use of high-performance TEEs in PPML, this work represents the first to show that even tiny secure hardware with really limited performance can be leveraged to significantly speed-up distributed PPML protocols if the protocol can be carefully designed for lightweight trusted hardware. △ Less

Submitted 17 June, 2024; v1 submitted 18 October, 2022; originally announced October 2022.

Comments: IEEE S&P'24 submitted

Reducing Stress and Anxiety in the Metaverse: A Systematic Review of Meditation, Mindfulness and Virtual Reality

Authors: Xian Wang, Xiaoyu Mo, Mingming Fan, Lik-Hang Lee, Bertram E. Shi, Pan Hui

Abstract: Meditation, or mindfulness, is widely used to improve mental health. With the emergence of Virtual Reality technology, many studies have provided evidence that meditation with VR can bring health benefits. However, to our knowledge, there are no guidelines and comprehensive reviews in the literature on how to conduct such research in virtual reality. In order to understand the role of VR technolog… ▽ More Meditation, or mindfulness, is widely used to improve mental health. With the emergence of Virtual Reality technology, many studies have provided evidence that meditation with VR can bring health benefits. However, to our knowledge, there are no guidelines and comprehensive reviews in the literature on how to conduct such research in virtual reality. In order to understand the role of VR technology in meditation and future research opportunities, we conducted a systematic literature review in the IEEE and ACM databases. Our process yielded 19 eligible papers and we conducted a structured analysis. We understand the state-of-art of meditation type, design consideration and VR and technology through these papers and conclude research opportunities and challenges for the future. △ Less

Submitted 29 September, 2022; originally announced September 2022.

Decentralized Coordinated Precoding Design in Cell-Free Massive MIMO Systems for URLLC

Authors: Enyu Shi, Jing Zhang, Jiayi Zhang, Derrick Wing Kwan Ng, Bo Ai

Abstract: Cell-free massive multiple-input multiple-output (MIMO) is a promising network to offer huge improvement of the achievable rate compared with conventional cellular massive MIMO systems. However, the commonly adopted Shannon-type achievable rate is only valid in the long block length regime that is not applicable to the emerging short-packet communication. To realize ultra-reliable and low-latency… ▽ More Cell-free massive multiple-input multiple-output (MIMO) is a promising network to offer huge improvement of the achievable rate compared with conventional cellular massive MIMO systems. However, the commonly adopted Shannon-type achievable rate is only valid in the long block length regime that is not applicable to the emerging short-packet communication. To realize ultra-reliable and low-latency communication (URLLC) in cell-free massive MIMO systems, we optimize the precoding vector at the access points (APs) to maximize the minimum user rate in both the centralized and decentralized fashion. The design takes into account the impact of URLLC and we propose path-following algorithms (PFA) to address the considered problem which generates a sequence of advanced feasible points and converges to at least a locally optimal solution of the design problem. Moreover, we investigate the requirement of the precoding schemes, the length of the transmission duration, the number of antennas equipped at each AP, and the size of each AP cluster on the URLLC rate. Numerical results show that the decentralized PFA precoding can achieve 80\% of the 95\%-likely URLLC rate of the centralized precoding and 89\% of the average URLLC rate with only 12\% computational complexity of the centralized precoding. △ Less

Submitted 28 September, 2022; originally announced September 2022.

Comments: 6 pages,3 figures

What Can Cryptography Do For Decentralized Mechanism Design

Authors: Elaine Shi, Hao Chung, Ke Wu

Abstract: Recent works of Roughgarden (EC'21) and Chung and Shi (SODA'23) initiate the study of a new decentralized mechanism design problem called transaction fee mechanism design (TFM). Unlike the classical mechanism design literature, in the decentralized environment, even the auctioneer (i.e., the miner) can be a strategic player, and it can even collude with a subset of the users facilitated by binding… ▽ More Recent works of Roughgarden (EC'21) and Chung and Shi (SODA'23) initiate the study of a new decentralized mechanism design problem called transaction fee mechanism design (TFM). Unlike the classical mechanism design literature, in the decentralized environment, even the auctioneer (i.e., the miner) can be a strategic player, and it can even collude with a subset of the users facilitated by binding side contracts. Chung and Shi showed two main impossibility results that rule out the existence of a {\it dream} TFM. First, any TFM that provides incentive compatibility for individual users and miner-user coalitions must always have zero miner revenue, no matter whether the block size is finite or infinite. Second, assuming finite block size, no non-trivial TFM can simultaenously provide incentive compatibility for any individual user, and for any miner-user coalition. In this work, we explore what new models and meaningful relaxations can allow us to circumvent the impossibility results of Chung and Shi. Besides today's model that does not employ cryptography, we introduce a new MPC-assisted model where the TFM is implemented by a joint multi-party computation (MPC) protocol among the miners. We prove several feasibility and infeasibility results for achieving {\it strict} and {\it approximate} incentive compatibility, respectively, in the plain model as well as the MPC-assisted model. We show that while cryptography is not a panacea, it indeed allows us to overcome some impossibility results pertaining to the plain model, leading to non-trivial mechanisms with useful guarantees that are otherwise impossible in the plain model. Our work is also the first to characterize the mathematical landscape of transaction fee mechanism design under approximate incentive compatibility, as well as in a cryptography-assisted model. △ Less

Submitted 19 February, 2023; v1 submitted 28 September, 2022; originally announced September 2022.

Uplink Performance of RIS-aided Cell-Free Massive MIMO System Over Spatially Correlated Channels

Authors: Enyu Shi, Jiayi Zhang, Zhe Wang, Derrick Wing Kwan Ng, Bo Ai

Abstract: We consider a practical spatially correlated reconfigurable intelligent surface (RIS)-aided cell-free (CF) massive multiple-input-multiple-output (mMIMO) system with multi-antenna access points (APs) over spatially correlated Rician fading channels. The minimum mean square error (MMSE) channel estimator is adopted to estimate the aggregated RIS channels. Then, we investigate the uplink spectral ef… ▽ More We consider a practical spatially correlated reconfigurable intelligent surface (RIS)-aided cell-free (CF) massive multiple-input-multiple-output (mMIMO) system with multi-antenna access points (APs) over spatially correlated Rician fading channels. The minimum mean square error (MMSE) channel estimator is adopted to estimate the aggregated RIS channels. Then, we investigate the uplink spectral efficiency (SE) with the maximum ratio (MR) and the local minimum mean squared error (L-MMSE) combining at the APs and obtain the closed-form expression for characterizing the performance of the former. The accuracy of our derived analytical results has been verified by extensive Monte-Carlo simulations. Our results show that increasing the number of RIS elements is always beneficial, but with diminishing returns when the number of RIS elements is sufficiently large. Furthermore, the effect of the number of AP antennas on system performance is more pronounced under a small number of RIS elements, while the spatial correlation of RIS elements imposes a more severe negative impact on the system performance than that of the AP antennas. △ Less

Submitted 28 September, 2022; originally announced September 2022.

Comments: 6 pages, 5 figures

Journal ref: early access,Globecom 2022

The structures of higher rank lattice actions on dendrites

Authors: Enhui Shi, Hui Xu

Abstract: Let $Γ$ be a higher rank lattice acting on a nondegenerate dendrite $X$ with no infinite order points. We show that there exists a nondegenerate subdendrite $Y$ which is $Γ$-invariant and satisfies the following items: (1) There is an inverse system of finite actions $\{(Y_i, Γ):i=1,2,3,\cdots\}$ with monotone bonding maps $φ_i: Y_{i+1}\rightarrow Y_i$ and with each $Y_i$ being a dendrite, such… ▽ More Let $Γ$ be a higher rank lattice acting on a nondegenerate dendrite $X$ with no infinite order points. We show that there exists a nondegenerate subdendrite $Y$ which is $Γ$-invariant and satisfies the following items: (1) There is an inverse system of finite actions $\{(Y_i, Γ):i=1,2,3,\cdots\}$ with monotone bonding maps $φ_i: Y_{i+1}\rightarrow Y_i$ and with each $Y_i$ being a dendrite, such that $(Y, Γ|Y)$ is topologically conjugate to the inverse limit $(\underset{\longleftarrow}{\lim}(Y_i, Γ), Γ)$. (2) The first point map $r:X\rightarrow Y$ is a factor map from $(X, Γ)$ to $(Y, Γ|Y)$; if $x\in X\setminus Y$, then $r(x)$ is an end point of $Y$ with infinite orbit; for each $y\in Y$, $r^{-1}(y)$ is contractible, that is there is a sequence $g_i\in Γ$ with ${\rm diam}(g_ir^{-1}(y))\rightarrow 0$. △ Less

Submitted 3 July, 2022; originally announced July 2022.

Rigidity for higher rank lattice actions on dendrites

Authors: Enhui Shi, Hui Xu

Abstract: We study the rigidity in the sense of Zimmer for higher rank lattice actions on dendrites and show that: (1) if $Γ$ is a higher rank lattice and $X$ is a nondegenerate dendrite with no infinite order points, then any action of $Γ$ on $X$ cannot be almost free; (2) if $Γ$ is further a finite index subgroup of $SL_n(\mathbb Z)$ with $n\geq 3$, then every action of $Γ$ on $X$ has a nontrivial almost… ▽ More We study the rigidity in the sense of Zimmer for higher rank lattice actions on dendrites and show that: (1) if $Γ$ is a higher rank lattice and $X$ is a nondegenerate dendrite with no infinite order points, then any action of $Γ$ on $X$ cannot be almost free; (2) if $Γ$ is further a finite index subgroup of $SL_n(\mathbb Z)$ with $n\geq 3$, then every action of $Γ$ on $X$ has a nontrivial almost finite subsystem. During the proof, we get a new characterization of the left-orderability of a finitely generated group through its actions on dendrites. △ Less

Submitted 13 June, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

Comments: The second version corrected some typos and improved the expression of Theorem 1.2

CoCoSoDa: Effective Contrastive Learning for Code Search

Authors: Ensheng Shi, Yanlin Wang, Wenchao Gu, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Hongbin Sun

Abstract: Code search aims to retrieve semantically relevant code snippets for a given natural language query. Recently, many approaches employing contrastive learning have shown promising results on code representation learning and greatly improved the performance of code search. However, there is still a lot of room for improvement in using contrastive learning for code search. In this paper, we propose C… ▽ More Code search aims to retrieve semantically relevant code snippets for a given natural language query. Recently, many approaches employing contrastive learning have shown promising results on code representation learning and greatly improved the performance of code search. However, there is still a lot of room for improvement in using contrastive learning for code search. In this paper, we propose CoCoSoDa to effectively utilize contrastive learning for code search via two key factors in contrastive learning: data augmentation and negative samples. Specifically, soft data augmentation is to dynamically masking or replacing some tokens with their types for input sequences to generate positive samples. Momentum mechanism is used to generate large and consistent representations of negative samples in a mini-batch through maintaining a queue and a momentum encoder. In addition, multimodal contrastive learning is used to pull together representations of code-query pairs and push apart the unpaired code snippets and queries. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages. Experimental results show that: (1) CoCoSoDa outperforms 14 baselines and especially exceeds CodeBERT, GraphCodeBERT, and UniXcoder by 13.3%, 10.5%, and 5.9% on average MRR scores, respectively. (2) The ablation studies show the effectiveness of each component of our approach. (3) We adapt our techniques to several different pre-trained models such as RoBERTa, CodeBERT, and GraphCodeBERT and observe a significant boost in their performance in code search. (4) Our model performs robustly under different hyper-parameters. Furthermore, we perform qualitative and quantitative analyses to explore reasons behind the good performance of our model. △ Less

Submitted 12 February, 2023; v1 submitted 7 April, 2022; originally announced April 2022.

Comments: Accepted by ICSE 2023 (The 45th International Conference on Software Engineering)

An adaptive model checking test for functional linear model

Authors: Enze Shi, Yi Liu, Ke Sun, Lingzhu Li, Linglong Kong

Abstract: Numerous studies have been devoted to the estimation and inference problems for functional linear models (FLM). However, few works focus on model checking problem that ensures the reliability of results. Limited tests in this area do not have tractable null distributions or asymptotic analysis under alternatives. Also, the functional predictor is usually assumed to be fully observed, which is impr… ▽ More Numerous studies have been devoted to the estimation and inference problems for functional linear models (FLM). However, few works focus on model checking problem that ensures the reliability of results. Limited tests in this area do not have tractable null distributions or asymptotic analysis under alternatives. Also, the functional predictor is usually assumed to be fully observed, which is impractical. To address these problems, we propose an adaptive model checking test for FLM. It combines regular moment-based and conditional moment-based tests, and achieves model adaptivity via the dimension of a residual-based subspace. The advantages of our test are manifold. First, it has a tractable chi-squared null distribution and higher powers under the alternatives than its components. Second, asymptotic properties under different underlying models are developed, including the unvisited local alternatives. Third, the test statistic is constructed upon finite grid points, which incorporates the discrete nature of collected data. We develop the desirable relationship between sample size and number of grid points to maintain the asymptotic properties. Besides, we provide a data-driven approach to estimate the dimension leading to model adaptivity, which is promising in sufficient dimension reduction. We conduct comprehensive numerical experiments to demonstrate the advantages the test inherits from its two simple components. △ Less

Submitted 5 June, 2022; v1 submitted 4 April, 2022; originally announced April 2022.

The nonexistence of expansive polycyclic group actions on the circle $\mathbb S^1$

Authors: Enhui Shi, Suhua Wang, Zhiwen Xie, Hui Xu

Abstract: We show that the circle $\mathbb S^1$ admits no expansive polycyclic group actions. We show that the circle $\mathbb S^1$ admits no expansive polycyclic group actions. △ Less

Submitted 23 March, 2022; originally announced March 2022.

RACE: Retrieval-Augmented Commit Message Generation

Authors: Ensheng Shi, Yanlin Wang, Wei Tao, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Hongbin Sun

Abstract: Commit messages are important for software development and maintenance. Many neural network-based approaches have been proposed and shown promising results on automatic commit message generation. However, the generated commit messages could be repetitive or redundant. In this paper, we propose RACE, a new retrieval-augmented neural commit message generation method, which treats the retrieved simil… ▽ More Commit messages are important for software development and maintenance. Many neural network-based approaches have been proposed and shown promising results on automatic commit message generation. However, the generated commit messages could be repetitive or redundant. In this paper, we propose RACE, a new retrieval-augmented neural commit message generation method, which treats the retrieved similar commit as an exemplar and leverages it to generate an accurate commit message. As the retrieved commit message may not always accurately describe the content/intent of the current code diff, we also propose an exemplar guider, which learns the semantic similarity between the retrieved and current code diff and then guides the generation of commit message based on the similarity. We conduct extensive experiments on a large public dataset with five programming languages. Experimental results show that RACE can outperform all baselines. Furthermore, RACE can boost the performance of existing Seq2Seq models in commit message generation. △ Less

Submitted 22 October, 2022; v1 submitted 5 March, 2022; originally announced March 2022.

Comments: Accepted by EMNLP 2022 (The 2022 Conference on Empirical Methods in Natural Language Processing)