Showing 1–29 of 29 results for author: Tong, K

Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task

Authors: Huiping Zhuang, Di Fang, Kai Tong, Yuchen Liu, Ziqian Zeng, Xu Zhou, Cen Chen

Abstract: In the field of autonomous driving, even a meticulously trained model can encounter failures when faced with unfamiliar sceanrios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To addres… ▽ More In the field of autonomous driving, even a meticulously trained model can encounter failures when faced with unfamiliar sceanrios. One of these scenarios can be formulated as an online continual learning (OCL) problem. That is, data come in an online fashion, and models are updated according to these streaming data. Two major OCL challenges are catastrophic forgetting and data imbalance. To address these challenges, in this paper, we propose an Analytic Exemplar-Free Online Continual Learning (AEF-OCL). The AEF-OCL leverages analytic continual learning principles and employs ridge regression as a classifier for features extracted by a large backbone network. It solves the OCL problem by recursively calculating the analytical solution, ensuring an equalization between the continual learning and its joint-learning counterpart, and works without the need to save any used samples (i.e., exemplar-free). Additionally, we introduce a Pseudo-Features Generator (PFG) module that recursively estimates the deviation of real features. The PFG generates offset pseudo-features following a normal distribution, thereby addressing the data imbalance issue. Experimental results demonstrate that despite being an exemplar-free strategy, our method outperforms various methods on the autonomous driving SODA10M dataset. Source code is available at https://github.com/ZHUANGHP/Analytic-continual-learning. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Analytic Federated Learning

Authors: Huiping Zhuang, Run He, Kai Tong, Di Fang, Han Sun, Haoran Li, Tianyi Chen, Ziqian Zeng

Abstract: In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) community. Our AFL draws inspiration from analytic learning -- a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch trai… ▽ More In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) community. Our AFL draws inspiration from analytic learning -- a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch training, eliminating the necessity for multi-epoch updates. In the aggregation stage, we derive an absolute aggregation (AA) law. This AA law allows a single-round aggregation, removing the need for multiple aggregation rounds. More importantly, the AFL exhibits a \textit{weight-invariant} property, meaning that regardless of how the full dataset is distributed among clients, the aggregated result remains identical. This could spawn various potentials, such as data heterogeneity invariance, client-number invariance, absolute convergence, and being hyperparameter-free (our AFL is the first hyperparameter-free method in FL history). We conduct experiments across various FL settings including extremely non-IID ones, and scenarios with a large number of clients (e.g., $\ge 1000$). In all these settings, our AFL constantly performs competitively while existing FL techniques encounter various obstacles. Code is available at \url{https://github.com/ZHUANGHP/Analytic-federated-learning} △ Less

Submitted 25 May, 2024; originally announced May 2024.

DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

Authors: Huiping Zhuang, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Zhiping Lin

Abstract: Class-incremental learning (CIL) under an exemplar-free constraint has presented a significant challenge. Existing methods adhering to this constraint are prone to catastrophic forgetting, far more so than replay-based techniques that retain access to past samples. In this paper, to solve the exemplar-free CIL problem, we propose a Dual-Stream Analytic Learning (DS-AL) approach. The DS-AL contains… ▽ More Class-incremental learning (CIL) under an exemplar-free constraint has presented a significant challenge. Existing methods adhering to this constraint are prone to catastrophic forgetting, far more so than replay-based techniques that retain access to past samples. In this paper, to solve the exemplar-free CIL problem, we propose a Dual-Stream Analytic Learning (DS-AL) approach. The DS-AL contains a main stream offering an analytical (i.e., closed-form) linear solution, and a compensation stream improving the inherent under-fitting limitation due to adopting linear mapping. The main stream redefines the CIL problem into a Concatenated Recursive Least Squares (C-RLS) task, allowing an equivalence between the CIL and its joint-learning counterpart. The compensation stream is governed by a Dual-Activation Compensation (DAC) module. This module re-activates the embedding with a different activation function from the main stream one, and seeks fitting compensation by projecting the embedding to the null space of the main stream's linear mapping. Empirical results demonstrate that the DS-AL, despite being an exemplar-free technique, delivers performance comparable with or better than that of replay-based methods across various datasets, including CIFAR-100, ImageNet-100 and ImageNet-Full. Additionally, the C-RLS' equivalent property allows the DS-AL to execute CIL in a phase-invariant manner. This is evidenced by a never-before-seen 500-phase CIL ImageNet task, which performs on a level identical to a 5-phase one. Our codes are available at https://github.com/ZHUANGHP/Analytic-continual-learning. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: Accepted in AAAI 2024

Cache-Enabled Millimetre-Wave Fluid Antenna Systems: Modeling and Performance

Authors: Farshad Rostami Ghadi, Kai-Kit Wong, Kin-Fai Tong, Yangyang Zhang

Abstract: This letter investigates the performance of content caching in a heterogeneous cellular network (HetNet) consisting of fluid antenna system (FAS)-equipped mobile users (MUs) and millimeter-wave (mm-wave) single-antenna small base stations (SBSs), distributed according to the independent homogeneous Poisson point processes (HPPP). In particular, it is assumed that the most popular contents are cach… ▽ More This letter investigates the performance of content caching in a heterogeneous cellular network (HetNet) consisting of fluid antenna system (FAS)-equipped mobile users (MUs) and millimeter-wave (mm-wave) single-antenna small base stations (SBSs), distributed according to the independent homogeneous Poisson point processes (HPPP). In particular, it is assumed that the most popular contents are cached in the SBSs to serve the FAS-equipped MUs requests. To assess the system performance, we derive compact expressions for the successful content delivery probability (SCDP) and the content delivery delay (CDD) using the Gauss-Laguerre quadrature technique. Our numerical results show that the performance of cache-enabled mm-wave HetNets can be greatly improved, when the FAS is utilized at the MUs instead of traditional fixed-antenna system deployment. △ Less

Submitted 25 March, 2024; originally announced March 2024.

AOCIL: Exemplar-free Analytic Online Class Incremental Learning with Low Time and Resource Consumption

Authors: Huiping Zhuang, Yuchen Liu, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Yi Wang, Lap-Pui Chau

Abstract: Online Class Incremental Learning (OCIL) aims to train the model in a task-by-task manner, where data arrive in mini-batches at a time while previous data are not accessible. A significant challenge is known as Catastrophic Forgetting, i.e., loss of the previous knowledge on old data. To address this, replay-based methods show competitive results but invade data privacy, while exemplar-free method… ▽ More Online Class Incremental Learning (OCIL) aims to train the model in a task-by-task manner, where data arrive in mini-batches at a time while previous data are not accessible. A significant challenge is known as Catastrophic Forgetting, i.e., loss of the previous knowledge on old data. To address this, replay-based methods show competitive results but invade data privacy, while exemplar-free methods protect data privacy but struggle for accuracy. In this paper, we proposed an exemplar-free approach -- Analytic Online Class Incremental Learning (AOCIL). Instead of back-propagation, we design the Analytic Classifier (AC) updated by recursive least square, cooperating with a frozen backbone. AOCIL simultaneously achieves high accuracy, low resource consumption and data privacy protection. We conduct massive experiments on four existing benchmark datasets, and the results demonstrate the strong capability of handling OCIL scenarios. Codes will be ready. △ Less

Submitted 23 March, 2024; originally announced March 2024.

G-ACIL: Analytic Learning for Exemplar-Free Generalized Class Incremental Learning

Authors: Huiping Zhuang, Yizhu Chen, Di Fang, Run He, Kai Tong, Hongxin Wei, Ziqian Zeng, Cen Chen

Abstract: Class incremental learning (CIL) trains a network on sequential tasks with separated categories but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution, leadin… ▽ More Class incremental learning (CIL) trains a network on sequential tasks with separated categories but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution, leading to intensified forgetting. Existing attempts for the GCIL either have poor performance, or invade data privacy by saving historical exemplars. To address this, in this paper, we propose an exemplar-free generalized analytic class incremental learning (G-ACIL). The G-ACIL adopts analytic learning (a gradient-free training technique), and delivers an analytical solution (i.e., closed-form) to the GCIL scenario. This solution is derived via decomposing the incoming data into exposed and unexposed classes, allowing an equivalence between the incremental learning and its joint training, i.e., the weight-invariant property. Such an equivalence is theoretically validated through matrix analysis tools, and hence contributes interpretability in GCIL. It is also empirically evidenced by experiments on various datasets and settings of GCIL. The results show that the G-ACIL exhibits leading performance with high robustness compared with existing competitive GCIL methods. Codes will be ready at \url{https://github.com/ZHUANGHP/Analytic-continual-learning}. △ Less

Submitted 13 April, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

REAL: Representation Enhanced Analytic Learning for Exemplar-free Class-incremental Learning

Authors: Run He, Huiping Zhuang, Di Fang, Yizhu Chen, Kai Tong, Cen Chen

Abstract: Exemplar-free class-incremental learning (EFCIL) aims to mitigate catastrophic forgetting in class-incremental learning without available historical data. Compared with its counterpart (replay-based CIL) that stores historical samples, the EFCIL suffers more from forgetting issues under the exemplar-free constraint. In this paper, inspired by the recently developed analytic learning (AL) based CIL… ▽ More Exemplar-free class-incremental learning (EFCIL) aims to mitigate catastrophic forgetting in class-incremental learning without available historical data. Compared with its counterpart (replay-based CIL) that stores historical samples, the EFCIL suffers more from forgetting issues under the exemplar-free constraint. In this paper, inspired by the recently developed analytic learning (AL) based CIL, we propose a representation enhanced analytic learning (REAL) for EFCIL. The REAL constructs a dual-stream base pretraining (DS-BPT) and a representation enhancing distillation (RED) process to enhance the representation of the extractor. The DS-BPT pretrains model in streams of both supervised learning and self-supervised contrastive learning (SSCL) for base knowledge extraction. The RED process distills the supervised knowledge to the SSCL pretrained backbone and facilitates a subsequent AL-basd CIL that converts the CIL to a recursive least-square problem. Our method addresses the issue of insufficient discriminability in representations of unseen data caused by a frozen backbone in the existing AL-based CIL. Empirical results on various datasets including CIFAR-100, ImageNet-100 and ImageNet-1k, demonstrate that our REAL outperforms the state-of-the-arts in EFCIL, and achieves comparable or even more superior performance compared with the replay-based methods. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained Locality Learning Matters in Consistency Regularization

Authors: Wentao Pan, Zhe Xu, Jiangpeng Yan, Zihan Wu, Raymond Kai-yu Tong, Xiu Li, Jianhua Yao

Abstract: Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning, wherein the weak-to-strong consistency regularization framework, popularized by FixMatch, is widely used as a benchmark scheme. Despite its effectiveness, we observe that such scheme struggles with satisfactory segmentation for the local regions. This can… ▽ More Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning, wherein the weak-to-strong consistency regularization framework, popularized by FixMatch, is widely used as a benchmark scheme. Despite its effectiveness, we observe that such scheme struggles with satisfactory segmentation for the local regions. This can be because it originally stems from the image classification task and lacks specialized mechanisms to capture fine-grained local semantics that prioritizes in dense prediction. To address this issue, we propose a novel framework called \texttt{MaskMatch}, which enables fine-grained locality learning to achieve better dense segmentation. On top of the original teacher-student framework, we design a masked modeling proxy task that encourages the student model to predict the segmentation given the unmasked image patches (even with 30\% only) and enforces the predictions to be consistent with pseudo-labels generated by the teacher model using the complete image. Such design is motivated by the intuition that if the predictions are more consistent given insufficient neighboring information, stronger fine-grained locality perception is achieved. Besides, recognizing the importance of reliable pseudo-labels in the above locality learning and the original consistency learning scheme, we design a multi-scale ensembling strategy that considers context at different levels of abstraction for pseudo-label generation. Extensive experiments on benchmark datasets demonstrate the superiority of our method against previous approaches and its plug-and-play flexibility. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Fluid Antenna-Assisted Dirty Multiple Access Channels over Composite Fading

Authors: Farshad Rostami Ghadi, Kai-Kit Wong, F. Javier Lopez-Martinez, Chan-Byoung Chae, Kin-Fai Tong, Yangyang Zhang

Abstract: This letter investigates the application of the emerging fluid antenna (FA) technology in multiuser communication systems when side information (SI) is available at the transmitters. In particular, we consider a K-user dirty multiple access channel (DMAC) with non-causally known SI at the transmitters, where K users send independent messages to a common receiver with a FA capable of changing its l… ▽ More This letter investigates the application of the emerging fluid antenna (FA) technology in multiuser communication systems when side information (SI) is available at the transmitters. In particular, we consider a K-user dirty multiple access channel (DMAC) with non-causally known SI at the transmitters, where K users send independent messages to a common receiver with a FA capable of changing its location depending on the channel condition. By connecting Jakes' model to copula theory through Spearman's ρ rank correlation coefficient, we accurately describe the spatial correlation between the FA channels, and derive a closed-form expression for the outage probability (OP) under Fisher-Snedecor F fading. Numerical results illustrate how considering FA can improve the performance of multiuser communication systems in terms of the OP and also support a large number of users using only one FA at the common receiver in a few wavelengths of space. △ Less

Submitted 14 September, 2023; originally announced September 2023.

A Gaussian Copula Approach to the Performance Analysis of Fluid Antenna Systems

Authors: Farshad Rostami Ghadi, Kai-Kit Wong, F. Javier Lopez-Martinez, Chan-Byoung Chae, Kin-Fai Tong, Yangyang Zhang

Abstract: This paper investigates the performance of a singleuser fluid antenna system (FAS), by exploiting a class of elliptical copulas to describe the structure of dependency amongst the fluid antenna ports. By expressing Jakes' model in terms of the Gaussian copula, we consider two cases: (i) the general case, i.e., any arbitrary correlated fading distribution; and (ii) the specific case, i.e., correlat… ▽ More This paper investigates the performance of a singleuser fluid antenna system (FAS), by exploiting a class of elliptical copulas to describe the structure of dependency amongst the fluid antenna ports. By expressing Jakes' model in terms of the Gaussian copula, we consider two cases: (i) the general case, i.e., any arbitrary correlated fading distribution; and (ii) the specific case, i.e., correlated Nakagami-m fading. For both scenarios, we first derive analytical expressions for the cumulative distribution function (CDF) and probability density function (PDF) of the equivalent channel in terms of multivariate normal distribution. Then, we obtain the outage probability (OP) and the delay outage rate (DOR) to analyze the performance of the FAS. By employing the popular rank correlation coefficients such as Spearman's \{rho} and Kendall's τ, we measure the degree of dependency in correlated arbitrary fading channels and illustrate how the Gaussian copula can be accurately connected to Jakes' model in FAS without complicated mathematical analysis. Numerical results show that increasing the fluid antenna size provides lower OP and DOR, but the system performance saturates as the number of antenna ports increases. In addition, our results indicate that FAS provides better performance compared to conventional single-fixed antenna systems even when the size of fluid antenna is small. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Rethinking the Backward Propagation for Adversarial Transferability

Authors: Xiaosen Wang, Kangheng Tong, Kun He

Abstract: Transfer-based attacks generate adversarial examples on the surrogate model, which can mislead other black-box models without access, making it promising to attack real-world applications. Recently, several works have been proposed to boost adversarial transferability, in which the surrogate model is usually overlooked. In this work, we identify that non-linear layers (e.g., ReLU, max-pooling, etc… ▽ More Transfer-based attacks generate adversarial examples on the surrogate model, which can mislead other black-box models without access, making it promising to attack real-world applications. Recently, several works have been proposed to boost adversarial transferability, in which the surrogate model is usually overlooked. In this work, we identify that non-linear layers (e.g., ReLU, max-pooling, etc.) truncate the gradient during backward propagation, making the gradient w.r.t. input image imprecise to the loss function. We hypothesize and empirically validate that such truncation undermines the transferability of adversarial examples. Based on these findings, we propose a novel method called Backward Propagation Attack (BPA) to increase the relevance between the gradient w.r.t. input image and loss function so as to generate adversarial examples with higher transferability. Specifically, BPA adopts a non-monotonic function as the derivative of ReLU and incorporates softmax with temperature to smooth the derivative of max-pooling, thereby mitigating the information loss during the backward propagation of gradients. Empirical results on the ImageNet dataset demonstrate that not only does our method substantially boost the adversarial transferability, but it is also general to existing transfer-based attacks. Code is available at https://github.com/Trustworthy-AI-Group/RPA. △ Less

Submitted 20 November, 2023; v1 submitted 22 June, 2023; originally announced June 2023.

Comments: Accepted by NeurIPS 2023

Copula-based Performance Analysis for Fluid Antenna Systems under Arbitrary Fading Channels

Authors: Farshad Rostami Ghadi, Kai-Kit Wong, F. Javier Lopez-Martinez, Kin-Fai Tong

Abstract: In this letter, we study the performance of a single-user fluid antenna system (FAS) under arbitrary fading distributions, in which the fading channel coefficients over the ports are correlated. We adopt copula theory to model the structure of dependency between fading coefficients. Specifically, we first derive an exact closed-from expression for the outage probability in the most general case, i… ▽ More In this letter, we study the performance of a single-user fluid antenna system (FAS) under arbitrary fading distributions, in which the fading channel coefficients over the ports are correlated. We adopt copula theory to model the structure of dependency between fading coefficients. Specifically, we first derive an exact closed-from expression for the outage probability in the most general case, i.e., for any arbitrary choice of fading distribution and copula. Afterwards, for an important specific case, we analyze the performance of the outage probability under correlated Nakagami-$m$ fading channels by exploiting popular Archimedean copulas, namely, Frank, Clayton, and Gumbel. The results demonstrate that FAS outperforms the conventional single fixed-antenna system in terms of the outage probability. We also see that the spatial correlation dependency structure for the FAS is a key factor to determine its performance, which is natively captured through the choice of copula. △ Less

Submitted 16 May, 2023; originally announced May 2023.

Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention

Authors: Kedeng Tong, Xin Jin, Yuqing Yang, Chen Wang, Jinshi Kang, Fan Jiang

Abstract: Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously with higher spatial resolution relative to traditional plenoptic cameras, which facilitate various applications in computer vision. However, the existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the micr… ▽ More Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously with higher spatial resolution relative to traditional plenoptic cameras, which facilitate various applications in computer vision. However, the existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the microlens relay imaging and long-distance correlations among the microimages. In this paper, a lossy end-to-end learning architecture is proposed to compress the focused plenoptic images efficiently. First, a data preprocessing scheme is designed according to the imaging principle to remove the sub-aperture image ineffective pixels in the recorded light field and align the microimages to the rectangular grid. Then, the global attention module with large receptive field is proposed to capture the global correlation among the feature maps using pixel-wise vector attention computed in the resampling process. Also, a new image dataset consisting of 1910 focused plenoptic images with content and depth diversity is built to benefit training and testing. Extensive experimental evaluations demonstrate the effectiveness of the proposed approach. It outperforms intra coding of HEVC and VVC by an average of 62.57% and 51.67% bitrate reduction on the 20 preprocessed focused plenoptic images, respectively. Also, it achieves 18.73% bitrate saving and generates perceptually pleasant reconstructions compared to the state-of-the-art end-to-end image compression methods, which benefits the applications of focused plenoptic cameras greatly. The dataset and code are publicly available at https://github.com/VincentChandelier/GACN. △ Less

Submitted 30 April, 2023; originally announced May 2023.

Comments: 14 pages, 15 figures, accepted by IEEE Transactions on Multimedia

QVRF: A Quantization-error-aware Variable Rate Framework for Learned Image Compression

Authors: Kedeng Tong, Yaojun Wu, Yue Li, Kai Zhang, Li Zhang, Xin Jin

Abstract: Learned image compression has exhibited promising compression performance, but variable bitrates over a wide range remain a challenge. State-of-the-art variable rate methods compromise the loss of model performance and require numerous additional parameters. In this paper, we present a Quantization-error-aware Variable Rate Framework (QVRF) that utilizes a univariate quantization regulator a to ac… ▽ More Learned image compression has exhibited promising compression performance, but variable bitrates over a wide range remain a challenge. State-of-the-art variable rate methods compromise the loss of model performance and require numerous additional parameters. In this paper, we present a Quantization-error-aware Variable Rate Framework (QVRF) that utilizes a univariate quantization regulator a to achieve wide-range variable rates within a single model. Specifically, QVRF defines a quantization regulator vector coupled with predefined Lagrange multipliers to control quantization error of all latent representation for discrete variable rates. Additionally, the reparameterization method makes QVRF compatible with a round quantizer. Exhaustive experiments demonstrate that existing fixed-rate VAE-based methods equipped with QVRF can achieve wide-range continuous variable rates within a single model without significant performance degradation. Furthermore, QVRF outperforms contemporary variable-rate methods in rate-distortion performance with minimal additional parameters. △ Less

Submitted 10 March, 2023; originally announced March 2023.

Comments: 7 pages, 6 figures

An Information-Theoretic Characterization of MIMO-FAS: Optimization, Diversity-Multiplexing Tradeoff and $q$-Outage Capacity

Authors: Wee Kiat New, Kai-Kit Wong, Hao Xu, Kin-Fai Tong, Chan-Byoung Chae

Abstract: Multiple-input multiple-output (MIMO) system has been the defining mobile communications technology in recent generations. With the ever-increasing demands looming towards the sixth generation (6G), we are in need of additional degrees of freedom that deliver further gains beyond MIMO. To this goal, fluid antenna system (FAS) has emerged as a new way to obtain spatial diversity using reconfigurabl… ▽ More Multiple-input multiple-output (MIMO) system has been the defining mobile communications technology in recent generations. With the ever-increasing demands looming towards the sixth generation (6G), we are in need of additional degrees of freedom that deliver further gains beyond MIMO. To this goal, fluid antenna system (FAS) has emerged as a new way to obtain spatial diversity using reconfigurable position-switchable antennas. Considering the case with more than one ports activated on a 2D fluid antenna surface at both ends, we take the information-theoretic approach to study the achievable performance limits of the MIMO-FAS. First of all, we propose a suboptimal scheme, referred to as QR MIMO-FAS, to maximize the rate at high signal-to-noise ratio (SNR) via joint port selection, transmit and receive beamforming and power allocation. We then derive the optimal diversity and multiplexing tradeoff (DMT) of MIMO-FAS. From the DMT, we highlight that MIMO-FAS outperforms traditional MIMO antenna systems. Further, we introduce a new metric, namely q-outage capacity, which can jointly consider rate and outage probability. Through this metric, our results indicate that MIMO-FAS surpasses traditional MIMO greatly. △ Less

Submitted 25 October, 2023; v1 submitted 3 March, 2023; originally announced March 2023.

Comments: 15 pages, 12 figures, 2 tables, 1 algorithm. Accepted by IEEE Transactions on Wireless Communications

Fluid Antenna System: New Insights on Outage Probability and Diversity Gain

Authors: Wee Kiat New, Kai-Kit Wong, Hao Xu, Kin-Fai Tong, Chan-Byoung Chae

Abstract: To enable innovative applications and services, both industry and academia are exploring new technologies for sixth generation (6G) communications. One of the promising candidates is fluid antenna system (FAS). Unlike existing systems, FAS is a novel communication technology where its antenna can freely change its position and shape within a given space. Compared to the traditional systems, this u… ▽ More To enable innovative applications and services, both industry and academia are exploring new technologies for sixth generation (6G) communications. One of the promising candidates is fluid antenna system (FAS). Unlike existing systems, FAS is a novel communication technology where its antenna can freely change its position and shape within a given space. Compared to the traditional systems, this unique capability has the potential of providing higher diversity and interference-free communications. Nevertheless, the performance limits of FAS remain unclear as its system properties are difficult to analyze. To address this, we approximate the outage probability and diversity gain of FAS in closed-form expressions. We then propose a suboptimal FAS with $N^{*}$ ports, where a significant gain can be obtained over FAS with $N^{*}-1$ ports whilst FAS with $N^{*}+1$ ports only yields marginal improvement over the proposed suboptimal FAS. In this paper, we also provide analytical and simulation results to unfold the key factors that affect the performance of FAS. Limited to systems with one active radio frequency (RF)-chain, we show that the proposed suboptimal FAS outperforms single-antenna (SISO) system and selection combining (SC) system in terms of outage probability. Interestingly, when the given space is $\fracλ{2}$, the outage probability of the proposed suboptimal FAS with one active RF-chain achieves near to that of the maximal ratio combining (MRC) system with multiple active RF-chains. △ Less

Submitted 11 May, 2023; v1 submitted 30 December, 2022; originally announced January 2023.

Comments: 25 pages, 12 figures. Accepted by IEEE Transactions on Wireless Communications

ICME 2022 Few-shot LOGO detection top 9 solution

Authors: Ka Ho Tong, Ka Wai Cheung, Xiaochuan Yu

Abstract: ICME-2022 few-shot logo detection competition is held in May, 2022. Participants are required to develop a single model to detect logos by handling tiny logo instances, similar brands, and adversarial images at the same time, with limited annotations. Our team achieved rank 16 and 11 in the first and second round of the competition respectively, with a final rank of 9th. This technical report summ… ▽ More ICME-2022 few-shot logo detection competition is held in May, 2022. Participants are required to develop a single model to detect logos by handling tiny logo instances, similar brands, and adversarial images at the same time, with limited annotations. Our team achieved rank 16 and 11 in the first and second round of the competition respectively, with a final rank of 9th. This technical report summarized our major techniques used in this competitions, and potential improvement. △ Less

Submitted 22 June, 2022; originally announced June 2022.

SADN: Learned Light Field Image Compression with Spatial-Angular Decorrelation

Authors: Kedeng Tong, Xin Jin, Chen Wang, Fan Jiang

Abstract: Light field image becomes one of the most promising media types for immersive video applications. In this paper, we propose a novel end-to-end spatial-angular-decorrelated network (SADN) for high-efficiency light field image compression. Different from the existing methods that exploit either spatial or angular consistency in the light field image, SADN decouples the angular and spatial informatio… ▽ More Light field image becomes one of the most promising media types for immersive video applications. In this paper, we propose a novel end-to-end spatial-angular-decorrelated network (SADN) for high-efficiency light field image compression. Different from the existing methods that exploit either spatial or angular consistency in the light field image, SADN decouples the angular and spatial information by dilation convolution and stride convolution in spatial-angular interaction, and performs feature fusion to compress spatial and angular information jointly. To train a stable and robust algorithm, a large-scale dataset consisting of 7549 light field images is proposed and built. The proposed method provides 2.137 times and 2.849 times higher compression efficiency relative to H.266/VVC and H.265/HEVC inter coding, respectively. It also outperforms the end-to-end image compression networks by an average of 79.6% bitrate saving with much higher subjective quality and light field consistency. △ Less

Submitted 22 February, 2022; originally announced February 2022.

Triangle Attack: A Query-efficient Decision-based Adversarial Attack

Authors: Xiaosen Wang, Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, Wei Liu

Abstract: Decision-based attack poses a severe threat to real-world applications since it regards the target model as a black box and only accesses the hard prediction label. Great efforts have been made recently to decrease the number of queries; however, existing decision-based attacks still require thousands of queries in order to generate good quality adversarial examples. In this work, we find that a b… ▽ More Decision-based attack poses a severe threat to real-world applications since it regards the target model as a black box and only accesses the hard prediction label. Great efforts have been made recently to decrease the number of queries; however, existing decision-based attacks still require thousands of queries in order to generate good quality adversarial examples. In this work, we find that a benign sample, the current and the next adversarial examples can naturally construct a triangle in a subspace for any iterative attacks. Based on the law of sines, we propose a novel Triangle Attack (TA) to optimize the perturbation by utilizing the geometric information that the longer side is always opposite the larger angle in any triangle. However, directly applying such information on the input image is ineffective because it cannot thoroughly explore the neighborhood of the input sample in the high dimensional space. To address this issue, TA optimizes the perturbation in the low frequency space for effective dimensionality reduction owing to the generality of such geometric property. Extensive evaluations on ImageNet dataset show that TA achieves a much higher attack success rate within 1,000 queries and needs a much less number of queries to achieve the same attack success rate under various perturbation budgets than existing decision-based attacks. With such high efficiency, we further validate the applicability of TA on real-world API, i.e., Tencent Cloud API. △ Less

Submitted 21 July, 2022; v1 submitted 13 December, 2021; originally announced December 2021.

Comments: Accepted by ECCV 2022, code is available at https://github.com/xiaosen-wang/TA

All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation

Authors: Zhe Xu, Yixin Wang, Donghuan Lu, Lequan Yu, Jiangpeng Yan, Jie Luo, Kai Ma, Yefeng Zheng, Raymond Kai-yu Tong

Abstract: Semi-supervised learning has substantially advanced medical image segmentation since it alleviates the heavy burden of acquiring the costly expert-examined annotations. Especially, the consistency-based approaches have attracted more attention for their superior performance, wherein the real labels are only utilized to supervise their paired images via supervised loss while the unlabeled images ar… ▽ More Semi-supervised learning has substantially advanced medical image segmentation since it alleviates the heavy burden of acquiring the costly expert-examined annotations. Especially, the consistency-based approaches have attracted more attention for their superior performance, wherein the real labels are only utilized to supervise their paired images via supervised loss while the unlabeled images are exploited by enforcing the perturbation-based \textit{"unsupervised"} consistency without explicit guidance from those real labels. However, intuitively, the expert-examined real labels contain more reliable supervision signals. Observing this, we ask an unexplored but interesting question: can we exploit the unlabeled data via explicit real label supervision for semi-supervised training? To this end, we discard the previous perturbation-based consistency but absorb the essence of non-parametric prototype learning. Based on the prototypical network, we then propose a novel cyclic prototype consistency learning (CPCL) framework, which is constructed by a labeled-to-unlabeled (L2U) prototypical forward process and an unlabeled-to-labeled (U2L) backward process. Such two processes synergistically enhance the segmentation network by encouraging more discriminative and compact features. In this way, our framework turns previous \textit{"unsupervised"} consistency into new \textit{"supervised"} consistency, obtaining the \textit{"all-around real label supervision"} property of our method. Extensive experiments on brain tumor segmentation from MRI and kidney segmentation from CT images show that our CPCL can effectively exploit the unlabeled data and outperform other state-of-the-art semi-supervised medical image segmentation methods. △ Less

Submitted 15 March, 2022; v1 submitted 28 September, 2021; originally announced September 2021.

Comments: 11 pages

Fluid Antenna Multiple Access

Authors: Kai-Kit Wong, Kin-Fai Tong

Abstract: Fluid antenna is a novel technology that can make an antenna appear instantly at one of N preset locations in a predefined space. An important application is to adopt fluid antenna in a small space of mobile device for obtaining the tremendous diversity hidden in the small space. Previous results have revealed that a single-antenna fluid antenna system, even with a very small space, can outperform… ▽ More Fluid antenna is a novel technology that can make an antenna appear instantly at one of N preset locations in a predefined space. An important application is to adopt fluid antenna in a small space of mobile device for obtaining the tremendous diversity hidden in the small space. Previous results have revealed that a single-antenna fluid antenna system, even with a very small space, can outperform a multiple antenna maximum ratio combining (MRC) system if $N$ is large enough. This paper explores the potential of using fluid antenna for multiple access through performance analysis. Fluid antenna multiple access (FAMA) exploits moments of deep fade experienced by the interference to achieve a favourable channel condition for the desired signal, without requiring sophisticated signal processing. We analyze the FAMA system by first deriving the outage probability of the signal-to-interference ratio (SIR) in a double integral form. We then obtain an outage probability upper bound in closed form and an average outage capacity lower bound for the FAMA system, with an arbitrary number of interferers, from which the multiplexing gain of FAMA is characterized. We also estimate how large N is required to achieve a given multiplexing gain using fluid antennas with a given size. Results illustrate that it is possible for FAMA to support hundreds of users using only one fluid antenna at each user in a few wavelengths of space, giving rise to significant enhancement in the network outage capacity. △ Less

Submitted 9 June, 2020; originally announced June 2020.

Comments: 27 pages, 8 figures

A Vision to Smart Radio Environment: Surface Wave Communication Superhighways

Authors: Kai-Kit Wong, Kin-Fai Tong, Zhiyuan Chu, Yangyang Zhang

Abstract: Complementary to traditional approaches that focus on transceiver design for bringing the best out of unstable, lossy fading channels, one radical development in wireless communications that has recently emerged is to pursue a smart radio environment by using software-defined materials or programmable metasurfaces for establishing favourable propagation conditions. This article portraits a vision… ▽ More Complementary to traditional approaches that focus on transceiver design for bringing the best out of unstable, lossy fading channels, one radical development in wireless communications that has recently emerged is to pursue a smart radio environment by using software-defined materials or programmable metasurfaces for establishing favourable propagation conditions. This article portraits a vision of communication superhighways enabled by surface wave (SW) propagation on "smart surfaces" for future smart radio environments. The concept differs from the mainstream efforts of using passive elements on a large surface for bouncing off radio waves intelligently towards intended user terminals. In this vision, energy efficiency will be ultra-high, due to much less pathloss compared to free space propagation, and the fact that SW is inherently confined to the smart surface not only greatly simplifies the task of interference management, but also makes possible exceptionally localized high-speed interference-free data access. We shall outline the opportunities and associated challenges arisen from the SW paradigm. We shall also attempt to shed light on several key enabling technologies that make this realizable. One important technology which will be discussed is a software-controlled fluidic waveguiding architecture that permits dynamic creation of high-throughput data highways. △ Less

Submitted 28 May, 2020; originally announced May 2020.

Comments: 7 pages, 6 figures

Performance Limits of Fluid Antenna Systems

Authors: Kai-Kit Wong, Arman Shojaeifard, Kin-Fai Tong, Yangyang Zhang

Abstract: Fluid antenna represents a concept where a mechanically flexible antenna can switch its location freely within a given space. Recently, it has been reported that even with a tiny space, a single-antenna fluid antenna system (FAS) can outperform an L-antenna maximum ratio combining (MRC) system in terms of outage probability if the number of locations (or ports) the fluid antenna can be switched to… ▽ More Fluid antenna represents a concept where a mechanically flexible antenna can switch its location freely within a given space. Recently, it has been reported that even with a tiny space, a single-antenna fluid antenna system (FAS) can outperform an L-antenna maximum ratio combining (MRC) system in terms of outage probability if the number of locations (or ports) the fluid antenna can be switched to, is large enough. This letter aims to study if extraordinary capacity can also be achieved by FAS with a small space. We do this by deriving the ergodic capacity, and a capacity lower bound. This letter also derives the level crossing rate (LCR) and average fade duration (AFD) for the FAS. △ Less

Submitted 27 May, 2020; originally announced May 2020.

Comments: 4 pages, 5 figures

Fluid Antenna Systems

Authors: Kai-Kit Wong, Arman Shojaeifard, Kin-Fai Tong, Yangyang Zhang

Abstract: Over the past decades, multiple antenna technologies have appeared in many different forms, most notably as multiple-input multiple-output (MIMO), to transform wireless communications for extraordinary diversity and multiplexing gains. The variety of technologies has been based on placing a number of antennas at fixed locations which dictates the fundamental limit on the achievable performance. By… ▽ More Over the past decades, multiple antenna technologies have appeared in many different forms, most notably as multiple-input multiple-output (MIMO), to transform wireless communications for extraordinary diversity and multiplexing gains. The variety of technologies has been based on placing a number of antennas at fixed locations which dictates the fundamental limit on the achievable performance. By contrast, this paper envisages the scenario where the physical position of an antenna can be switched freely to one of the N positions over a fixed-length line space to pick up the strongest signal in the manner of traditional selection combining. We refer to this system as a fluid antenna system (FAS) for tremendous flexibility in its possible shape and position. The aim of this paper is to study the achievable performance of a single-antenna FAS system with a fixed length and N in arbitrarily correlated Rayleigh fading channels. Our contributions include exact and approximate closed-form expressions for the outage probability of FAS. We also derive an upper bound for the outage probability, from which it is shown that a single-antenna FAS given any arbitrarily small space can outperform an L-antenna maximum ratio combining (MRC) system if N is large enough. Our analysis also reveals the minimum required size of the FAS, and how large N is considered enough for the FAS to surpass MRC. △ Less

Submitted 23 May, 2020; originally announced May 2020.

Comments: 26 pages, 5 figures

Overview of Tools Supporting Planning for Automated Driving

Authors: Kailin Tong, Zlatan Ajanovic, Georg Stettinger

Abstract: Planning is an essential topic in the realm of automated driving. Besides planning algorithms that are widely covered in the literature, planning requires different software tools for its development, validation, and execution. This paper presents a survey of such tools including map representations, communication, traffic rules, open-source planning stacks and middleware, simulation, and visualiz… ▽ More Planning is an essential topic in the realm of automated driving. Besides planning algorithms that are widely covered in the literature, planning requires different software tools for its development, validation, and execution. This paper presents a survey of such tools including map representations, communication, traffic rules, open-source planning stacks and middleware, simulation, and visualization tools as well as benchmarks. We start by defining the planning task and different supporting tools. Next, we provide a comprehensive review of state-of-the-art developments and analysis of relations among them. Finally, we discuss the current gaps and suggest future research directions. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Visualized Insights into the Optimization Landscape of Fully Convolutional Networks

Authors: Jianjie Lu, Kai-yu Tong

Abstract: Many image processing tasks involve image-to-image mapping, which can be addressed well by fully convolutional networks (FCN) without any heavy preprocessing. Although empirically designing and training FCNs can achieve satisfactory results, reasons for the improvement in performance are slightly ambiguous. Our study is to make progress in understanding their generalization abilities through visua… ▽ More Many image processing tasks involve image-to-image mapping, which can be addressed well by fully convolutional networks (FCN) without any heavy preprocessing. Although empirically designing and training FCNs can achieve satisfactory results, reasons for the improvement in performance are slightly ambiguous. Our study is to make progress in understanding their generalization abilities through visualizing the optimization landscapes. The visualization of objective functions is obtained by choosing a solution and projecting its vicinity onto a 3D space. We compare three FCN-based networks (two existing models and a new proposed in this paper for comparison) on multiple datasets. It has been observed in practice that the connections from the pre-pooled feature maps to the post-upsampled can achieve better results. We investigate the cause and provide experiments to shows that the skip-layer connections in FCN can promote flat optimization landscape, which is well known to generalize better. Additionally, we explore the relationship between the models generalization ability and loss surface under different batch sizes. Results show that large-batch training makes the model converge to sharp minimizers with chaotic vicinities while small-batch method leads the model to flat minimizers with smooth and nearly convex regions. Our work may contribute to insights and analysis for designing and training FCNs. △ Less

Submitted 20 January, 2019; originally announced January 2019.

Comments: In AAAI-19 Workshop on Network Interpretability for Deep Learning

Randomized Load-balanced Routing for Fat-tree Networks

Authors: Suzhen Wang, Jingjing Luo, Bruce Kwong-Bun Tong, Wing S. Wong

Abstract: Fat-tree networks have been widely adopted to High Performance Computing (HPC) clusters and to Data Center Networks (DCN). These parallel systems usually have a large number of servers and hosts, which generate large volumes of highly-volatile traffic. Thus, distributed load-balancing routing design becomes critical to achieve high bandwidth utilization, and low-latency packet delivery. Existing d… ▽ More Fat-tree networks have been widely adopted to High Performance Computing (HPC) clusters and to Data Center Networks (DCN). These parallel systems usually have a large number of servers and hosts, which generate large volumes of highly-volatile traffic. Thus, distributed load-balancing routing design becomes critical to achieve high bandwidth utilization, and low-latency packet delivery. Existing distributed designs rely on remote congestion feedbacks to address congestion, which add overheads to collect and react to network-wide congestion information. In contrast, we propose a simple but effective load-balancing scheme, called Dynamic Randomized load-Balancing (DRB), to achieve network-wide low levels of path collisions through local-link adjustment which is free of communications and cooperations between switches. First, we use D-mod-k path selection scheme to allocate default paths to all source-destination (S-D) pairs in a fat-tree network, guaranteeing low levels of path collision over downlinks for any set of active S-D pairs. Then, we propose Threshold-based Two-Choice (TTC) randomized technique to balance uplink traffic through local uplink adjustment at each switch. We theoretically show that the proposed TTC for the uplink-load balancing in a fat-tree network have a similar performance as the two-choice technique in the area of randomized load balancing. Simulation results show that DRB with TTC technique achieves a significant improvement over many randomized routing schemes for fat-tree networks. △ Less

Submitted 30 August, 2017; originally announced August 2017.

Comments: 13 pages, 1 table, 6 figure,

Performance of Joint Channel and Physical Network Coding Based on Alamouti STBC

Authors: Yi Fang, Lin Wang, Kai-Kit Wong, Kin-Fai Tong

Abstract: This work considers the protograph-coded physical network coding (PNC) based on Alamouti space-time block coding (STBC) over Nakagami-fading two-way relay channels, in which both the two sources and relay possess two antennas. We first propose a novel precoding scheme at the two sources so as to implement the iterative decoder efficiently at the relay. We further address a simplified updating rule… ▽ More This work considers the protograph-coded physical network coding (PNC) based on Alamouti space-time block coding (STBC) over Nakagami-fading two-way relay channels, in which both the two sources and relay possess two antennas. We first propose a novel precoding scheme at the two sources so as to implement the iterative decoder efficiently at the relay. We further address a simplified updating rule of the log-likelihood-ratio (LLR) in such a decoder. Based on the simplified LLR-updating rule and Gaussian approximation, we analyze the theoretical bit-error-rate (BER) of the system, which is shown to be consistent with the decoding thresholds and simulated results. Moreover, the theoretical analysis has lower computational complexity than the protograph extrinsic information transfer (PEXIT) algorithm. Consequently, the analysis not only provides a simple way to evaluate the error performance but also facilitates the design of the joint channel-and-PNC (JCNC) in wireless communication scenarios. △ Less

Submitted 22 May, 2013; originally announced May 2013.

Comments: 6 pages, 4 figures, accpeted

Journal ref: 2013 IEEE ICUWB

Performance Analysis of Protograph LDPC Codes for Nakagami-$m$ Fading Relay Channels

Authors: Yi Fang, Kai-Kit Wong, Lin Wang, Kin-Fai Tong

Abstract: In this paper, we investigate the error performance of the protograph (LDPC) codes over Nakagami-$m$ fading relay channels. We first calculate the decoding thresholds of the protograph codes over such channels with different fading depths (i.e., different values of $m$) by exploiting the modified protograph extrinsic information transfer (PEXIT) algorithm. Furthermore, based on the PEXIT analysis… ▽ More In this paper, we investigate the error performance of the protograph (LDPC) codes over Nakagami-$m$ fading relay channels. We first calculate the decoding thresholds of the protograph codes over such channels with different fading depths (i.e., different values of $m$) by exploiting the modified protograph extrinsic information transfer (PEXIT) algorithm. Furthermore, based on the PEXIT analysis and using Gaussian approximation, we derive the bit-error-rate (BER) expressions for the error-free (EF) relaying protocol and decode-and-forward (DF) relaying protocol. We finally compare the threshold with the theoretical BER and the simulated BER results of the protograph codes. It reveals that the performance of DF protocol is approximately the same as that of EF protocol. Moreover, the theoretical BER expressions, which are shown to be reasonably consistent with the decoding thresholds and the simulated BERs, are able to evaluate the system performance and predict the decoding threshold with lower complexity as compared to the modified PEXIT algorithm. As a result, this work can facilitate the design of the protograph codes for the wireless communication systems. △ Less

Submitted 24 April, 2013; originally announced April 2013.

Comments: 15 pages, 3 figures, accepted, IET Commun., Apri. 2013