Showing 1–50 of 154 results for author: Qiu, Q

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Authors: Yanting Yang, Minghao Chen, Qibo Qiu, Jiahao Wu, Wenxiao Wang, Binbin Lin, Ziyu Guan, Xiaofei He

Abstract: For a general-purpose robot to operate in reality, executing a broad range of instructions across various environments is imperative. Central to the reinforcement learning and planning for such robotic agents is a generalizable reward function. Recent advances in vision-language models, such as CLIP, have shown remarkable performance in the domain of deep learning, paving the way for open-domain v… ▽ More For a general-purpose robot to operate in reality, executing a broad range of instructions across various environments is imperative. Central to the reinforcement learning and planning for such robotic agents is a generalizable reward function. Recent advances in vision-language models, such as CLIP, have shown remarkable performance in the domain of deep learning, paving the way for open-domain visual recognition. However, collecting data on robots executing various language instructions across multiple environments remains a challenge. This paper aims to transfer video-language models with robust generalization into a generalizable language-conditioned reward function, only utilizing robot video data from a minimal amount of tasks in a singular environment. Unlike common robotic datasets used for training reward functions, human video-language datasets rarely contain trivial failure videos. To enhance the model's ability to distinguish between successful and failed robot executions, we cluster failure video features to enable the model to identify patterns within. For each cluster, we integrate a newly trained failure prompt into the text encoder to represent the corresponding failure mode. Our language-conditioned reward function shows outstanding generalization to new environments and new instructions for robot planning and reinforcement learning. △ Less

Submitted 20 July, 2024; originally announced July 2024.

Comments: ECCV 2024 camera-ready

Constructing Concept-based Models to Mitigate Spurious Correlations with Minimal Human Effort

Authors: Jeeyung Kim, Ze Wang, Qiang Qiu

Abstract: Enhancing model interpretability can address spurious correlations by revealing how models draw their predictions. Concept Bottleneck Models (CBMs) can provide a principled way of disclosing and guiding model behaviors through human-understandable concepts, albeit at a high cost of human efforts in data annotation. In this paper, we leverage a synergy of multiple foundation models to construct CBM… ▽ More Enhancing model interpretability can address spurious correlations by revealing how models draw their predictions. Concept Bottleneck Models (CBMs) can provide a principled way of disclosing and guiding model behaviors through human-understandable concepts, albeit at a high cost of human efforts in data annotation. In this paper, we leverage a synergy of multiple foundation models to construct CBMs with nearly no human effort. We discover undesirable biases in CBMs built on pre-trained models and propose a novel framework designed to exploit pre-trained models while being immune to these biases, thereby reducing vulnerability to spurious correlations. Specifically, our method offers a seamless pipeline that adopts foundation models for assessing potential spurious correlations in datasets, annotating concepts for images, and refining the annotations for improved robustness. We evaluate the proposed method on multiple datasets, and the results demonstrate its effectiveness in reducing model reliance on spurious correlations while preserving its interpretability. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Reliable Projection Based Unsupervised Learning for Semi-Definite QCQP with Application of Beamforming Optimization

Authors: Xiucheng Wang, Qi Qiu, Nan Cheng

Abstract: In this paper, we investigate a special class of quadratic-constrained quadratic programming (QCQP) with semi-definite constraints. Traditionally, since such a problem is non-convex and N-hard, the neural network (NN) is regarded as a promising method to obtain a high-performing solution. However, due to the inherent prediction error, it is challenging to ensure all solution output by the NN is fe… ▽ More In this paper, we investigate a special class of quadratic-constrained quadratic programming (QCQP) with semi-definite constraints. Traditionally, since such a problem is non-convex and N-hard, the neural network (NN) is regarded as a promising method to obtain a high-performing solution. However, due to the inherent prediction error, it is challenging to ensure all solution output by the NN is feasible. Although some existing methods propose some naive methods, they only focus on reducing the constraint violation probability, where not all solutions are feasibly guaranteed. To deal with the above challenge, in this paper a computing efficient and reliable projection is proposed, where all solution output by the NN are ensured to be feasible. Moreover, unsupervised learning is used, so the NN can be trained effectively and efficiently without labels. Theoretically, the solution of the NN after projection is proven to be feasible, and we also prove the projection method can enhance the convergence performance and speed of the NN. To evaluate our proposed method, the quality of service (QoS)-contained beamforming scenario is studied, where the simulation results show the proposed method can achieve high-performance which is competitive with the lower bound. △ Less

Submitted 9 July, 2024; v1 submitted 4 July, 2024; originally announced July 2024.

Multi-agent Cooperative Games Using Belief Map Assisted Training

Authors: Qinwei Huang, Chen Luo, Alex B. Wu, Simon Khan, Hai Li, Qinru Qiu

Abstract: In a multi-agent system, agents share their local observations to gain global situational awareness for decision making and collaboration using a message passing system. When to send a message, how to encode a message, and how to leverage the received messages directly affect the effectiveness of the collaboration among agents. When training a multi-agent cooperative game using reinforcement learn… ▽ More In a multi-agent system, agents share their local observations to gain global situational awareness for decision making and collaboration using a message passing system. When to send a message, how to encode a message, and how to leverage the received messages directly affect the effectiveness of the collaboration among agents. When training a multi-agent cooperative game using reinforcement learning (RL), the message passing system needs to be optimized together with the agent policies. This consequently increases the model's complexity and poses significant challenges to the convergence and performance of learning. To address this issue, we propose the Belief-map Assisted Multi-agent System (BAMS), which leverages a neuro-symbolic belief map to enhance training. The belief map decodes the agent's hidden state to provide a symbolic representation of the agent's understanding of the environment and other agent's status. The simplicity of symbolic representation allows the gathering and comparison of the ground truth information with the belief, which provides an additional channel of feedback for the learning. Compared to the sporadic and delayed feedback coming from the reward in RL, the feedback from the belief map is more consistent and reliable. Agents using BAMS can learn a more effective message passing network to better understand each other, resulting in better performance in a cooperative predator and prey game with varying levels of map complexity and compare it to previous multi-agent message passing models. The simulation results showed that BAMS reduced training epochs by 66\%, and agents who apply the BAMS model completed the game with 34.62\% fewer steps on average. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Journal ref: ECAI 2023. IOS Press, 2023: 1617-1624

FLOW: Fusing and Shuffling Global and Local Views for Cross-User Human Activity Recognition with IMUs

Authors: Qi Qiu, Tao Zhu, Furong Duan, Kevin I-Kai Wang, Liming Chen, Mingxing Nie, Mingxing Nie

Abstract: Inertial Measurement Unit (IMU) sensors are widely employed for Human Activity Recognition (HAR) due to their portability, energy efficiency, and growing research interest. However, a significant challenge for IMU-HAR models is achieving robust generalization performance across diverse users. This limitation stems from substantial variations in data distribution among individual users. One primary… ▽ More Inertial Measurement Unit (IMU) sensors are widely employed for Human Activity Recognition (HAR) due to their portability, energy efficiency, and growing research interest. However, a significant challenge for IMU-HAR models is achieving robust generalization performance across diverse users. This limitation stems from substantial variations in data distribution among individual users. One primary reason for this distribution disparity lies in the representation of IMU sensor data in the local coordinate system, which is susceptible to subtle user variations during IMU wearing. To address this issue, we propose a novel approach that extracts a global view representation based on the characteristics of IMU data, effectively alleviating the data distribution discrepancies induced by wearing styles. To validate the efficacy of the global view representation, we fed both global and local view data into model for experiments. The results demonstrate that global view data significantly outperforms local view data in cross-user experiments. Furthermore, we propose a Multi-view Supervised Network (MVFNet) based on Shuffling to effectively fuse local view and global view data. It supervises the feature extraction of each view through view division and view shuffling, so as to avoid the model ignoring important features as much as possible. Extensive experiments conducted on OPPORTUNITY and PAMAP2 datasets demonstrate that the proposed algorithm outperforms the current state-of-the-art methods in cross-user HAR. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Non-Markovian Collective Emission of Giant emitters in the Zeno Regime

Authors: Qing-Yang Qiu, Xin-You Lü

Abstract: We explore the collective Zeno dynamics of giant artificial atoms that are coupled, via multiple coupling points, to a common photonic or acoustic reservoir. In this regime, the establishment of atomic cooperativity and the revivification of exponential decay, are highly intertwined, which is utterly beyond the non-Markovian regime with only retarded backaction. We reveal that giant atoms build up… ▽ More We explore the collective Zeno dynamics of giant artificial atoms that are coupled, via multiple coupling points, to a common photonic or acoustic reservoir. In this regime, the establishment of atomic cooperativity and the revivification of exponential decay, are highly intertwined, which is utterly beyond the non-Markovian regime with only retarded backaction. We reveal that giant atoms build up their collective emission smoothly from the decay rate of zero to that predicted by Markovian approximation, and show great disparity between different waveguide QED setups. As a comparison, the step-like growth of instantaneous decay rates in the retardation-only picture has also been shown. All of these theoretical pictures predict the same collective behavior in the long time limit. From a phenomenological standpoint, we observe that the atomic superradiance exhabits significant directional property. In addition, the subradiant photons feature prolonged oscillation in the early stage of collective radiance, where the energy is exchanged remarkably between giant emitters and the field. Our results might be probed in state-of-art waveguide QED experiments, and fundamentally broaden the fields of collective emission in systems with giant atoms. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 14 pages, 6 figures

CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets

Authors: Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, Jingyi Yu

Abstract: In the realm of digital creativity, our potential to craft intricate 3D worlds from imagination is often hampered by the limitations of existing digital tools, which demand extensive expertise and efforts. To narrow this disparity, we introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic… ▽ More In the realm of digital creativity, our potential to craft intricate 3D worlds from imagination is often hampered by the limitations of existing digital tools, which demand extensive expertise and efforts. To narrow this disparity, we introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic text or image inputs as well as 3D-aware controls from diverse primitives (multi-view images, voxels, bounding boxes, point clouds, implicit representations, etc). At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT), to extract rich 3D priors directly from a diverse range of 3D geometries. Specifically, it adopts neural fields to represent continuous and complete surfaces and uses a geometry generative module with pure transformer blocks in latent space. We present a progressive training scheme to train CLAY on an ultra large 3D model dataset obtained through a carefully designed processing pipeline, resulting in a 3D native geometry generator with 1.5 billion parameters. For appearance generation, CLAY sets out to produce physically-based rendering (PBR) textures by employing a multi-view material diffusion model that can generate 2K resolution textures with diffuse, roughness, and metallic modalities. We demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details. Even first time users can easily use CLAY to bring their vivid 3D imaginations to life, unleashing unlimited creativity. △ Less

Submitted 30 May, 2024; originally announced June 2024.

Comments: Project page: https://sites.google.com/view/clay-3dlm Video: https://youtu.be/YcKFp4U2Voo

Van-Hove annihilation and nematic instability on a Kagome lattice

Authors: Yu-Xiao Jiang, Sen Shao, Wei Xia, M. Michael Denner, Julian Ingham, Md Shafayat Hossain, Qingzheng Qiu, Xiquan Zheng, Hongyu Chen, Zi-Jia Cheng, Xian P. Yang, Byunghoon Kim, Jia-Xin Yin, Songbo Zhang, Maksim Litskevich, Qi Zhang, Tyler A. Cochran, Yingying Peng, Guoqing Chang, Yanfeng Guo, Ronny Thomale, Titus Neupert, M. Zahid Hasan

Abstract: Novel states of matter arise in quantum materials due to strong interactions among electrons. A nematic phase breaks the point group symmetry of the crystal lattice and is known to emerge in correlated materials. Here we report the observation of an intra-unit-cell nematic order and signatures of Pomeranchuk instability in the Kagome metal ScV6Sn6. Using scanning tunneling microscopy and spectrosc… ▽ More Novel states of matter arise in quantum materials due to strong interactions among electrons. A nematic phase breaks the point group symmetry of the crystal lattice and is known to emerge in correlated materials. Here we report the observation of an intra-unit-cell nematic order and signatures of Pomeranchuk instability in the Kagome metal ScV6Sn6. Using scanning tunneling microscopy and spectroscopy, we reveal a stripe-like nematic order breaking the crystal rotational symmetry within the Kagome lattice itself. Moreover, we identify a set of van Hove singularities adhering to the Kagome layer electrons, which appear along one direction of the Brillouin zone while being annihilated along other high-symmetry directions, revealing a rotational symmetry breaking. Via detailed spectroscopic maps, we further observe an elliptical deformation of Fermi surface, which provides direct evidence for an electronically mediated nematic order. Our work not only bridges the gap between electronic nematicity and Kagome physics, but also sheds light on the potential mechanism for realizing symmetry-broken phases in correlated electron systems. △ Less

Submitted 17 July, 2024; v1 submitted 19 June, 2024; originally announced June 2024.

Comments: 19 pages, 5 figures

Journal ref: Nat. Mater. (2024)

Genuine Multipartite Entanglement induced by a Thermal Acoustic Reservoir

Authors: Qing-Yang Qiu, Zhi-Guang Lu, Qiongyi He, Ying Wu, Xin-You Lü

Abstract: Genuine multipartite entanglement (GME) is not only fundamental interesting for the study of quantum-to-classical transition, but also is essential for realizing universal quantum computing and quantum networks. Here we investigate the multipartite entanglement (ME) dynamics in a linear chain of N LC resonators interacting optomechanically with a common thermal acoustic reservoir. By presenting th… ▽ More Genuine multipartite entanglement (GME) is not only fundamental interesting for the study of quantum-to-classical transition, but also is essential for realizing universal quantum computing and quantum networks. Here we investigate the multipartite entanglement (ME) dynamics in a linear chain of N LC resonators interacting optomechanically with a common thermal acoustic reservoir. By presenting the exact analytical solutions of system evolution, we predict the periodic generation of non-Gaussian ME, including the discrete and continuous variables entanglement. Interestingly, the GME is obtained even though the system is in a heat bath. The mechanism relies on the special acoustic environment featuring frequency comb structure. More importantly, our proposed model also allows the periodic generation of entangled multipartite cat states (MCSs), i.e., a typical GHZ state, with high fidelity. This work fundamentally broadens the fields of ME, and have wide applications in implementing thermal-noise-resistant quantum information processing and many-body quantum simulation. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 25 pages, 9 figures

Trapezoidal Gradient Descent for Effective Reinforcement Learning in Spiking Networks

Authors: Yuhao Pan, Xiucheng Wang, Nan Cheng, Qi Qiu

Abstract: With the rapid development of artificial intelligence technology, the field of reinforcement learning has continuously achieved breakthroughs in both theory and practice. However, traditional reinforcement learning algorithms often entail high energy consumption during interactions with the environment. Spiking Neural Network (SNN), with their low energy consumption characteristics and performance… ▽ More With the rapid development of artificial intelligence technology, the field of reinforcement learning has continuously achieved breakthroughs in both theory and practice. However, traditional reinforcement learning algorithms often entail high energy consumption during interactions with the environment. Spiking Neural Network (SNN), with their low energy consumption characteristics and performance comparable to deep neural networks, have garnered widespread attention. To reduce the energy consumption of practical applications of reinforcement learning, researchers have successively proposed the Pop-SAN and MDC-SAN algorithms. Nonetheless, these algorithms use rectangular functions to approximate the spike network during the training process, resulting in low sensitivity, thus indicating room for improvement in the training effectiveness of SNN. Based on this, we propose a trapezoidal approximation gradient method to replace the spike network, which not only preserves the original stable learning state but also enhances the model's adaptability and response sensitivity under various signal dynamics. Simulation results show that the improved algorithm, using the trapezoidal approximation gradient to replace the spike network, achieves better convergence speed and performance compared to the original algorithm and demonstrates good training stability. △ Less

Submitted 19 June, 2024; originally announced June 2024.

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU

Authors: Qi Qiu, Tao Zhu, Helin Gong, Liming Chen, Huansheng Ning

Abstract: Limited by the complexity of basis function (B-spline) calculations, Kolmogorov-Arnold Networks (KAN) suffer from restricted parallel computing capability on GPUs. This paper proposes a novel ReLU-KAN implementation that inherits the core idea of KAN. By adopting ReLU (Rectified Linear Unit) and point-wise multiplication, we simplify the design of KAN's basis function and optimize the computation… ▽ More Limited by the complexity of basis function (B-spline) calculations, Kolmogorov-Arnold Networks (KAN) suffer from restricted parallel computing capability on GPUs. This paper proposes a novel ReLU-KAN implementation that inherits the core idea of KAN. By adopting ReLU (Rectified Linear Unit) and point-wise multiplication, we simplify the design of KAN's basis function and optimize the computation process for efficient CUDA computing. The proposed ReLU-KAN architecture can be readily implemented on existing deep learning frameworks (e.g., PyTorch) for both inference and training. Experimental results demonstrate that ReLU-KAN achieves a 20x speedup compared to traditional KAN with 4-layer networks. Furthermore, ReLU-KAN exhibits a more stable training process with superior fitting ability while preserving the "catastrophic forgetting avoidance" property of KAN. You can get the code in https://github.com/quiqi/relu_kan △ Less

Submitted 12 August, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Using magnetic dynamics to measure the spin gap in a candidate Kitaev material

Authors: Xinyi Jiang, Qingzheng Qiu, Cheng Peng, Hoyoung Jang, Wenjie Chen, Xianghong Jin, Li Yue, Byungjune Lee, Sang-Youn Park, Minseok Kim, Hyeong-Do Kim, Xinqiang Cai, Qizhi Li, Tao Dong, Nanlin Wang, Joshua J. Turner, Yuan Li, Yao Wang, Yingying Peng

Abstract: Materials potentially hosting Kitaev spin-liquid states are considered crucial for realizing topological quantum computing. However, the intricate nature of spin interactions within these materials complicates the precise measurement of low-energy spin excitations indicative of fractionalized excitations. Using Na$_{2}$Co$_2$TeO$_{6}$ as an example, we study these low-energy spin excitations using… ▽ More Materials potentially hosting Kitaev spin-liquid states are considered crucial for realizing topological quantum computing. However, the intricate nature of spin interactions within these materials complicates the precise measurement of low-energy spin excitations indicative of fractionalized excitations. Using Na$_{2}$Co$_2$TeO$_{6}$ as an example, we study these low-energy spin excitations using the time-resolved resonant elastic x-ray scattering (tr-REXS). Our observations unveil remarkably slow spin dynamics at the magnetic peak, whose recovery timescale is several nanoseconds. This timescale aligns with the extrapolated spin gap of $\sim$ 1 $μ$eV, obtained by density matrix renormalization group (DMRG) simulations in the thermodynamic limit. The consistency demonstrates the efficacy of tr-REXS in discerning low-energy spin gaps inaccessible to conventional spectroscopic techniques. △ Less

Submitted 6 May, 2024; originally announced May 2024.

Comments: 9 pages, 6 figures

Wide Binary Evaporation by Dark Solitons: Implications from the GAIA Catalog

Authors: Qiming Qiu, Yu Gao, Hai-jun Tian, Kechen Wang, Zihang Wang, Xiang-Ming Yang

Abstract: An analytic calculation is given for binary star evaporation under the tidal perturbation from randomly distributed, spatially extended dark objects. In particular, the Milky Way's wide binary star population are susceptible to such disruption from dark matter solitons of comparable and larger sizes. We identify high-probability `halo-like' wide binaries in GAIA EDR3 with separations larger than 0… ▽ More An analytic calculation is given for binary star evaporation under the tidal perturbation from randomly distributed, spatially extended dark objects. In particular, the Milky Way's wide binary star population are susceptible to such disruption from dark matter solitons of comparable and larger sizes. We identify high-probability `halo-like' wide binaries in GAIA EDR3 with separations larger than 0.1 parsec. Survival of the farthest-separated candidates will provide a novel gravitational probe to dark matter in the form of solitons. In case of dilute axion-like solitons, the observational sensitivity is shown to extend into the axion mass range $m_a \sim 10^{-17}-10^{-15}$ eV. △ Less

Submitted 28 April, 2024; originally announced April 2024.

Comments: 15 pages, 4 figures, 3 tables

Heuristic Optimization of Amplifier Reconfiguration Process for Autonomous Driving Optical Networks

Authors: Qizhi Qiu, Xiaomin Liu, Yihao Zhang, Lilin Yi, Weisheng Hu, Qunbi Zhuge

Abstract: We propose a heuristic-based optimization scheme for reliable optical amplifier reconfiguration process in ADON. In the experiment on a commercial testbed, the scheme prevents a 1.0-dB Q-factor degradation and outperforms 98.5% random solutions. We propose a heuristic-based optimization scheme for reliable optical amplifier reconfiguration process in ADON. In the experiment on a commercial testbed, the scheme prevents a 1.0-dB Q-factor degradation and outperforms 98.5% random solutions. △ Less

Submitted 18 July, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Model-Agnostic Human Preference Inversion in Diffusion Models

Authors: Jeeyung Kim, Ze Wang, Qiang Qiu

Abstract: Efficient text-to-image generation remains a challenging task due to the high computational costs associated with the multi-step sampling in diffusion models. Although distillation of pre-trained diffusion models has been successful in reducing sampling steps, low-step image generation often falls short in terms of quality. In this study, we propose a novel sampling design to achieve high-quality… ▽ More Efficient text-to-image generation remains a challenging task due to the high computational costs associated with the multi-step sampling in diffusion models. Although distillation of pre-trained diffusion models has been successful in reducing sampling steps, low-step image generation often falls short in terms of quality. In this study, we propose a novel sampling design to achieve high-quality one-step image generation aligning with human preferences, particularly focusing on exploring the impact of the prior noise distribution. Our approach, Prompt Adaptive Human Preference Inversion (PAHI), optimizes the noise distributions for each prompt based on human preferences without the need for fine-tuning diffusion models. Our experiments showcase that the tailored noise distributions significantly improve image quality with only a marginal increase in computational cost. Our findings underscore the importance of noise optimization and pave the way for efficient and high-quality text-to-image synthesis. △ Less

Submitted 31 March, 2024; originally announced April 2024.

First Principles Studies of Stacking Fault Energies in Ternary Magnesium Alloys

Authors: Qiwen Qiu, Stephen Yue, Jun Song

Abstract: Magnesium (Mg) alloys have emerged as promising materials due to their low density and high strength-to-weight ratio, offering a wide range of applications across multiple industries. Nevertheless, the inherent brittleness of Mg alloys poses a significant hurdle, necessitating innovative approaches to enhance their mechanical performance. Among the various strategies, manipulating stacking fault e… ▽ More Magnesium (Mg) alloys have emerged as promising materials due to their low density and high strength-to-weight ratio, offering a wide range of applications across multiple industries. Nevertheless, the inherent brittleness of Mg alloys poses a significant hurdle, necessitating innovative approaches to enhance their mechanical performance. Among the various strategies, manipulating stacking fault energy (SFE) has been a key focus, although primarily within the realm of binary alloys. This study investigates SFE in Mg alloys, focusing on ternary compositions. Utilizing first-principles DFT calculations, we analyze solute interactions and their influence on SFE, particularly in Mg-Al-X and Mg-Zn-X configurations. Predictive models are developed for estimating SFE effects, revealing solute pairs that mimic rare earth elements and show potential for improved ductility. The findings contribute to fundamental insights into Mg alloy behavior, offering practical directions for designing advanced materials with superior mechanical properties. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Comments: 29 pages, 15 figures

Generative Quanta Color Imaging

Authors: Vishal Purohit, Junjie Luo, Yiheng Chi, Qi Guo, Stanley H. Chan, Qiang Qiu

Abstract: The astonishing development of single-photon cameras has created an unprecedented opportunity for scientific and industrial imaging. However, the high data throughput generated by these 1-bit sensors creates a significant bottleneck for low-power applications. In this paper, we explore the possibility of generating a color image from a single binary frame of a single-photon camera. We evidently fi… ▽ More The astonishing development of single-photon cameras has created an unprecedented opportunity for scientific and industrial imaging. However, the high data throughput generated by these 1-bit sensors creates a significant bottleneck for low-power applications. In this paper, we explore the possibility of generating a color image from a single binary frame of a single-photon camera. We evidently find this problem being particularly difficult to standard colorization approaches due to the substantial degree of exposure variation. The core innovation of our paper is an exposure synthesis model framed under a neural ordinary differential equation (Neural ODE) that allows us to generate a continuum of exposures from a single observation. This innovation ensures consistent exposure in binary images that colorizers take on, resulting in notably enhanced colorization. We demonstrate applications of the method in single-image and burst colorization and show superior generative performance over baselines. Project website can be found at https://vishal-s-p.github.io/projects/2023/generative_quanta_color.html. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: Accepted at IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

A Novel Quantum Algorithm for Ant Colony Optimization

Authors: Qian Qiu, Mohan Wu, Qichun Sun, Xiaogang Li, Hua Xu

Abstract: Quantum ant colony optimization (QACO) has drew much attention since it combines the advantages of quantum computing and ant colony optimization (ACO) algorithms and overcomes some limitations of the traditional ACO algorithm. However, due to the hardware resource limitations of currently available quantum computers, such as the limited number of qubits, lack of high-fidelity gating operation, and… ▽ More Quantum ant colony optimization (QACO) has drew much attention since it combines the advantages of quantum computing and ant colony optimization (ACO) algorithms and overcomes some limitations of the traditional ACO algorithm. However, due to the hardware resource limitations of currently available quantum computers, such as the limited number of qubits, lack of high-fidelity gating operation, and low noisy tolerance, the practical application of the QACO is quite challenging. In this paper, we introduce a hybrid quantum-classical algorithm by combining the clustering algorithm with QACO algorithm, so that this extended QACO can handle large-scale optimization problems, which makes the practical application of QACO based on available quantum computation resource possible. To verify the effectiveness and performance of the algorithm, we tested the developed QACO algorithm with the Travelling Salesman Problem (TSP) as benchmarks. The developed QACO algorithm shows better performance under multiple data set. In addition, the developed QACO algorithm also manifests the robustness to noise of calculation process, which is typically a major barrier for practical application of quantum computers. Our work shows that the combination of the clustering algorithm with QACO has effectively extended the application scenario of QACO in current NISQ era of quantum computing. △ Less

Submitted 1 March, 2024; originally announced March 2024.

Large Convolutional Model Tuning via Filter Subspace

Authors: Wei Chen, Zichen Miao, Qiang Qiu

Abstract: Efficient fine-tuning methods are critical to address the high computational and parameter complexity while adapting large pre-trained models to downstream tasks. Our study is inspired by prior research that represents each convolution filter as a linear combination of a small set of filter subspace elements, referred to as filter atoms. In this paper, we propose to fine-tune pre-trained models by… ▽ More Efficient fine-tuning methods are critical to address the high computational and parameter complexity while adapting large pre-trained models to downstream tasks. Our study is inspired by prior research that represents each convolution filter as a linear combination of a small set of filter subspace elements, referred to as filter atoms. In this paper, we propose to fine-tune pre-trained models by adjusting only filter atoms, which are responsible for spatial-only convolution, while preserving spatially-invariant channel combination knowledge in atom coefficients. In this way, we bring a new filter subspace view for model tuning. Furthermore, each filter atom can be recursively decomposed as a combination of another set of atoms, which naturally expands the number of tunable parameters in the filter subspace. By only adapting filter atoms constructed by a small number of parameters, while maintaining the rest of model parameters constant, the proposed approach is highly parameter-efficient. It effectively preserves the capabilities of pre-trained models and prevents overfitting to downstream tasks. Extensive experiments show that such a simple scheme surpasses previous tuning baselines for both discriminate and generative tasks. △ Less

Submitted 11 March, 2024; v1 submitted 29 February, 2024; originally announced March 2024.

Training Bayesian Neural Networks with Sparse Subspace Variational Inference

Authors: Junbo Li, Zichen Miao, Qiang Qiu, Ruqi Zhang

Abstract: Bayesian neural networks (BNNs) offer uncertainty quantification but come with the downside of substantially increased training and inference costs. Sparse BNNs have been investigated for efficient inference, typically by either slowly introducing sparsity throughout the training or by post-training compression of dense BNNs. The dilemma of how to cut down massive training costs remains, particula… ▽ More Bayesian neural networks (BNNs) offer uncertainty quantification but come with the downside of substantially increased training and inference costs. Sparse BNNs have been investigated for efficient inference, typically by either slowly introducing sparsity throughout the training or by post-training compression of dense BNNs. The dilemma of how to cut down massive training costs remains, particularly given the requirement to learn about the uncertainty. To solve this challenge, we introduce Sparse Subspace Variational Inference (SSVI), the first fully sparse BNN framework that maintains a consistently highly sparse Bayesian model throughout the training and inference phases. Starting from a randomly initialized low-dimensional sparse subspace, our approach alternately optimizes the sparse subspace basis selection and its associated parameters. While basis selection is characterized as a non-differentiable problem, we approximate the optimal solution with a removal-and-addition strategy, guided by novel criteria based on weight distribution statistics. Our extensive experiments show that SSVI sets new benchmarks in crafting sparse BNNs, achieving, for instance, a 10-20x compression in model size with under 3\% performance drop, and up to 20x FLOPs reduction during training compared with dense VI training. Remarkably, SSVI also demonstrates enhanced robustness to hyperparameters, reducing the need for intricate tuning in VI and occasionally even surpassing VI-trained dense BNNs on both accuracy and uncertainty metrics. △ Less

Submitted 16 February, 2024; originally announced February 2024.

Journal ref: Published at International Conference on Learning Representations (ICLR) 2024

Predictive Temporal Attention on Event-based Video Stream for Energy-efficient Situation Awareness

Authors: Yiming Bu, Jiayang Liu, Qinru Qiu

Abstract: The Dynamic Vision Sensor (DVS) is an innovative technology that efficiently captures and encodes visual information in an event-driven manner. By combining it with event-driven neuromorphic processing, the sparsity in DVS camera output can result in high energy efficiency. However, similar to many embedded systems, the off-chip communication between the camera and processor presents a bottleneck… ▽ More The Dynamic Vision Sensor (DVS) is an innovative technology that efficiently captures and encodes visual information in an event-driven manner. By combining it with event-driven neuromorphic processing, the sparsity in DVS camera output can result in high energy efficiency. However, similar to many embedded systems, the off-chip communication between the camera and processor presents a bottleneck in terms of power consumption. Inspired by the predictive coding model and expectation suppression phenomenon found in human brain, we propose a temporal attention mechanism to throttle the camera output and pay attention to it only when the visual events cannot be well predicted. The predictive attention not only reduces power consumption in the sensor-processor interface but also effectively decreases the computational workload by filtering out noisy events. We demonstrate that the predictive attention can reduce 46.7% of data communication between the camera and the processor and reduce 43.8% computation activities in the processor. △ Less

Submitted 13 February, 2024; originally announced February 2024.

SemanticSLAM: Learning based Semantic Map Construction and Robust Camera Localization

Authors: Mingyang Li, Yue Ma, Qinru Qiu

Abstract: Current techniques in Visual Simultaneous Localization and Mapping (VSLAM) estimate camera displacement by comparing image features of consecutive scenes. These algorithms depend on scene continuity, hence requires frequent camera inputs. However, processing images frequently can lead to significant memory usage and computation overhead. In this study, we introduce SemanticSLAM, an end-to-end visu… ▽ More Current techniques in Visual Simultaneous Localization and Mapping (VSLAM) estimate camera displacement by comparing image features of consecutive scenes. These algorithms depend on scene continuity, hence requires frequent camera inputs. However, processing images frequently can lead to significant memory usage and computation overhead. In this study, we introduce SemanticSLAM, an end-to-end visual-inertial odometry system that utilizes semantic features extracted from an RGB-D sensor. This approach enables the creation of a semantic map of the environment and ensures reliable camera localization. SemanticSLAM is scene-agnostic, which means it doesn't require retraining for different environments. It operates effectively in indoor settings, even with infrequent camera input, without prior knowledge. The strength of SemanticSLAM lies in its ability to gradually refine the semantic map and improve pose estimation. This is achieved by a convolutional long-short-term-memory (ConvLSTM) network, trained to correct errors during map construction. Compared to existing VSLAM algorithms, SemanticSLAM improves pose estimation by 17%. The resulting semantic map provides interpretable information about the environment and can be easily applied to various downstream tasks, such as path planning, obstacle avoidance, and robot navigation. The code will be publicly available at https://github.com/Leomingyangli/SemanticSLAM △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: 2023 IEEE Symposium Series on Computational Intelligence (SSCI) 6 pages

Journal ref: 2023 IEEE Symposium Series on Computational Intelligence (SSCI)

An Efficient Generalizable Framework for Visuomotor Policies via Control-aware Augmentation and Privilege-guided Distillation

Authors: Yinuo Zhao, Kun Wu, Tianjiao Yi, Zhiyuan Xu, Xiaozhu Ju, Zhengping Che, Qinru Qiu, Chi Harold Liu, Jian Tang

Abstract: Visuomotor policies, which learn control mechanisms directly from high-dimensional visual observations, confront challenges in adapting to new environments with intricate visual variations. Data augmentation emerges as a promising method for bridging these generalization gaps by enriching data variety. However, straightforwardly augmenting the entire observation shall impose excessive burdens on p… ▽ More Visuomotor policies, which learn control mechanisms directly from high-dimensional visual observations, confront challenges in adapting to new environments with intricate visual variations. Data augmentation emerges as a promising method for bridging these generalization gaps by enriching data variety. However, straightforwardly augmenting the entire observation shall impose excessive burdens on policy learning and may even result in performance degradation. In this paper, we propose to improve the generalization ability of visuomotor policies as well as preserve training stability from two aspects: 1) We learn a control-aware mask through a self-supervised reconstruction task with three auxiliary losses and then apply strong augmentation only to those control-irrelevant regions based on the mask to reduce the generalization gaps. 2) To address training instability issues prevalent in visual reinforcement learning (RL), we distill the knowledge from a pretrained RL expert processing low-level environment states, to the student visuomotor policy. The policy is subsequently deployed to unseen environments without any further finetuning. We conducted comparison and ablation studies across various benchmarks: the DMControl Generalization Benchmark (DMC-GB), the enhanced Robot Manipulation Distraction Benchmark (RMDB), and a specialized long-horizontal drawer-opening robotic task. The extensive experimental results well demonstrate the effectiveness of our method, e.g., showing a 17\% improvement over previous methods in the video-hard setting of DMC-GB. △ Less

Submitted 17 January, 2024; originally announced January 2024.

SWBT: Similarity Weighted Behavior Transformer with the Imperfect Demonstration for Robotic Manipulation

Authors: Kun Wu, Ning Liu, Zhen Zhao, Di Qiu, Jinming Li, Zhengping Che, Zhiyuan Xu, Qinru Qiu, Jian Tang

Abstract: Imitation learning (IL), aiming to learn optimal control policies from expert demonstrations, has been an effective method for robot manipulation tasks. However, previous IL methods either only use expensive expert demonstrations and omit imperfect demonstrations or rely on interacting with the environment and learning from online experiences. In the context of robotic manipulation, we aim to conq… ▽ More Imitation learning (IL), aiming to learn optimal control policies from expert demonstrations, has been an effective method for robot manipulation tasks. However, previous IL methods either only use expensive expert demonstrations and omit imperfect demonstrations or rely on interacting with the environment and learning from online experiences. In the context of robotic manipulation, we aim to conquer the above two challenges and propose a novel framework named Similarity Weighted Behavior Transformer (SWBT). SWBT effectively learn from both expert and imperfect demonstrations without interaction with environments. We reveal that the easy-to-get imperfect demonstrations, such as forward and inverse dynamics, significantly enhance the network by learning fruitful information. To the best of our knowledge, we are the first to attempt to integrate imperfect demonstrations into the offline imitation learning setting for robot manipulation tasks. Extensive experiments on the ManiSkill2 benchmark built on the high-fidelity Sapien simulator and real-world robotic manipulation tasks demonstrated that the proposed method can extract better features and improve the success rates for all tasks. Our code will be released upon acceptance of the paper. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: 8 pages, 5 figures

ACM Class: I.2.9

Improved Physics-based Raman Amplifier Model in C+L Networks through Input Parameter Refinement

Authors: Yihao Zhang, Xiaomin Liu, Qizhi Qiu, Yichen Liu, Lilin Yi, Weisheng Hu, Qunbi Zhuge

Abstract: We propose a n input parameter refinement scheme for the physics-based Raman amplifier model. Experiments over C+L band are conducted. Results show the scheme can lower the physical model's maximum estimation error by 2.13 dB. We propose a n input parameter refinement scheme for the physics-based Raman amplifier model. Experiments over C+L band are conducted. Results show the scheme can lower the physical model's maximum estimation error by 2.13 dB. △ Less

Submitted 11 January, 2024; originally announced January 2024.

Comments: Accpted by Optical Fiber Communication Conference(OFC) 2024, session abbreviation: M4K

Journal ref: Optical Fiber Communication Conference (OFC) 2024, Technical Digest Series (Optica Publishing Group, 2024), paper M4K.4

Cross-Modal Reasoning with Event Correlation for Video Question Answering

Authors: Chengxiang Yin, Zhengping Che, Kun Wu, Zhiyuan Xu, Qinru Qiu, Jian Tang

Abstract: Video Question Answering (VideoQA) is a very attractive and challenging research direction aiming to understand complex semantics of heterogeneous data from two domains, i.e., the spatio-temporal video content and the word sequence in question. Although various attention mechanisms have been utilized to manage contextualized representations by modeling intra- and inter-modal relationships of the t… ▽ More Video Question Answering (VideoQA) is a very attractive and challenging research direction aiming to understand complex semantics of heterogeneous data from two domains, i.e., the spatio-temporal video content and the word sequence in question. Although various attention mechanisms have been utilized to manage contextualized representations by modeling intra- and inter-modal relationships of the two modalities, one limitation of the predominant VideoQA methods is the lack of reasoning with event correlation, that is, sensing and analyzing relationships among abundant and informative events contained in the video. In this paper, we introduce the dense caption modality as a new auxiliary and distill event-correlated information from it to infer the correct answer. To this end, we propose a novel end-to-end trainable model, Event-Correlated Graph Neural Networks (EC-GNNs), to perform cross-modal reasoning over information from the three modalities (i.e., caption, video, and question). Besides the exploitation of a brand new modality, we employ cross-modal reasoning modules for explicitly modeling inter-modal relationships and aggregating relevant information across different modalities, and we propose a question-guided self-adaptive multi-modal fusion module to collect the question-oriented and event-correlated evidence through multi-step reasoning. We evaluate our model on two widely-used benchmark datasets and conduct an ablation study to justify the effectiveness of each proposed component. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Observation of giant circular dichroism induced by electronic chirality

Authors: Qian Xiao, Oleg Janson, Sonia Francoual, Qingzheng Qiu, Qizhi Li, Shilong Zhang, Wu Xie, Pablo Bereciartua, Jeroen van den Brink, Jasper van Wezel, Yingying Peng

Abstract: Chiral phases of matter, characterized by a definite handedness, abound in nature, ranging from the crystal structure of quartz to spiraling spin states in helical magnets. In $1T$-TiSe$_2$ a source of chirality has been proposed that stands apart from these classical examples as it arises from combined electronic charge and quantum orbital fluctuations. This may allow its chirality to be accessed… ▽ More Chiral phases of matter, characterized by a definite handedness, abound in nature, ranging from the crystal structure of quartz to spiraling spin states in helical magnets. In $1T$-TiSe$_2$ a source of chirality has been proposed that stands apart from these classical examples as it arises from combined electronic charge and quantum orbital fluctuations. This may allow its chirality to be accessed and manipulated without imposing either structural or magnetic handedness. However, direct bulk evidence that broken inversion symmetry and chirality are intrinsic to TiSe$_2$ remains elusive. Here, employing resonant elastic scattering of x-rays, we reveal the presence of giant circular dichroism up to $\sim$ 40$\%$ at forbidden Bragg peaks that emerge at the charge and orbital ordering transition. The dichroism varies dramatically with incident energy and azimuthal angle. Comparison to calculated scattering intensities unambiguously traces its origin to bulk chiral electronic order in ${\mathrm{TiSe}}_2$ and establishes resonant elastic x-ray scattering as a sensitive probe to electronic chirality. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 6 pages, 4 figures

Dynamic Frequency Domain Graph Convolutional Network for Traffic Forecasting

Authors: Yujie Li, Zezhi Shao, Yongjun Xu, Qiang Qiu, Zhaogang Cao, Fei Wang

Abstract: Complex spatial dependencies in transportation networks make traffic prediction extremely challenging. Much existing work is devoted to learning dynamic graph structures among sensors, and the strategy of mining spatial dependencies from traffic data, known as data-driven, tends to be an intuitive and effective approach. However, Time-Shift of traffic patterns and noise induced by random factors h… ▽ More Complex spatial dependencies in transportation networks make traffic prediction extremely challenging. Much existing work is devoted to learning dynamic graph structures among sensors, and the strategy of mining spatial dependencies from traffic data, known as data-driven, tends to be an intuitive and effective approach. However, Time-Shift of traffic patterns and noise induced by random factors hinder data-driven spatial dependence modeling. In this paper, we propose a novel dynamic frequency domain graph convolution network (DFDGCN) to capture spatial dependencies. Specifically, we mitigate the effects of time-shift by Fourier transform, and introduce the identity embedding of sensors and time embedding when capturing data for graph learning since traffic data with noise is not entirely reliable. The graph is combined with static predefined and self-adaptive graphs during graph convolution to predict future traffic data through classical causal convolutions. Extensive experiments on four real-world datasets demonstrate that our model is effective and outperforms the baselines. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Intell-dragonfly: A Cybersecurity Attack Surface Generation Engine Based On Artificial Intelligence-generated Content Technology

Authors: Xingchen Wu, Qin Qiu, Jiaqi Li, Yang Zhao

Abstract: With the rapid development of the Internet, cyber security issues have become increasingly prominent. Traditional cyber security defense methods are limited in the face of ever-changing threats, so it is critical to seek innovative attack surface generation methods. This study proposes Intell-dragonfly, a cyber security attack surface generation engine based on artificial intelligence generation t… ▽ More With the rapid development of the Internet, cyber security issues have become increasingly prominent. Traditional cyber security defense methods are limited in the face of ever-changing threats, so it is critical to seek innovative attack surface generation methods. This study proposes Intell-dragonfly, a cyber security attack surface generation engine based on artificial intelligence generation technology, to meet the challenges of cyber security. Based on ChatGPT technology, this paper designs an automated attack surface generation process, which can generate diversified and personalized attack scenarios, targets, elements and schemes. Through experiments in a real network environment, the effect of the engine is verified and compared with traditional methods, which improves the authenticity and applicability of the attack surface. The experimental results show that the ChatGPT-based method has significant advantages in the accuracy, diversity and operability of attack surface generation. Furthermore, we explore the strengths and limitations of the engine and discuss its potential applications in the field of cyber security. This research provides a novel approach to the field of cyber security that is expected to have a positive impact on defense and prevention of cyberthreats. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Comments: 17 pages, 7 figures, 2 tables

PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learning

Authors: Wenhao Yu, Jie Peng, Quecheng Qiu, Hanyu Wang, Lu Zhang, Jianmin Ji

Abstract: Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the l… ▽ More Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the long-term execution. An alternative method is to train a DRL policy that outputs the navigation path directly. However, two roadblocks arise for training a DRL policy that outputs paths: (1) The action space for potential paths often involves higher dimensions comparing to low-level commands, which increases the difficulties of training; (2) It takes multiple time steps to track a path instead of a single time step, which requires the path to predicate the interactions of the robot w.r.t. the dynamic environment in multiple time steps. This, in turn, amplifies the challenges associated with training. In response to these challenges, we propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. Specifically, we employ specific action space discretization techniques and tailored state space representation methods to address the associated challenges. In our experiments, PathRL achieves better success rates and reduces angular rotation variability compared to other DRL navigation methods, facilitating stable and smooth robot movement. We demonstrate the competitive edge of PathRL in both real-world scenarios and multiple challenging simulation environments. △ Less

Submitted 20 October, 2023; originally announced October 2023.

UniPAD: A Universal Pre-training Paradigm for Autonomous Driving

Authors: Honghui Yang, Sha Zhang, Di Huang, Xiaoyang Wu, Haoyi Zhu, Tong He, Shixiang Tang, Hengshuang Zhao, Qibo Qiu, Binbin Lin, Xiaofei He, Wanli Ouyang

Abstract: In the context of autonomous driving, the significance of effective feature learning is widely acknowledged. While conventional 3D self-supervised pre-training methods have shown widespread success, most methods follow the ideas originally designed for 2D images. In this paper, we present UniPAD, a novel self-supervised learning paradigm applying 3D volumetric differentiable rendering. UniPAD impl… ▽ More In the context of autonomous driving, the significance of effective feature learning is widely acknowledged. While conventional 3D self-supervised pre-training methods have shown widespread success, most methods follow the ideas originally designed for 2D images. In this paper, we present UniPAD, a novel self-supervised learning paradigm applying 3D volumetric differentiable rendering. UniPAD implicitly encodes 3D space, facilitating the reconstruction of continuous 3D shape structures and the intricate appearance characteristics of their 2D projections. The flexibility of our method enables seamless integration into both 2D and 3D frameworks, enabling a more holistic comprehension of the scenes. We manifest the feasibility and effectiveness of UniPAD by conducting extensive experiments on various downstream 3D tasks. Our method significantly improves lidar-, camera-, and lidar-camera-based baseline by 9.1, 7.7, and 6.9 NDS, respectively. Notably, our pre-training pipeline achieves 73.2 NDS for 3D object detection and 79.4 mIoU for 3D semantic segmentation on the nuScenes validation set, achieving state-of-the-art results in comparison with previous methods. The code will be available at https://github.com/Nightmare-n/UniPAD. △ Less

Submitted 7 April, 2024; v1 submitted 12 October, 2023; originally announced October 2023.

Comments: CVPR2024

Probing electronic-vibrational dynamics of N2+ induced by strong-field ionization

Authors: Qian Zhang, Jing Zhao, Guangru Bai, Bin Zhang, Wenkai Tao, Qianyu Qiu, Hongbin Lei, Yue Lang, Jinlei Liu, Xiaowei Wang, Zengxiu Zhao

Abstract: The coupled electronic-vibrational dynamics of nitrogen ions induced by strong-field ionization is investigated theoretically to corroborate the recent transient X-ray K-edge absorption experiment [PRL 129, 123002 (2022)], where the population distribution of three electronic states in air lasing of N2+ was determined for the first time. By extending the ionization-coupling model to include the tr… ▽ More The coupled electronic-vibrational dynamics of nitrogen ions induced by strong-field ionization is investigated theoretically to corroborate the recent transient X-ray K-edge absorption experiment [PRL 129, 123002 (2022)], where the population distribution of three electronic states in air lasing of N2+ was determined for the first time. By extending the ionization-coupling model to include the transient absorption, we successfully reproduce the time-resolved X-ray absorption spectra of nitrogen ions observed in the experiment. By identifying the contributions from different electronic states, the study provides different interpretation revealing the significant role of excited state A arising from the strong coupling between vibrational states in strong laser fields. It indicates that the electronic population inversion occurs at least for certain alignment of nitrogen molecules. The theory helps uncovering new features of absorption from forbidden transitions during ionization and confirming that the vibration coherence at each electronic channel induces the modulation of absorbance after strong field ionization. A new scheme is proposed to determine the population transfer at different probing geometry to avoid the spectral overlap. This work offers valuable insights into the intricate interplay between electronic and vibrational dynamics and helps to resolve the debate on nitrogen air lasing. △ Less

Submitted 6 October, 2023; originally announced October 2023.

M$^3$CS: Multi-Target Masked Point Modeling with Learnable Codebook and Siamese Decoders

Authors: Qibo Qiu, Honghui Yang, Wenxiao Wang, Shun Zhang, Haiming Gao, Haochao Ying, Wei Hua, Xiaofei He

Abstract: Masked point modeling has become a promising scheme of self-supervised pre-training for point clouds. Existing methods reconstruct either the original points or related features as the objective of pre-training. However, considering the diversity of downstream tasks, it is necessary for the model to have both low- and high-level representation modeling capabilities to capture geometric details and… ▽ More Masked point modeling has become a promising scheme of self-supervised pre-training for point clouds. Existing methods reconstruct either the original points or related features as the objective of pre-training. However, considering the diversity of downstream tasks, it is necessary for the model to have both low- and high-level representation modeling capabilities to capture geometric details and semantic contexts during pre-training. To this end, M$^3$CS is proposed to enable the model with the above abilities. Specifically, with masked point cloud as input, M$^3$CS introduces two decoders to predict masked representations and the original points simultaneously. While an extra decoder doubles parameters for the decoding process and may lead to overfitting, we propose siamese decoders to keep the amount of learnable parameters unchanged. Further, we propose an online codebook projecting continuous tokens into discrete ones before reconstructing masked points. In such way, we can enforce the decoder to take effect through the combinations of tokens rather than remembering each token. Comprehensive experiments show that M$^3$CS achieves superior performance at both classification and segmentation tasks, outperforming existing methods. △ Less

Submitted 22 September, 2023; originally announced September 2023.

Charge redistribution, charge order and plasmon in La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ superlattices

Authors: Qizhi Li, Lele Ju, Hsiaoyu Huang, Yuxuan Zhang, Changwei Zou, Tianshuang Ren, A. Singh, Shilong Zhang, Qingzheng Qiu, Qian Xiao, Di-Jing Huang, Yanwu Xie, Zhen Chen, Yingying Peng

Abstract: Interfacial superconductors have the potential to revolutionize electronics, quantum computing, and fundamental physics due to their enhanced superconducting properties and ability to create new types of superconductors. The emergence of superconductivity at the interface of La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ (LSCO/LCO), with a T$_c$ enhancement of $\sim$ 10 K compared to the La… ▽ More Interfacial superconductors have the potential to revolutionize electronics, quantum computing, and fundamental physics due to their enhanced superconducting properties and ability to create new types of superconductors. The emergence of superconductivity at the interface of La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ (LSCO/LCO), with a T$_c$ enhancement of $\sim$ 10 K compared to the La$_{2-x}$Sr$_{x}$CuO$_{4}$ bulk single crystals, provides an exciting opportunity to study quantum phenomena in reduced dimensions. To investigate the carrier distribution and excitations in interfacial superconductors, we combine O K-edge resonant inelastic X-ray scattering and atomic-resolved scanning transmission electron microscopy measurements to study La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ superlattices (x=0.15, 0.45) and bulk La$_{1.55}$Sr$_{0.45}$CuO$_{4}$ films. We find direct evidence of charge redistribution, charge order and plasmon in LSCO/LCO superlattices. Notably, the observed behaviors of charge order and plasmon deviate from the anticipated properties of individual constituents or the average doping level of the superlattice. Instead, they conform harmoniously to the effective doping, a critical parameter governed by the T$_c$ of interfacial superconductors. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: 8 pages, 5 figures

Multimodal Color Recommendation in Vector Graphic Documents

Authors: Qianru Qiu, Xueting Wang, Mayu Otani

Abstract: Color selection plays a critical role in graphic document design and requires sufficient consideration of various contexts. However, recommending appropriate colors which harmonize with the other colors and textual contexts in documents is a challenging task, even for experienced designers. In this study, we propose a multimodal masked color model that integrates both color and textual contexts to… ▽ More Color selection plays a critical role in graphic document design and requires sufficient consideration of various contexts. However, recommending appropriate colors which harmonize with the other colors and textual contexts in documents is a challenging task, even for experienced designers. In this study, we propose a multimodal masked color model that integrates both color and textual contexts to provide text-aware color recommendation for graphic documents. Our proposed model comprises self-attention networks to capture the relationships between colors in multiple palettes, and cross-attention networks that incorporate both color and CLIP-based text representations. Our proposed method primarily focuses on color palette completion, which recommends colors based on the given colors and text. Additionally, it is applicable for another color recommendation task, full palette generation, which generates a complete color palette corresponding to the given text. Experimental results demonstrate that our proposed approach surpasses previous color palette completion methods on accuracy, color distribution, and user experience, as well as full palette generation methods concerning color diversity and similarity to the ground truth palettes. △ Less

Submitted 8 August, 2023; originally announced August 2023.

Comments: Accepted to ACM MM 2023

A Study of Unsupervised Evaluation Metrics for Practical and Automatic Domain Adaptation

Authors: Minghao Chen, Zepeng Gao, Shuai Zhao, Qibo Qiu, Wenxiao Wang, Binbin Lin, Xiaofei He

Abstract: Unsupervised domain adaptation (UDA) methods facilitate the transfer of models to target domains without labels. However, these methods necessitate a labeled target validation set for hyper-parameter tuning and model selection. In this paper, we aim to find an evaluation metric capable of assessing the quality of a transferred model without access to target validation labels. We begin with the met… ▽ More Unsupervised domain adaptation (UDA) methods facilitate the transfer of models to target domains without labels. However, these methods necessitate a labeled target validation set for hyper-parameter tuning and model selection. In this paper, we aim to find an evaluation metric capable of assessing the quality of a transferred model without access to target validation labels. We begin with the metric based on mutual information of the model prediction. Through empirical analysis, we identify three prevalent issues with this metric: 1) It does not account for the source structure. 2) It can be easily attacked. 3) It fails to detect negative transfer caused by the over-alignment of source and target features. To address the first two issues, we incorporate source accuracy into the metric and employ a new MLP classifier that is held out during training, significantly improving the result. To tackle the final issue, we integrate this enhanced metric with data augmentation, resulting in a novel unsupervised UDA metric called the Augmentation Consistency Metric (ACM). Additionally, we empirically demonstrate the shortcomings of previous experiment settings and conduct large-scale experiments to validate the effectiveness of our proposed metric. Furthermore, we employ our metric to automatically search for the optimal hyper-parameter set, achieving superior performance compared to manually tuned sets across four common benchmarks. Codes will be available soon. △ Less

Submitted 18 September, 2023; v1 submitted 1 August, 2023; originally announced August 2023.

Neuromorphic Online Learning for Spatiotemporal Patterns with a Forward-only Timeline

Authors: Zhenhang Zhang, Jingang Jin, Haowen Fang, Qinru Qiu

Abstract: Spiking neural networks (SNNs) are bio-plausible computing models with high energy efficiency. The temporal dynamics of neurons and synapses enable them to detect temporal patterns and generate sequences. While Backpropagation Through Time (BPTT) is traditionally used to train SNNs, it is not suitable for online learning of embedded applications due to its high computation and memory cost as well… ▽ More Spiking neural networks (SNNs) are bio-plausible computing models with high energy efficiency. The temporal dynamics of neurons and synapses enable them to detect temporal patterns and generate sequences. While Backpropagation Through Time (BPTT) is traditionally used to train SNNs, it is not suitable for online learning of embedded applications due to its high computation and memory cost as well as extended latency. Previous works have proposed online learning algorithms, but they often utilize highly simplified spiking neuron models without synaptic dynamics and reset feedback, resulting in subpar performance. In this work, we present Spatiotemporal Online Learning for Synaptic Adaptation (SOLSA), specifically designed for online learning of SNNs composed of Leaky Integrate and Fire (LIF) neurons with exponentially decayed synapses and soft reset. The algorithm not only learns the synaptic weight but also adapts the temporal filters associated to the synapses. Compared to the BPTT algorithm, SOLSA has much lower memory requirement and achieves a more balanced temporal workload distribution. Moreover, SOLSA incorporates enhancement techniques such as scheduled weight update, early stop training and adaptive synapse filter, which speed up the convergence and enhance the learning performance. When compared to other non-BPTT based SNN learning, SOLSA demonstrates an average learning accuracy improvement of 14.2%. Furthermore, compared to BPTT, SOLSA achieves a 5% higher average learning accuracy with a 72% reduction in memory cost. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 9 pages,8 figures

SelFLoc: Selective Feature Fusion for Large-scale Point Cloud-based Place Recognition

Authors: Qibo Qiu, Haiming Gao, Wenxiao Wang, Zhiyi Su, Tian Xie, Wei Hua, Xiaofei He

Abstract: Point cloud-based place recognition is crucial for mobile robots and autonomous vehicles, especially when the global positioning sensor is not accessible. LiDAR points are scattered on the surface of objects and buildings, which have strong shape priors along different axes. To enhance message passing along particular axes, Stacked Asymmetric Convolution Block (SACB) is designed, which is one of t… ▽ More Point cloud-based place recognition is crucial for mobile robots and autonomous vehicles, especially when the global positioning sensor is not accessible. LiDAR points are scattered on the surface of objects and buildings, which have strong shape priors along different axes. To enhance message passing along particular axes, Stacked Asymmetric Convolution Block (SACB) is designed, which is one of the main contributions in this paper. Comprehensive experiments demonstrate that asymmetric convolution and its corresponding strategies employed by SACB can contribute to the more effective representation of point cloud feature. On this basis, Selective Feature Fusion Block (SFFB), which is formed by stacking point- and channel-wise gating layers in a predefined sequence, is proposed to selectively boost salient local features in certain key regions, as well as to align the features before fusion phase. SACBs and SFFBs are combined to construct a robust and accurate architecture for point cloud-based place recognition, which is termed SelFLoc. Comparative experimental results show that SelFLoc achieves the state-of-the-art (SOTA) performance on the Oxford and other three in-house benchmarks with an improvement of 1.6 absolute percentages on mean average recall@1. △ Less

Submitted 5 June, 2023; v1 submitted 1 June, 2023; originally announced June 2023.

Binary Latent Diffusion

Authors: Ze Wang, Jiang Wang, Zicheng Liu, Qiang Qiu

Abstract: In this paper, we show that a binary latent space can be explored for compact yet expressive image representations. We model the bi-directional mappings between an image and the corresponding latent binary representation by training an auto-encoder with a Bernoulli encoding distribution. On the one hand, the binary latent space provides a compact discrete image representation of which the distribu… ▽ More In this paper, we show that a binary latent space can be explored for compact yet expressive image representations. We model the bi-directional mappings between an image and the corresponding latent binary representation by training an auto-encoder with a Bernoulli encoding distribution. On the one hand, the binary latent space provides a compact discrete image representation of which the distribution can be modeled more efficiently than pixels or continuous latent representations. On the other hand, we now represent each image patch as a binary vector instead of an index of a learned cookbook as in discrete image representations with vector quantization. In this way, we obtain binary latent representations that allow for better image quality and high-resolution image representations without any multi-stage hierarchy in the latent space. In this binary latent space, images can now be generated effectively using a binary latent diffusion model tailored specifically for modeling the prior over the binary image representations. We present both conditional and unconditional image generation experiments with multiple datasets, and show that the proposed method performs comparably to state-of-the-art methods while dramatically improving the sampling efficiency to as few as 16 steps without using any test-time acceleration. The proposed framework can also be seamlessly scaled to $1024 \times 1024$ high-resolution image generation without resorting to latent hierarchy or multi-stage refinements. △ Less

Submitted 10 April, 2023; originally announced April 2023.

DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance

Authors: Longwen Zhang, Qiwei Qiu, Hongyang Lin, Qixuan Zhang, Cheng Shi, Wei Yang, Ye Shi, Sibei Yang, Lan Xu, Jingyi Yu

Abstract: Emerging Metaverse applications demand accessible, accurate, and easy-to-use tools for 3D digital human creations in order to depict different cultures and societies as if in the physical world. Recent large-scale vision-language advances pave the way to for novices to conveniently customize 3D content. However, the generated CG-friendly assets still cannot represent the desired facial traits for… ▽ More Emerging Metaverse applications demand accessible, accurate, and easy-to-use tools for 3D digital human creations in order to depict different cultures and societies as if in the physical world. Recent large-scale vision-language advances pave the way to for novices to conveniently customize 3D content. However, the generated CG-friendly assets still cannot represent the desired facial traits for human characteristics. In this paper, we present DreamFace, a progressive scheme to generate personalized 3D faces under text guidance. It enables layman users to naturally customize 3D facial assets that are compatible with CG pipelines, with desired shapes, textures, and fine-grained animation capabilities. From a text input to describe the facial traits, we first introduce a coarse-to-fine scheme to generate the neutral facial geometry with a unified topology. We employ a selection strategy in the CLIP embedding space, and subsequently optimize both the details displacements and normals using Score Distillation Sampling from generic Latent Diffusion Model. Then, for neutral appearance generation, we introduce a dual-path mechanism, which combines the generic LDM with a novel texture LDM to ensure both the diversity and textural specification in the UV space. We also employ a two-stage optimization to perform SDS in both the latent and image spaces to significantly provides compact priors for fine-grained synthesis. Our generated neutral assets naturally support blendshapes-based facial animations. We further improve the animation ability with personalized deformation characteristics by learning the universal expression prior using the cross-identity hypernetwork. Notably, DreamFace can generate of realistic 3D facial assets with physically-based rendering quality and rich animation ability from video footage, even for fashion icons or exotic characters in cartoons and fiction movies. △ Less

Submitted 1 April, 2023; originally announced April 2023.

Comments: Go to DreamFace project page https://sites.google.com/view/dreamface watch our video at https://youtu.be/yCuvzgGMvPM and experience DreamFace online at https://hyperhuman.top

Deep Reinforcement Learning for Localizability-Enhanced Navigation in Dynamic Human Environments

Authors: Yuan Chen, Quecheng Qiu, Xiangyu Liu, Guangda Chen, Shunyi Yao, Jie Peng, Jianmin Ji, Yanyong Zhang

Abstract: Reliable localization is crucial for autonomous robots to navigate efficiently and safely. Some navigation methods can plan paths with high localizability (which describes the capability of acquiring reliable localization). By following these paths, the robot can access the sensor streams that facilitate more accurate location estimation results by the localization algorithms. However, most of the… ▽ More Reliable localization is crucial for autonomous robots to navigate efficiently and safely. Some navigation methods can plan paths with high localizability (which describes the capability of acquiring reliable localization). By following these paths, the robot can access the sensor streams that facilitate more accurate location estimation results by the localization algorithms. However, most of these methods require prior knowledge and struggle to adapt to unseen scenarios or dynamic changes. To overcome these limitations, we propose a novel approach for localizability-enhanced navigation via deep reinforcement learning in dynamic human environments. Our proposed planner automatically extracts geometric features from 2D laser data that are helpful for localization. The planner learns to assign different importance to the geometric features and encourages the robot to navigate through areas that are helpful for laser localization. To facilitate the learning of the planner, we suggest two techniques: (1) an augmented state representation that considers the dynamic changes and the confidence of the localization results, which provides more information and allows the robot to make better decisions, (2) a reward metric that is capable to offer both sparse and dense feedback on behaviors that affect localization accuracy. Our method exhibits significant improvements in lost rate and arrival rate when tested in previously unseen environments. △ Less

Submitted 22 March, 2023; originally announced March 2023.

CrossFormer++: A Versatile Vision Transformer Hinging on Cross-scale Attention

Authors: Wenxiao Wang, Wei Chen, Qibo Qiu, Long Chen, Boxi Wu, Binbin Lin, Xiaofei He, Wei Liu

Abstract: While features of different scales are perceptually important to visual inputs, existing vision transformers do not yet take advantage of them explicitly. To this end, we first propose a cross-scale vision transformer, CrossFormer. It introduces a cross-scale embedding layer (CEL) and a long-short distance attention (LSDA). On the one hand, CEL blends each token with multiple patches of different… ▽ More While features of different scales are perceptually important to visual inputs, existing vision transformers do not yet take advantage of them explicitly. To this end, we first propose a cross-scale vision transformer, CrossFormer. It introduces a cross-scale embedding layer (CEL) and a long-short distance attention (LSDA). On the one hand, CEL blends each token with multiple patches of different scales, providing the self-attention module itself with cross-scale features. On the other hand, LSDA splits the self-attention module into a short-distance one and a long-distance counterpart, which not only reduces the computational burden but also keeps both small-scale and large-scale features in the tokens. Moreover, through experiments on CrossFormer, we observe another two issues that affect vision transformers' performance, i.e., the enlarging self-attention maps and amplitude explosion. Thus, we further propose a progressive group size (PGS) paradigm and an amplitude cooling layer (ACL) to alleviate the two issues, respectively. The CrossFormer incorporating with PGS and ACL is called CrossFormer++. Extensive experiments show that CrossFormer++ outperforms the other vision transformers on image classification, object detection, instance segmentation, and semantic segmentation tasks. The code will be available at: https://github.com/cheerss/CrossFormer. △ Less

Submitted 30 November, 2023; v1 submitted 13 March, 2023; originally announced March 2023.

Comments: 16 pages, 7 figures

Pressure-induced color change in the lutetium dihydride LuH2

Authors: Pengfei Shan, Ningning Wang, Xiquan Zheng, Qingzheng Qiu, Yingying Peng, Jinguang Cheng

Abstract: The lutetium dihydride LuH2 is stable at ambient conditions. Here we show that its color undergoes sequential changes from dark blue at ambient pressure to pink at ~2.2 GPa and then to bright red at ~4 GPa upon compression in a diamond anvil cell. Such a pressure-induced color change in LuH2 is reversible and it is very similar to that recently reported in the N-doped lutetium hydride. However, ou… ▽ More The lutetium dihydride LuH2 is stable at ambient conditions. Here we show that its color undergoes sequential changes from dark blue at ambient pressure to pink at ~2.2 GPa and then to bright red at ~4 GPa upon compression in a diamond anvil cell. Such a pressure-induced color change in LuH2 is reversible and it is very similar to that recently reported in the N-doped lutetium hydride. However, our preliminary resistance measurements on LuH2 under pressures up to 7.7 GPa evidenced no superconductivity down to 1.5 K. △ Less

Submitted 12 March, 2023; originally announced March 2023.

Journal ref: Chin. Phys. Lett. 40, 046101, 2023

Absence of localized $5d^1$ electrons in KTaO$_3$ interface superconductors

Authors: Xinqiang Cai, Jungho Kim, Leonardo Martinelli, Piero Florio, Matteo Corti, Weiliang Qiao, Yanqiu Sun, Jiasen Niu, Quentin Faure, Christoph Sahle, Qingzheng Qiu, Qian Xiao, Xiquan Zheng, Qizhi Li, Changwei Zou, Xinyi Jiang, Giacomo Ghiringhelli, Wei Han, Yanwu Xie, Yi Lu, Marco Moretti Sala, Yingying Peng

Abstract: Recently, an exciting discovery of orientation-dependent superconductivity was made in two-dimensional electron gas (2DEG) at the interfaces of LaAlO$_3$/KTaO$_3$ (LAO/KTO) or EuO/KTaO$_3$ (EuO/KTO). The superconducting transition temperature can reach a $T_c$ of up to $\sim$ 2.2 K, which is significantly higher than its 3$d$ counterpart LaAlO$_3$/SrTiO$_3$ (LAO/STO) with a $T_c$ of $\sim$ 0.2 K.… ▽ More Recently, an exciting discovery of orientation-dependent superconductivity was made in two-dimensional electron gas (2DEG) at the interfaces of LaAlO$_3$/KTaO$_3$ (LAO/KTO) or EuO/KTaO$_3$ (EuO/KTO). The superconducting transition temperature can reach a $T_c$ of up to $\sim$ 2.2 K, which is significantly higher than its 3$d$ counterpart LaAlO$_3$/SrTiO$_3$ (LAO/STO) with a $T_c$ of $\sim$ 0.2 K. However, the underlying origin remains to be understood. To uncover the nature of electrons in KTO-based interfaces, we employ x-ray absorption spectroscopy (XAS) and resonant inelastic x-ray spectroscopy (RIXS) to study LAO/KTO and EuO/KTO with different orientations. We reveal the absence of $dd$ orbital excitations in all the measured samples. Our RIXS results are well reproduced by calculations that considered itinerant $5d$ electrons hybridized with O $2p$ electrons. This suggests that there is a lack of localized Ta $5d^1$ electrons in KTO interface superconductors, which is consistent with the absence of magnetic hysteresis observed in magneto-resistance (MR) measurements. These findings offer new insights into our understanding of superconductivity in Ta $5d$ interface superconductors and their potential applications. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Comments: 8 pages, 6 figures

Journal ref: Phys. Rev. B 108, 235167 (2023)

Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation

Authors: Gaurav Patel, Konda Reddy Mopuri, Qiang Qiu

Abstract: Data-free Knowledge Distillation (DFKD) has gained popularity recently, with the fundamental idea of carrying out knowledge transfer from a Teacher neural network to a Student neural network in the absence of training data. However, in the Adversarial DFKD framework, the student network's accuracy, suffers due to the non-stationary distribution of the pseudo-samples under multiple generator update… ▽ More Data-free Knowledge Distillation (DFKD) has gained popularity recently, with the fundamental idea of carrying out knowledge transfer from a Teacher neural network to a Student neural network in the absence of training data. However, in the Adversarial DFKD framework, the student network's accuracy, suffers due to the non-stationary distribution of the pseudo-samples under multiple generator updates. To this end, at every generator update, we aim to maintain the student's performance on previously encountered examples while acquiring knowledge from samples of the current distribution. Thus, we propose a meta-learning inspired framework by treating the task of Knowledge-Acquisition (learning from newly generated samples) and Knowledge-Retention (retaining knowledge on previously met samples) as meta-train and meta-test, respectively. Hence, we dub our method as Learning to Retain while Acquiring. Moreover, we identify an implicit aligning factor between the Knowledge-Retention and Knowledge-Acquisition tasks indicating that the proposed student update strategy enforces a common gradient direction for both tasks, alleviating interference between the two objectives. Finally, we support our hypothesis by exhibiting extensive evaluation and comparison of our method with prior arts on multiple datasets. △ Less

Submitted 27 February, 2023; originally announced February 2023.

Comments: Accepted at CVPR 2023

Energy-Inspired Self-Supervised Pretraining for Vision Models

Authors: Ze Wang, Jiang Wang, Zicheng Liu, Qiang Qiu

Abstract: Motivated by the fact that forward and backward passes of a deep network naturally form symmetric mappings between input and output representations, we introduce a simple yet effective self-supervised vision model pretraining framework inspired by energy-based models (EBMs). In the proposed framework, we model energy estimation and data restoration as the forward and backward passes of a single ne… ▽ More Motivated by the fact that forward and backward passes of a deep network naturally form symmetric mappings between input and output representations, we introduce a simple yet effective self-supervised vision model pretraining framework inspired by energy-based models (EBMs). In the proposed framework, we model energy estimation and data restoration as the forward and backward passes of a single network without any auxiliary components, e.g., an extra decoder. For the forward pass, we fit a network to an energy function that assigns low energy scores to samples that belong to an unlabeled dataset, and high energy otherwise. For the backward pass, we restore data from corrupted versions iteratively using gradient-based optimization along the direction of energy minimization. In this way, we naturally fold the encoder-decoder architecture widely used in masked image modeling into the forward and backward passes of a single vision model. Thus, our framework now accepts a wide range of pretext tasks with different data corruption methods, and permits models to be pretrained from masked image modeling, patch sorting, and image restoration, including super-resolution, denoising, and colorization. We support our findings with extensive experiments, and show the proposed method delivers comparable and even better performance with remarkably fewer epochs of training compared to the state-of-the-art self-supervised vision model pretraining methods. Our findings shed light on further exploring self-supervised vision model pretraining and pretext tasks beyond masked image modeling. △ Less

Submitted 2 February, 2023; originally announced February 2023.

Journal ref: ICLR 2023

Color Recommendation for Vector Graphic Documents based on Multi-Palette Representation

Authors: Qianru Qiu, Xueting Wang, Mayu Otani, Yuki Iwazaki

Abstract: Vector graphic documents present multiple visual elements, such as images, shapes, and texts. Choosing appropriate colors for multiple visual elements is a difficult but crucial task for both amateurs and professional designers. Instead of creating a single color palette for all elements, we extract multiple color palettes from each visual element in a graphic document, and then combine them into… ▽ More Vector graphic documents present multiple visual elements, such as images, shapes, and texts. Choosing appropriate colors for multiple visual elements is a difficult but crucial task for both amateurs and professional designers. Instead of creating a single color palette for all elements, we extract multiple color palettes from each visual element in a graphic document, and then combine them into a color sequence. We propose a masked color model for color sequence completion and recommend the specified colors based on color context in multi-palette with high probability. We train the model and build a color recommendation system on a large-scale dataset of vector graphic documents. The proposed color recommendation method outperformed other state-of-the-art methods by both quantitative and qualitative evaluations on color prediction and our color recommendation system received positive feedback from professional designers in an interview study. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Comments: Accepted to WACV 2023

PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer

Authors: Qibo Qiu, Haiming Gao, Wei Hua, Gang Huang, Xiaofei He

Abstract: Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper… ▽ More Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper proposes a novel and general framework called PriorLane, which is used to enhance the segmentation performance of the fully vision transformer by introducing the low-cost local prior knowledge. Specifically, PriorLane utilizes an encoder-only transformer to fuse the feature extracted by a pre-trained segmentation model with prior knowledge embeddings. Note that a Knowledge Embedding Alignment (KEA) module is adapted to enhance the fusion performance by aligning the knowledge embedding. Extensive experiments on our Zjlab dataset show that PriorLane outperforms SOTA lane detection methods by a 2.82% mIoU when prior knowledge is employed, and the code will be released at: https://github.com/vincentqqb/PriorLane. △ Less

Submitted 7 February, 2023; v1 submitted 14 September, 2022; originally announced September 2022.

Comments: Accepted by ICRA 2023

Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition

Authors: Gaurav Patel, Jan Allebach, Qiang Qiu

Abstract: This paper looks at semi-supervised learning (SSL) for image-based text recognition. One of the most popular SSL approaches is pseudo-labeling (PL). PL approaches assign labels to unlabeled data before re-training the model with a combination of labeled and pseudo-labeled data. However, PL methods are severely degraded by noise and are prone to over-fitting to noisy labels, due to the inclusion of… ▽ More This paper looks at semi-supervised learning (SSL) for image-based text recognition. One of the most popular SSL approaches is pseudo-labeling (PL). PL approaches assign labels to unlabeled data before re-training the model with a combination of labeled and pseudo-labeled data. However, PL methods are severely degraded by noise and are prone to over-fitting to noisy labels, due to the inclusion of erroneous high confidence pseudo-labels generated from poorly calibrated models, thus, rendering threshold-based selection ineffective. Moreover, the combinatorial complexity of the hypothesis space and the error accumulation due to multiple incorrect autoregressive steps posit pseudo-labeling challenging for sequence models. To this end, we propose a pseudo-label generation and an uncertainty-based data selection framework for semi-supervised text recognition. We first use Beam-Search inference to yield highly probable hypotheses to assign pseudo-labels to the unlabelled examples. Then we adopt an ensemble of models, sampled by applying dropout, to obtain a robust estimate of the uncertainty associated with the prediction, considering both the character-level and word-level predictive distribution to select good quality pseudo-labels. Extensive experiments on several benchmark handwriting and scene-text datasets show that our method outperforms the baseline approaches and the previous state-of-the-art semi-supervised text-recognition methods. △ Less

Submitted 6 October, 2022; v1 submitted 30 August, 2022; originally announced September 2022.

Comments: Accepted at WACV 2023

Prevailing charge order in overdoped cuprates beyond the superconducting dome

Authors: Qizhi Li, Hsiao-Yu Huang, Tianshuang Ren, Eugen Weschke, Lele Ju, Changwei Zou, Shilong Zhang, Qingzheng Qiu, Jiarui Liu, Shuhan Ding, Amol Singh, Oleksandr Prokhnenko, Di-Jing Huang, Ilya Esterlis, Yao Wang, Yanwu Xie, Yingying Peng

Abstract: The extremely overdoped cuprates are generally considered to be Fermi liquid metals without exotic orders, whereas the underdoped cuprates harbor intertwined states. Contrary to this conventional wisdom, using Cu $L_3$ edge and O $K$ edge resonant x-ray scattering, we reveal a charge order (CO) in overdoped La$_{2-x}$Sr$_x$CuO$_4$ (0.35 $\leq$ x $\leq$ 0.6) beyond the superconducting dome. This CO… ▽ More The extremely overdoped cuprates are generally considered to be Fermi liquid metals without exotic orders, whereas the underdoped cuprates harbor intertwined states. Contrary to this conventional wisdom, using Cu $L_3$ edge and O $K$ edge resonant x-ray scattering, we reveal a charge order (CO) in overdoped La$_{2-x}$Sr$_x$CuO$_4$ (0.35 $\leq$ x $\leq$ 0.6) beyond the superconducting dome. This CO has a periodicity of $\sim$ 6 lattice units with correlation lengths of $\sim 3 - 20$ lattice units. It shows similar in-plane momentum and polarization dependence and dispersive excitations as the CO of underdoped cuprates, but its maximum intensity differs along the c-direction and persists up to 300 K. This CO cannot be explained by either the Fermi surface instability or the doped Hubbard model and its origin remains to be understood. Our results suggest that CO is prevailing in the overdoped metallic regime and superconductivity emerges out of the CO phase upon decreasing hole carriers. △ Less

Submitted 17 October, 2022; v1 submitted 18 August, 2022; originally announced August 2022.

Comments: 6 pages, 4 figures

Journal ref: Physical Review Letters 131, 116002 (2023)