Showing 1–50 of 1,019 results for author: Hong, J

Observations of the Formation and Disappearance of a Funnel Prominence

Authors: Bo Yang, Jiayan Yang, Yi Bi, Junchao Hong

Abstract: We present an observational study of the formation and disappearance of a funnel prominence. Before the funnel prominence formed, cool materials from the top of a preexisting polar crown prominence flowed along saddle-shaped coronal loops to their base, forming a smaller prominence. Meanwhile, the saddle-shaped coronal loops gradually rose, and U-shaped coronal loops, termed prominence horns, bega… ▽ More We present an observational study of the formation and disappearance of a funnel prominence. Before the funnel prominence formed, cool materials from the top of a preexisting polar crown prominence flowed along saddle-shaped coronal loops to their base, forming a smaller prominence. Meanwhile, the saddle-shaped coronal loops gradually rose, and U-shaped coronal loops, termed prominence horns, began to appear along with a coronal cavity. Afterwards, a cool column emerged from the chromosphere, rose vertically into the corona, and then moved laterally to be transported into the U-shaped coronal loops. The formed prominence slid into the chromosphere, while the U-shaped coronal loops and the coronal cavity became more pronounced. As cool materials accumulated at the base of the U-shaped coronal loops, these loops underwent a significant descent and a V-shaped structure appeared at the base of the cool materials, indicating that the U-shaped coronal loops may be dragged down to sag. Subsequently, cool materials from the V-shaped structure continued to flow almost vertically toward the chromosphere, forming the funnel prominence. The vertical downflows might be produced by magnetic reconnection within or between the sagging field lines. Due to persistent vertical downflows, the U-shaped coronal loops were lifted up and prominence materials followed along inclined coronal loops towards the chromosphere, causing the funnel prominence to disappear. Our observations suggest that chromospheric plasma transported into a coronal cavity and then drained out via vertical downflows can form a funnel prominence. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 18 pages, 5figures, and accept for publication in The Astrophysical Journal Letters

PRePair: Pointwise Reasoning Enhance Pairwise Evaluating for Robust Instruction-Following Assessments

Authors: Hawon Jeong, ChaeHun Park, Jimin Hong, Jaegul Choo

Abstract: Pairwise evaluation using large language models (LLMs) is widely used for evaluating natural language generation (NLG) tasks. However, the reliability of LLMs is often compromised by biases, such as favoring verbosity and authoritative tone. In the study, we focus on the comparison of two LLM-based evaluation approaches, pointwise and pairwise. Our findings demonstrate that pointwise evaluators ex… ▽ More Pairwise evaluation using large language models (LLMs) is widely used for evaluating natural language generation (NLG) tasks. However, the reliability of LLMs is often compromised by biases, such as favoring verbosity and authoritative tone. In the study, we focus on the comparison of two LLM-based evaluation approaches, pointwise and pairwise. Our findings demonstrate that pointwise evaluators exhibit more robustness against undesirable preferences. Further analysis reveals that pairwise evaluators can accurately identify the shortcomings of low-quality outputs even when their judgment is incorrect. These results indicate that LLMs are more severely influenced by their bias in a pairwise evaluation setup. To mitigate this, we propose a hybrid method that integrates pointwise reasoning into pairwise evaluation. Experimental results show that our method enhances the robustness of pairwise evaluators against adversarial samples while preserving accuracy on normal samples. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Advancing Solar Flare Prediction using Deep Learning with Active Region Patches

Authors: Chetraj Pandey, Temitope Adeyeha, Jinsu Hong, Rafal A. Angryk, Berkay Aydin

Abstract: In this paper, we introduce a novel methodology for leveraging shape-based characteristics of magnetograms of active region (AR) patches and provide a novel capability for predicting solar flares covering the entirety of the solar disk (AR patches spanning from -90$^{\circ}$ to +90$^{\circ}$ of solar longitude). We create three deep learning models: (i) ResNet34, (ii) MobileNet, and (iii) MobileVi… ▽ More In this paper, we introduce a novel methodology for leveraging shape-based characteristics of magnetograms of active region (AR) patches and provide a novel capability for predicting solar flares covering the entirety of the solar disk (AR patches spanning from -90$^{\circ}$ to +90$^{\circ}$ of solar longitude). We create three deep learning models: (i) ResNet34, (ii) MobileNet, and (iii) MobileViT to predict $\geq$M-class flares and assess the efficacy of these models across various ranges of solar longitude. Given the inherent imbalance in our data, we employ augmentation techniques alongside undersampling during the model training phase, while maintaining imbalanced partitions in the testing data for realistic evaluation. We use a composite skill score (CSS) as our evaluation metric, computed as the geometric mean of the True Skill Score (TSS) and the Heidke Skill Score (HSS) to rank and compare models. The primary contributions of this work are as follows: (i) We introduce a novel capability in solar flare prediction that allows predicting flares for each ARs throughout the solar disk and evaluate and compare the performance, (ii) Our candidate model (MobileNet) achieves a CSS=0.51 (TSS=0.60 and HSS=0.44), CSS=0.51 (TSS=0.59 and HSS=0.44), and CSS=0.48 (TSS=0.56 and HSS=0.40) for AR patches within $\pm$30$^{\circ}$, $\pm$60$^{\circ}$, $\pm$90$^{\circ}$ of solar longitude respectively. Additionally, we demonstrate the ability to issue flare forecasts for ARs in near-limb regions (regions between $\pm$60$^{\circ}$ to $\pm$90 $^{\circ}$) with a CSS=0.39 (TSS=0.48 and HSS=0.32), expanding the scope of AR-based models for solar flare prediction. This advancement opens new avenues for more reliable prediction of solar flares, thereby contributing to improved forecasting capabilities. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Comments: This is a preprint submitted to European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, (ECML PKDD), 2024

GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning

Authors: Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, Bo Li

Abstract: The rapid advancement of large language models (LLMs) has catalyzed the deployment of LLM-powered agents across numerous applications, raising new concerns regarding their safety and trustworthiness. Existing methods for enhancing the safety of LLMs are not directly transferable to LLM-powered agents due to their diverse objectives and output modalities. In this paper, we propose GuardAgent, the f… ▽ More The rapid advancement of large language models (LLMs) has catalyzed the deployment of LLM-powered agents across numerous applications, raising new concerns regarding their safety and trustworthiness. Existing methods for enhancing the safety of LLMs are not directly transferable to LLM-powered agents due to their diverse objectives and output modalities. In this paper, we propose GuardAgent, the first LLM agent as a guardrail to other LLM agents. Specifically, GuardAgent oversees a target LLM agent by checking whether its inputs/outputs satisfy a set of given guard requests defined by the users. GuardAgent comprises two steps: 1) creating a task plan by analyzing the provided guard requests, and 2) generating guardrail code based on the task plan and executing the code by calling APIs or using external engines. In both steps, an LLM is utilized as the core reasoning component, supplemented by in-context demonstrations retrieved from a memory module. Such knowledge-enabled reasoning allows GuardAgent to understand various textual guard requests and accurately "translate" them into executable code that provides reliable guardrails. Furthermore, GuardAgent is equipped with an extendable toolbox containing functions and APIs and requires no additional LLM training, which underscores its generalization capabilities and low operational overhead. Additionally, we propose two novel benchmarks: an EICU-AC benchmark for assessing privacy-related access control for healthcare agents and a Mind2Web-SC benchmark for safety evaluation for web agents. We show the effectiveness of GuardAgent on these two benchmarks with 98.7% and 90.0% accuracy in moderating invalid inputs and outputs for the two types of agents, respectively. We also show that GuardAgent is able to define novel functions in adaption to emergent LLM agents and guard requests, which underscores its strong generalization capabilities. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Field investigation of 3D snow settling dynamics under weak atmospheric turbulence

Authors: Jiaqi Li, Michele Guala, Jiarong Hong

Abstract: Research on settling dynamics of snow particles, considering their complex morphologies and real atmospheric conditions, remains scarce despite extensive simulations and laboratory studies. Our study bridges the gap through a comprehensive field investigation into the three-dimensional (3D) snow settling dynamics under weak atmospheric turbulence, enabled by a 3D particle tracking velocimetry (PTV… ▽ More Research on settling dynamics of snow particles, considering their complex morphologies and real atmospheric conditions, remains scarce despite extensive simulations and laboratory studies. Our study bridges the gap through a comprehensive field investigation into the three-dimensional (3D) snow settling dynamics under weak atmospheric turbulence, enabled by a 3D particle tracking velocimetry (PTV) system to record > a million trajectories, coupled with a snow particle analyzer for simultaneous aerodynamic property characterization of four distinct snow types (aggregates, graupels, dendrites, needles). Our findings indicate that while the terminal velocity predicted by the aerodynamic model aligns well with PTV-measured settling velocity for graupels, significant discrepancies arise for non-spherical particles, particularly dendrites, which exhibit higher drag coefficients than predicted. Qualitative observations of 3D settling trajectories highlight pronounced meandering in aggregates and dendrites, in contrast to the subtler meandering observed in needles and graupels, attributable to their smaller frontal areas. This meandering in aggregates and dendrites occurs at lower frequencies compared to that of graupels. Further quantification of trajectory acceleration and curvature suggests that the meandering frequencies in aggregates and dendrites are smaller than that of morphology-induced vortex shedding of disks, likely due to their rotational inertia, and those of graupels align with the small-scale atmospheric turbulence. Moreover, our analysis of vertical acceleration along trajectories elucidates that the orientation changes in dendrites and aggregates enhance their settling velocity. Such insights into settling dynamics refine models of snow settling velocity under weak atmospheric turbulence, with broader implications for more accurately predicting ground snow accumulation. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 32 pages, 15 figures

MOCCA: Global properties of tidally filling and underfilling globular star clusters with multiple stellar populations

Authors: Arkadiusz Hypki, Enrico Vesperini, Mirek Giersz, Jongsuk Hong, Abbas Askar, Magdalena Otulakowska-Hypka, Lucas Hellstrom, Grzegorz Wiktorowicz

Abstract: We explore the evolution of various properties of multiple-population globular clusters (GCs) for a broad range of initial conditions. We simulated over 200 GC models using the MOCCA Monte Carlo code and find that present-day properties (core and half-light radii, ratio of the number of second-generation (SG) stars to the total number of stars, NSG/NTOT) of these models cover the observed values o… ▽ More We explore the evolution of various properties of multiple-population globular clusters (GCs) for a broad range of initial conditions. We simulated over 200 GC models using the MOCCA Monte Carlo code and find that present-day properties (core and half-light radii, ratio of the number of second-generation (SG) stars to the total number of stars, NSG/NTOT) of these models cover the observed values of these quantities for Milky Way GCs. Starting with a relatively small value of the SG fraction (NSG/NTOT ~ 0.25) and a SG system concentrated in the inner regions of the cluster, we find, in agreement with previous studies, that systems in which the first-generation (FG) is initially tidally filling or slightly tidally underfilling best reproduce the observed ratios of NSG/NTOT and have values of the core and half-light radii typical of those of many Galactic globular clusters. Models in which the FG is initially tidally underfilling retain values of NSG/NTOT close to their initial values. These simulations expand previous investigations and serve to further constrain the viable range of initial parameters and better understand their influence on present-day GC properties. The results of this investigation also provide the basis for our future survey aimed at building specific models to reproduce the observed trends (or lack thereof) between the properties of multiple stellar populations and other clusters properties. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: Accepted to Astronomy and Astrophysics (A&A)

Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation

Authors: Se Jin Park, Chae Won Kim, Hyeongseop Rha, Minsu Kim, Joanna Hong, Jeong Hun Yeo, Yong Man Ro

Abstract: In this paper, we introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response, marking the initial step towards creating an avatar chatbot system without relying on intermediate text. To this end, we newly introduce MultiDialog, the first large-scale multimodal (i.e., audio and visual) spoken dialogue corp… ▽ More In this paper, we introduce a novel Face-to-Face spoken dialogue model. It processes audio-visual speech from user input and generates audio-visual speech as the response, marking the initial step towards creating an avatar chatbot system without relying on intermediate text. To this end, we newly introduce MultiDialog, the first large-scale multimodal (i.e., audio and visual) spoken dialogue corpus containing 340 hours of approximately 9,000 dialogues, recorded based on the open domain dialogue dataset, TopicalChat. The MultiDialog contains parallel audio-visual recordings of conversation partners acting according to the given script with emotion annotations, which we expect to open up research opportunities in multimodal synthesis. Our Face-to-Face spoken dialogue model incorporates a textually pretrained large language model and adapts it into the audio-visual spoken dialogue domain by incorporating speech-text joint pretraining. Through extensive experiments, we validate the effectiveness of our model in facilitating a face-to-face conversation. Demo and data are available at https://multidialog.github.io and https://huggingface.co/datasets/IVLLab/MultiDialog, respectively. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: Accepted to ACL 2024

Some quenched and annealed limit theorems of superprocesses in random environments

Authors: Zeteng Fan, Jieliang Hong, Jie Xiong

Abstract: Let $X=(X_t, t\geq 0)$ be a superprocess in a random environment described by a Gaussian noise $W=\{W(t,x), t\geq 0, x\in \mathbb{R}^d\}$ white in time and colored in space with correlation kernel $g(x,y)$. When $d\geq 3$, under the condition that the correlation function $g(x,y)$ is bounded above by some appropriate function $\bar{g}(x-y)$, we present the quenched and annealed Strong Law of Large… ▽ More Let $X=(X_t, t\geq 0)$ be a superprocess in a random environment described by a Gaussian noise $W=\{W(t,x), t\geq 0, x\in \mathbb{R}^d\}$ white in time and colored in space with correlation kernel $g(x,y)$. When $d\geq 3$, under the condition that the correlation function $g(x,y)$ is bounded above by some appropriate function $\bar{g}(x-y)$, we present the quenched and annealed Strong Law of Large Numbers and the Central Limit Theorems regarding the weighted occupation measure $\int_0^t X_s ds$ as $t\to \infty$. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 36 pages

MSC Class: 60H15; 60G57; 60J80

Margin-aware Preference Optimization for Aligning Diffusion Models without Reference

Authors: Jiwoo Hong, Sayak Paul, Noah Lee, Kashif Rasul, James Thorne, Jongheon Jeong

Abstract: Modern alignment techniques based on human preferences, such as RLHF and DPO, typically employ divergence regularization relative to the reference model to ensure training stability. However, this often limits the flexibility of models during alignment, especially when there is a clear distributional discrepancy between the preference data and the reference model. In this paper, we focus on the al… ▽ More Modern alignment techniques based on human preferences, such as RLHF and DPO, typically employ divergence regularization relative to the reference model to ensure training stability. However, this often limits the flexibility of models during alignment, especially when there is a clear distributional discrepancy between the preference data and the reference model. In this paper, we focus on the alignment of recent text-to-image diffusion models, such as Stable Diffusion XL (SDXL), and find that this "reference mismatch" is indeed a significant problem in aligning these models due to the unstructured nature of visual modalities: e.g., a preference for a particular stylistic aspect can easily induce such a discrepancy. Motivated by this observation, we propose a novel and memory-friendly preference alignment method for diffusion models that does not depend on any reference model, coined margin-aware preference optimization (MaPO). MaPO jointly maximizes the likelihood margin between the preferred and dispreferred image sets and the likelihood of the preferred sets, simultaneously learning general stylistic features and preferences. For evaluation, we introduce two new pairwise preference datasets, which comprise self-generated image pairs from SDXL, Pick-Style and Pick-Safety, simulating diverse scenarios of reference mismatch. Our experiments validate that MaPO can significantly improve alignment on Pick-Style and Pick-Safety and general preference alignment when used with Pick-a-Pic v2, surpassing the base SDXL and other existing methods. Our code, models, and datasets are publicly available via https://mapo-t2i.github.io △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Preprint

A Novel Generative AI-Based Framework for Anomaly Detection in Multicast Messages in Smart Grid Communications

Authors: Aydin Zaboli, Seong Lok Choi, Tai-Jin Song, Junho Hong

Abstract: Cybersecurity breaches in digital substations can pose significant challenges to the stability and reliability of power system operations. To address these challenges, defense and mitigation techniques are required. Identifying and detecting anomalies in information and communication technology (ICT) is crucial to ensure secure device interactions within digital substations. This paper proposes a… ▽ More Cybersecurity breaches in digital substations can pose significant challenges to the stability and reliability of power system operations. To address these challenges, defense and mitigation techniques are required. Identifying and detecting anomalies in information and communication technology (ICT) is crucial to ensure secure device interactions within digital substations. This paper proposes a task-oriented dialogue (ToD) system for anomaly detection (AD) in datasets of multicast messages e.g., generic object oriented substation event (GOOSE) and sampled value (SV) in digital substations using large language models (LLMs). This model has a lower potential error and better scalability and adaptability than a process that considers the cybersecurity guidelines recommended by humans, known as the human-in-the-loop (HITL) process. Also, this methodology significantly reduces the effort required when addressing new cyber threats or anomalies compared with machine learning (ML) techniques, since it leaves the models complexity and precision unaffected and offers a faster implementation. These findings present a comparative assessment, conducted utilizing standard and advanced performance evaluation metrics for the proposed AD framework and the HITL process. To generate and extract datasets of IEC 61850 communications, a hardware-in-the-loop (HIL) testbed was employed. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Comments: 10 pages, 10 figures, Submitted to IEEE Transactions on Information Forensics and Security

Strategically Conservative Q-Learning

Authors: Yutaka Shimizu, Joey Hong, Sergey Levine, Masayoshi Tomizuka

Abstract: Offline reinforcement learning (RL) is a compelling paradigm to extend RL's practical utility by leveraging pre-collected, static datasets, thereby avoiding the limitations associated with collecting online interactions. The major difficulty in offline RL is mitigating the impact of approximation errors when encountering out-of-distribution (OOD) actions; doing so ineffectively will lead to polici… ▽ More Offline reinforcement learning (RL) is a compelling paradigm to extend RL's practical utility by leveraging pre-collected, static datasets, thereby avoiding the limitations associated with collecting online interactions. The major difficulty in offline RL is mitigating the impact of approximation errors when encountering out-of-distribution (OOD) actions; doing so ineffectively will lead to policies that prefer OOD actions, which can lead to unexpected and potentially catastrophic results. Despite the variety of works proposed to address this issue, they tend to excessively suppress the value function in and around OOD regions, resulting in overly pessimistic value estimates. In this paper, we propose a novel framework called Strategically Conservative Q-Learning (SCQ) that distinguishes between OOD data that is easy and hard to estimate, ultimately resulting in less conservative value estimates. Our approach exploits the inherent strengths of neural networks to interpolate, while carefully navigating their limitations in extrapolation, to obtain pessimistic yet still property calibrated value estimates. Theoretical analysis also shows that the value function learned by SCQ is still conservative, but potentially much less so than that of Conservative Q-learning (CQL). Finally, extensive evaluation on the D4RL benchmark tasks shows our proposed method outperforms state-of-the-art methods. Our code is available through \url{https://github.com/purewater0901/SCQ}. △ Less

Submitted 6 June, 2024; originally announced June 2024.

SST-GCN: The Sequential based Spatio-Temporal Graph Convolutional networks for Minute-level and Road-level Traffic Accident Risk Prediction

Authors: Tae-wook Kim, Han-jin Lee, Hyeon-Jin Jung, Ji-Woong Yang, Ellen J. Hong

Abstract: Traffic accidents are recognized as a major social issue worldwide, causing numerous injuries and significant costs annually. Consequently, methods for predicting and preventing traffic accidents have been researched for many years. With advancements in the field of artificial intelligence, various studies have applied Machine Learning and Deep Learning techniques to traffic accident prediction. M… ▽ More Traffic accidents are recognized as a major social issue worldwide, causing numerous injuries and significant costs annually. Consequently, methods for predicting and preventing traffic accidents have been researched for many years. With advancements in the field of artificial intelligence, various studies have applied Machine Learning and Deep Learning techniques to traffic accident prediction. Modern traffic conditions change rapidly by the minute, and these changes vary significantly across different roads. In other words, the risk of traffic accidents changes minute by minute in various patterns for each road. Therefore, it is desirable to predict traffic accident risk at the Minute-Level and Road-Level. However, because roads have close and complex relationships with adjacent roads, research on predicting traffic accidents at the Minute-Level and Road-Level is challenging. Thus, it is essential to build a model that can reflect the spatial and temporal characteristics of roads for traffic accident prediction. Consequently, recent attempts have been made to use Graph Convolutional Networks to capture the spatial characteristics of roads and Recurrent Neural Networks to capture their temporal characteristics for predicting traffic accident risk. This paper proposes the Sequential based Spatio-Temporal Graph Convolutional Networks (SST-GCN), which combines GCN and LSTM, to predict traffic accidents at the Minute-Level and Road-Level using a road dataset constructed in Seoul, the capital of South Korea. Experiments have demonstrated that SST-GCN outperforms other state-of-the-art models in Minute-Level predictions. △ Less

Submitted 3 June, 2024; v1 submitted 28 May, 2024; originally announced May 2024.

Probing Intracluster Dynamics and Evolution of Globular Clusters through Cataclysmic Variable Populations

Authors: Kwangmin Oh, Jongsuk Hong, C. Y. Hui, Sangin Kim, Mirek Giersz

Abstract: Dynamical interactions in globular clusters (GCs) significantly impact the formation and evolution of binary sources, including cataclysmic variables (CVs). This study investigates the connection between dynamical states of GCs and X-ray luminosity ($L_{x}$) distributions of CV populations through both simulations and actual observations. Utilizing a Monte Carlo simulation tool, MOCCA, we categori… ▽ More Dynamical interactions in globular clusters (GCs) significantly impact the formation and evolution of binary sources, including cataclysmic variables (CVs). This study investigates the connection between dynamical states of GCs and X-ray luminosity ($L_{x}$) distributions of CV populations through both simulations and actual observations. Utilizing a Monte Carlo simulation tool, MOCCA, we categorize the simulated GCs into three different evolutionary stages which are referred to as Classes I/II/III. Significant differences are found in the $L_{x}$ distributions of the CVs among these three Classes. In observational aspects, we have analyzed 179 CV candidates in 18 GCs observed by the {\it Chandra} X-ray Observatory. By dividing these GCs into three Families of different dynamical ages, namely Families I/II/III, the $L_{x}$ distributions of the CV candidates also show significant differences among these three Families. Both simulations and observational results suggest that CVs in more dynamically evolved clusters (Class/Family III) exhibit brighter X-ray emission. This highlights the influence of the dynamical status of a GC on the properties of its hosted compact binaries. Similar to blue stragglers, CV populations can serve as tracers of a GC's dynamical history. Our findings provide insights for understanding the interplay between intracluster dynamics and the evolution of compact binaries in GCs. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Comments: 11 pages, 9 figures, 4 tables, Accepted for MNRAS

Automorphisms and deformations of regular semisimple Hessenberg varieties

Authors: Patrick Brosnan, Laura Escobar, Jaehyun Hong, Donggun Lee, Eunjeong Lee, Anton Mellit, Eric Sommers

Abstract: We show that regular semisimple Hessenberg varieties can have moduli. To be precise, suppose $X$ is a regular semisimple Hessenberg variety of codimension 1 in the flag variety $G/B$, where $G$ is a simple algebraic group of rank $r$ over $\mathbb{C}$ and $B$ is a Borel subgroup. We show that the space $\mathrm{H}^1(X,TX)$ of first order deformations of $X$ has dimension $r-1$ except in type… ▽ More We show that regular semisimple Hessenberg varieties can have moduli. To be precise, suppose $X$ is a regular semisimple Hessenberg variety of codimension 1 in the flag variety $G/B$, where $G$ is a simple algebraic group of rank $r$ over $\mathbb{C}$ and $B$ is a Borel subgroup. We show that the space $\mathrm{H}^1(X,TX)$ of first order deformations of $X$ has dimension $r-1$ except in type $A_2$. (In type $A_2$, the Hessenberg varieties in question are all isomorphic to the permutohedral toric surface, and $\mathrm{dim}\mathrm{H}^1(X,TX)=0$.) Moreover, we show that the Kodaira-Spencer map $\mathfrak{g}\to\mathrm{H}^1(X,TX)$ is onto, that the connected component of the automorphism group of $X$ is the maximal torus of $G$, and that $\mathrm{H}^i(X,TX)=0$ for $i\geq2$. Along the way, we prove several theorems of independent interest about the cohomology of homogeneous vector bundles on $G/B$. In type $A$, we can give an even more precise statement determining when two codimension $1$ regular semisimple Hessenberg varieties in $G/B$ are isomorphic. We also compute the automorphism groups explicitly in type $A_{n-1}$ in the terms of stabilizer subgroups of the action of the symmetric group $S_n$ on the moduli space $M_{0,n+1}$ of smooth genus $0$ curves with $n+1$ marked points. Using this, we describe the moduli stack of the regular semisimple Hessenberg varieties $X$ explicitly as a quotient stack of $M_{0,n+1}$. We prove several analogous results for Hessenberg varieties in generalized flag varieties $G/P$, where $P$ is a parabolic subgroup of $G$. In type $A$, these results are used in the proofs of the results for $G/B$, but they are also independently interesting because the associated moduli stacks are related directly to the action of $S_n$ on $M_{0,n}$. △ Less

Submitted 18 June, 2024; v1 submitted 28 May, 2024; originally announced May 2024.

Comments: 48 pages, minor changes

MSC Class: 14M15; 14J50; 14J10

Novel semi-explicit symplectic schemes for nonseparable stochastic Hamiltonian systems

Authors: Jialin Hong, Baohui Hou, Liying Sun

Abstract: In this manuscript, we propose efficient stochastic semi-explicit symplectic schemes tailored for nonseparable stochastic Hamiltonian systems (SHSs). These semi-explicit symplectic schemes are constructed by introducing augmented Hamiltonians and using symmetric projection. In the case of the artificial restraint in augmented Hamiltonians being zero, the proposed schemes also preserve quadratic in… ▽ More In this manuscript, we propose efficient stochastic semi-explicit symplectic schemes tailored for nonseparable stochastic Hamiltonian systems (SHSs). These semi-explicit symplectic schemes are constructed by introducing augmented Hamiltonians and using symmetric projection. In the case of the artificial restraint in augmented Hamiltonians being zero, the proposed schemes also preserve quadratic invariants, making them suitable for developing semi-explicit charge-preserved multi-symplectic schemes for stochastic cubic Schrödinger equations with multiplicative noise. Through numerical experiments that validate theoretical results, we demonstrate that the proposed stochastic semi-explicit symplectic scheme, which features a straightforward Newton iteration solver, outperforms the traditional stochastic midpoint scheme in terms of effectiveness and accuracy. △ Less

Submitted 23 May, 2024; originally announced May 2024.

From Role-Play to Drama-Interaction: An LLM Solution

Authors: Weiqi Wu, Hongqiu Wu, Lai Jiang, Xingyuan Liu, Jiale Hong, Hai Zhao, Min Zhang

Abstract: Drama is a form of storytelling inspired by human creativity, proceeding with a predefined storyline, carrying emotions and thoughts. This paper introduces \emph{LLM-based interactive drama}, which endows traditional drama with an unprecedented immersion, where a person is allowed to walk into it and interact with the characters and scenes. We define this new artistic genre by 6 essential elements… ▽ More Drama is a form of storytelling inspired by human creativity, proceeding with a predefined storyline, carrying emotions and thoughts. This paper introduces \emph{LLM-based interactive drama}, which endows traditional drama with an unprecedented immersion, where a person is allowed to walk into it and interact with the characters and scenes. We define this new artistic genre by 6 essential elements-plot, character, thought, diction, spectacle and interaction-and study the entire pipeline to forge a backbone \emph{drama LLM} to drive the playing process, which is challenged by limited drama resources, uncontrollable narrative development, and complicated instruction following. We propose \emph{Narrative Chain} to offer finer control over the narrative progression during interaction with players; \emph{Auto-Drama} to synthesize drama scripts given arbitrary stories; \emph{Sparse Instruction Tuning} to allow the model to follow sophisticated instructions. We manually craft 3 scripts, \emph{Detective Conan}, \emph{Harry Potter}, \emph{Romeo and Juliet}, and design a 5-dimension principle to evaluate the drama LLM comprehensively. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: Accepted by ACL 2024 Findings

Investigation of suppression of $Υ(nS)$ in relativistic heavy-ion collisions at RHIC and LHC energies

Authors: Junlee Kim, Jaebeom Park, Byungsik Hong, Juhee Hong, Eun-Joo Kim, Yongsun Kim, MinJung Kweon, Su Houng Lee, Sanghoon Lim, Jinjoo Seo

Abstract: The primary purpose of studying quarkonium production in relativistic heavy-ion collisions is to understand the properties of the quark-gluon plasma. At various collision systems, measurements of quarkonium states of different binding energies, such as $Υ(nS)$, can provide comprehensive information. A model study has been performed to investigate the modification of $Υ(nS)$ production in Pb-Pb col… ▽ More The primary purpose of studying quarkonium production in relativistic heavy-ion collisions is to understand the properties of the quark-gluon plasma. At various collision systems, measurements of quarkonium states of different binding energies, such as $Υ(nS)$, can provide comprehensive information. A model study has been performed to investigate the modification of $Υ(nS)$ production in Pb-Pb collisions at $\sqrt{s_{\mathrm{NN}}}=$ 5.02 TeV and Au-Au collisions at $\sqrt{s_{\mathrm{NN}}}=$ 200 GeV. The Monte-Carlo simulation study is performed with a publicly available hydrodynamic simulation package for the quark-gluon plasma medium and a theoretical calculation of temperature-dependent thermal width of $Υ(nS)$ considering the gluo-dissociation and inelastic parton scattering for dissociation inside the medium. In addition, we perform a systematic study with different descriptions of initial collision geometry and formation time of $Υ(nS)$ to investigate their impacts on yield modification. The model calculation with a varied parameter set can describe the experimental data of $Υ(nS)$ in Pb-Pb collisions at 5.02 TeV and $Υ(2S)$ in Au-Au collisions at 200 GeV but underestimates the modification of $Υ(1S)$ at the lower collision energy. The nuclear absorption mechanism is explored to understand the discrepancy between the data and simulation. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Comments: 9 pages, 11 figures

Power Variable Projection for Initialization-Free Large-Scale Bundle Adjustment

Authors: Simon Weber, Je Hyeong Hong, Daniel Cremers

Abstract: Initialization-free bundle adjustment (BA) remains largely uncharted. While Levenberg-Marquardt algorithm is the golden method to solve the BA problem, it generally relies on a good initialization. In contrast, the under-explored Variable Projection algorithm (VarPro) exhibits a wide convergence basin even without initialization. Coupled with object space error formulation, recent works have shown… ▽ More Initialization-free bundle adjustment (BA) remains largely uncharted. While Levenberg-Marquardt algorithm is the golden method to solve the BA problem, it generally relies on a good initialization. In contrast, the under-explored Variable Projection algorithm (VarPro) exhibits a wide convergence basin even without initialization. Coupled with object space error formulation, recent works have shown its ability to solve (small-scale) initialization-free bundle adjustment problem. We introduce Power Variable Projection (PoVar), extending a recent inverse expansion method based on power series. Importantly, we link the power series expansion to Riemannian manifold optimization. This projective framework is crucial to solve large-scale bundle adjustment problem without initialization. Using the real-world BAL dataset, we experimentally demonstrate that our solver achieves state-of-the-art results in terms of speed and accuracy. In particular, our work is the first, to our knowledge, that addresses the scalability of BA without initialization and opens new venues for initialization-free Structure-from-Motion. △ Less

Submitted 9 May, 2024; v1 submitted 8 May, 2024; originally announced May 2024.

Many-body Localization Transition of Ising Spin-1 Chains

Authors: Taotao Hu, Yining Zhang, Hang Ren, Yiwen Gao, Xiaodan Li, Jiameng Hong, Yuting Li

Abstract: In this paper, we theoretically investigate the many-body localization properties of one-dimensional Ising spin-1 chains by using the methods of exact matrix diagonalization. We compare it with the MBL properties of the Ising spin-1/2 chains. The results indicate that the one-dimensional Ising spin-1 chains can also undergo MBL phase transition. There are various forms of disorder, and we compare… ▽ More In this paper, we theoretically investigate the many-body localization properties of one-dimensional Ising spin-1 chains by using the methods of exact matrix diagonalization. We compare it with the MBL properties of the Ising spin-1/2 chains. The results indicate that the one-dimensional Ising spin-1 chains can also undergo MBL phase transition. There are various forms of disorder, and we compare the effects of different forms of quasi-disorder and random disorder on many-body localization in this paper. First, we calculate the exctied-state fidelity to study the MBL phase transtion. By changing the form of the quasi-disorder, we study the MBL transition of the system with different forms of quasi-disorder and compare them with those of the random disordered system. The results show that both random disorder and quasi-disorder can cause the MBL phase transition in the one-dimensional Ising spin-1 chains. In order to study the effect of spin interactions, we compare Ising spin-1 chains and spin-1/2 chains with the next-nearest-neighbour(N-N) two-body interactions and the next-next-nearest-neighbour (N-N-N)interactions. The results show that the critical point increases with the addition of the interaction. Then we study the dynamical properties of the model by the dynamical behavior of diagonal entropy (DE), local magnetization and the time evolution of fidelity to further prove the occurrence of MBL phase transition in the disordered Ising spin-1 chains with the (N-N) coupling term and distinguish the ergodic phase (thermal phase) and the many-body localized phase. Lastly, we delve into the impact of periodic driving on one-dimensional Ising spin-1 chains. And we compare it with the results obtained from the Ising spin-1/2 chains. It shows that periodic driving can cause Ising spin-1 chains and Ising spin-1/2 chains to occur the MBL transition. △ Less

Submitted 3 May, 2024; originally announced May 2024.

Subleading analysis for $S^3$ partition functions of $\mathcal{N}=2$ holographic SCFTs

Authors: Seppe Geukens, Junho Hong

Abstract: We investigate the 3-sphere partition functions of various 3d $\mathcal{N}=2$ holographic SCFTs arising from the $N$ stack of M2-branes in the 't Hooft limit both analytically and numerically. We first employ a saddle point approximation to evaluate the free energy $F=-\log Z$ at the planar level, tracking the first subleading corrections in the large 't Hooft coupling $λ$ expansion. Subsequently,… ▽ More We investigate the 3-sphere partition functions of various 3d $\mathcal{N}=2$ holographic SCFTs arising from the $N$ stack of M2-branes in the 't Hooft limit both analytically and numerically. We first employ a saddle point approximation to evaluate the free energy $F=-\log Z$ at the planar level, tracking the first subleading corrections in the large 't Hooft coupling $λ$ expansion. Subsequently, we improve these results by determining the planar free energy to all orders in the large $λ$ expansion via numerical analysis. Remarkably, the resulting planar free energies turn out to take a universal form, supporting a prediction that these $S^3$ partition functions are all given in terms of an Airy function even beyond the special cases where the Airy formulae were derived analytically in the literature; in this context we also present new Airy conjectures in several examples. The subleading behaviors we captured encode a part of quantum corrections to the M-theory path integrals around dual asymptotically Euclidean AdS$_4$ backgrounds with the corresponding internal manifolds through holographic duality. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: v1: 46 pages + appendices

Low-overhead General-purpose Near-Data Processing in CXL Memory Expanders

Authors: Hyungkyu Ham, Jeongmin Hong, Geonwoo Park, Yunseon Shin, Okkyun Woo, Wonhyuk Yang, Jinhoon Bae, Eunhyeok Park, Hyojin Sung, Euicheol Lim, Gwangsun Kim

Abstract: To overcome the memory capacity wall of large-scale AI and big data applications, Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL.mem protocol stack minimizes interconnect latency, CXL memory accesses can still result in significant slowdowns for memory-bound applications. While near-data processing (NDP) in CXL memory can overc… ▽ More To overcome the memory capacity wall of large-scale AI and big data applications, Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL.mem protocol stack minimizes interconnect latency, CXL memory accesses can still result in significant slowdowns for memory-bound applications. While near-data processing (NDP) in CXL memory can overcome such limitations, prior works propose application-specific HW units that are not suitable for practical CXL memory-based systems that should support various applications. On the other hand, existing CPU or GPU cores are not cost-effective for NDP because they are not optimized for memory-bound applications. In addition, the communication between the host processor and CXL controller for NDP offloading should achieve low latency, but the CXL$.$io (or PCIe) protocol incurs $μ$s-scale latency and is not suitable for fine-grain NDP. To achieve high-performance NDP end-to-end, we propose a low-overhead general-purpose NDP architecture for CXL memory referred to as Memory-Mapped NDP (M$^2$NDP), which comprises memory-mapped functions (M$^2$func) and memory-mapped $μ$threading (M$^2μ$thr). The M$^2$func is a CXL.mem-compatible low-overhead communication mechanism between the host processor and NDP controller in the CXL memory. The M$^2μ$thr enables low-cost, general-purpose NDP unit design by introducing lightweight $μ$threads that support highly concurrent execution of NDP kernels with minimal resource wastage. By combining them, our M$^2$NDP achieves significant speedups for various applications, including in-memory OLAP, key-value store, large language model, recommendation model, and graph analytics by up to 128$\times$ (11.5$\times$ overall) and reduces energy by up to 87.9\% (80.1\% overall) compared to a baseline CPU or GPU host with passive CXL memory. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Meta-Object: Interactive and Multisensory Virtual Object Learned from the Real World for the Post-Metaverse

Authors: Dooyoung Kim, Taewook Ha, Jinseok Hong, Seonji Kim, Selin Choi, Heejeong Ko, Woontack Woo

Abstract: With the proliferation of wearable Augmented Reality/Virtual Reality (AR/VR) devices, ubiquitous virtual experiences seamlessly integrate into daily life through metaverse platforms. To support immersive metaverse experiences akin to reality, we propose a next-generation virtual object, a meta-object, a property-embedded virtual object that contains interactive and multisensory characteristics lea… ▽ More With the proliferation of wearable Augmented Reality/Virtual Reality (AR/VR) devices, ubiquitous virtual experiences seamlessly integrate into daily life through metaverse platforms. To support immersive metaverse experiences akin to reality, we propose a next-generation virtual object, a meta-object, a property-embedded virtual object that contains interactive and multisensory characteristics learned from the real world. Current virtual objects differ significantly from real-world objects due to restricted sensory feedback based on limited physical properties. To leverage meta-objects in the metaverse, three key components are needed: meta-object modeling and property embedding, interaction-adaptive multisensory feedback, and an intelligence simulation-based post-metaverse platform. Utilizing meta-objects that enable both on-site and remote users to interact as if they were engaging with real objects could contribute to the advent of the post-metaverse era through wearable AR/VR devices. △ Less

Submitted 28 April, 2024; v1 submitted 26 April, 2024; originally announced April 2024.

Comments: 12 pages, 4 figures, under review in the IEEE CG&A magazine

Superiority of stochastic symplectic methods via the law of iterated logarithm

Authors: Chuchu Chen, Xinyu Chen, Tonghe Dang, Jialin Hong

Abstract: The superiority of stochastic symplectic methods over non-symplectic counterparts has been verified by plenty of numerical experiments, especially in capturing the asymptotic behaviour of the underlying solution process. How can one theoretically explain this superiority? This paper gives an answer to this problem from the perspective of the law of iterated logarithm, taking the linear stochastic… ▽ More The superiority of stochastic symplectic methods over non-symplectic counterparts has been verified by plenty of numerical experiments, especially in capturing the asymptotic behaviour of the underlying solution process. How can one theoretically explain this superiority? This paper gives an answer to this problem from the perspective of the law of iterated logarithm, taking the linear stochastic Hamiltonian system in Hilbert space as a test model. The main contribution is twofold. First, by fully utilizing the time-change theorem for martingales and the Borell--TIS inequality, we prove that the upper limit of the exact solution with a specific scaling function almost surely equals some non-zero constant, thus confirming the validity of the law of iterated logarithm. Second, we prove that stochastic symplectic fully discrete methods asymptotically preserve the law of iterated logarithm, but non-symplectic ones do not. This reveals the good ability of stochastic symplectic methods in characterizing the almost sure asymptotic growth of the utmost fluctuation of the underlying solution process. Applications of our results to the linear stochastic oscillator and the linear stochastic Schrodinger equation are also presented. △ Less

Submitted 23 April, 2024; originally announced April 2024.

End-to-End Verifiable Decentralized Federated Learning

Authors: Chaehyeon Lee, Jonathan Heiss, Stefan Tai, James Won-Ki Hong

Abstract: Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticit… ▽ More Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticity of data and computation extending verifiability to the data source. Addressing an inherent conflict of confidentiality and transparency, we introduce a two-step proving and verification (2PV) method that we apply to central system procedures: a registration workflow that enables non-disclosing verification of device certificates and a learning workflow that extends existing blockchain and ZKP-based FL systems through non-disclosing data authenticity proofs. Our evaluation on a prototypical implementation demonstrates the technical feasibility with only marginal overheads to state-of-the-art solutions. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Comments: 9 pages, 5 figures, This article has been accepted for presentation at the IEEE International Conference on Blockchain and Cryptocurrency (ICBC 2024)

Multiphoton super-resolution imaging via virtual structured illumination

Authors: Sumin Lim, Sungsam Kang, Jin-Hee Hong, Youngho Jin, Kalpak Gupta, Moonseok Kim, Suhyun Kim, Wonshik Choi, Seokchan Yoon

Abstract: Fluorescence imaging in thick biological tissues is challenging due to sample-induced aberration and scattering, which leads to severe degradation of image quality and resolution. Fluorescence imaging in reflection geometry further exacerbates this issue since the point spread function is distorted in both excitation and emission pathways. Here, we propose a novel approach termed adaptive optics v… ▽ More Fluorescence imaging in thick biological tissues is challenging due to sample-induced aberration and scattering, which leads to severe degradation of image quality and resolution. Fluorescence imaging in reflection geometry further exacerbates this issue since the point spread function is distorted in both excitation and emission pathways. Here, we propose a novel approach termed adaptive optics virtual structured illumination microscopy (AO V-SIM) that enables super-resolution multiphoton imaging through a scattering medium in reflection geometry. Our approach exploits the incoherent reflection matrix obtained using a conventional point-scanning fluorescence microscope with an array detector. We introduce V-SIM super-resolution reconstruction algorithm based on the incoherent reflection matrix. Furthermore, we introduce a software adaptive optics correction algorithm, AO V-SIM, which recovers unattenuated and phase-corrected optical transfer function for both excitation and emission pathways. The effectiveness of our proposed method is experimentally validated through sub-diffraction-limited two-photon fluorescence imaging of various samples in the presence of strong aberration. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Longtime behaviors of $θ$-Euler-Maruyama method for stochastic functional differential equations

Authors: Chuchu Chen, Tonghe Dang, Jialin Hong, Guoting Song

Abstract: This paper investigates longtime behaviors of the $θ$-Euler-Maruyama method for the stochastic functional differential equation with superlinearly growing coefficients. We focus on the longtime convergence analysis in mean-square sense and weak sense of the $θ$-Euler-Maruyama method, the convergence of the numerical invariant measure, the existence and convergence of the numerical density function… ▽ More This paper investigates longtime behaviors of the $θ$-Euler-Maruyama method for the stochastic functional differential equation with superlinearly growing coefficients. We focus on the longtime convergence analysis in mean-square sense and weak sense of the $θ$-Euler-Maruyama method, the convergence of the numerical invariant measure, the existence and convergence of the numerical density function, and the Freidlin-Wentzell large deviation principle of the method. The main contributions are outlined as follows. First, we obtain the longtime mean-square convergence of the $θ$-Euler-Maruyama method and show that the mean-square convergence rate is $\frac12$. A key step in the proof is to establish the time-independent boundedness of high-order moments of the numerical functional solution. Second, based on the technique of the Malliavin calculus, we present the longtime weak convergence of the $θ$-Euler-Maruyama method, which implies that the invariant measure of the $θ$-Euler-Maruyama functional solution converges to the exact one with rate $1.$ Third, by the analysis of the test-functional-independent weak convergence and negative moment estimates of the determinant of the corresponding Malliavin covariance matrix, we derive the existence, convergence, and the logarithmic estimate of the density function of the $θ$-Euler-Maruyama solution. At last, utilizing the weak convergence method, we obtain the Freidlin-Wentzell large deviation principle for the $θ$-Euler-Maruyama solution on the infinite time horizon. △ Less

Submitted 12 April, 2024; originally announced April 2024.

PID-Comm: A Fast and Flexible Collective Communication Framework for Commodity Processing-in-DIMM Devices

Authors: Si Ung Noh, Junguk Hong, Chaemin Lim, Seongyeon Park, Jeehyun Kim, Hanjun Kim, Youngsok Kim, Jinho Lee

Abstract: Recent dual in-line memory modules (DIMMs) are starting to support processing-in-memory (PIM) by associating their memory banks with processing elements (PEs), allowing applications to overcome the data movement bottleneck by offloading memory-intensive operations to the PEs. Many highly parallel applications have been shown to benefit from these PIM-enabled DIMMs, but further speedup is often lim… ▽ More Recent dual in-line memory modules (DIMMs) are starting to support processing-in-memory (PIM) by associating their memory banks with processing elements (PEs), allowing applications to overcome the data movement bottleneck by offloading memory-intensive operations to the PEs. Many highly parallel applications have been shown to benefit from these PIM-enabled DIMMs, but further speedup is often limited by the huge overhead of inter-PE communication. This mainly comes from the slow CPU-mediated inter-PE communication methods which incurs significant performance overheads, making it difficult for PIM-enabled DIMMs to accelerate a wider range of applications. Prior studies have tried to alleviate the communication bottleneck, but they lack enough flexibility and performance to be used for a wide range of applications. In this paper, we present PID-Comm, a fast and flexible collective inter-PE communication framework for commodity PIM-enabled DIMMs. The key idea of PID-Comm is to abstract the PEs as a multi-dimensional hypercube and allow multiple instances of collective inter-PE communication between the PEs belonging to certain dimensions of the hypercube. Leveraging this abstraction, PID-Comm first defines eight collective inter-PE communication patterns that allow applications to easily express their complex communication patterns. Then, PID-Comm provides high-performance implementations of the collective inter-PE communication patterns optimized for the DIMMs. Our evaluation using 16 UPMEM DIMMs and representative parallel algorithms shows that PID-Comm greatly improves the performance by up to 4.20x compared to the existing inter-PE communication implementations. The implementation of PID-Comm is available at https://github.com/AIS-SNU/PID-Comm. △ Less

Submitted 12 April, 2024; originally announced April 2024.

Comments: Accepted to ISCA 2024

Latent-based Diffusion Model for Long-tailed Recognition

Authors: Pengxiao Han, Changkun Ye, Jieming Zhou, Jing Zhang, Jie Hong, Xuesong Li

Abstract: Long-tailed imbalance distribution is a common issue in practical computer vision applications. Previous works proposed methods to address this problem, which can be categorized into several classes: re-sampling, re-weighting, transfer learning, and feature augmentation. In recent years, diffusion models have shown an impressive generation ability in many sub-problems of deep computer vision. Howe… ▽ More Long-tailed imbalance distribution is a common issue in practical computer vision applications. Previous works proposed methods to address this problem, which can be categorized into several classes: re-sampling, re-weighting, transfer learning, and feature augmentation. In recent years, diffusion models have shown an impressive generation ability in many sub-problems of deep computer vision. However, its powerful generation has not been explored in long-tailed problems. We propose a new approach, the Latent-based Diffusion Model for Long-tailed Recognition (LDMLR), as a feature augmentation method to tackle the issue. First, we encode the imbalanced dataset into features using the baseline model. Then, we train a Denoising Diffusion Implicit Model (DDIM) using these encoded features to generate pseudo-features. Finally, we train the classifier using the encoded and pseudo-features from the previous two steps. The model's accuracy shows an improvement on the CIFAR-LT and ImageNet-LT datasets by using the proposed method. △ Less

Submitted 23 April, 2024; v1 submitted 6 April, 2024; originally announced April 2024.

Comments: 8 pages, 3 figures. Accepted by L3DIVU-CVPR2024

TE-TAD: Towards Full End-to-End Temporal Action Detection via Time-Aligned Coordinate Expression

Authors: Ho-Joong Kim, Jung-Ho Hong, Heejo Kong, Seong-Whan Lee

Abstract: In this paper, we investigate that the normalized coordinate expression is a key factor as reliance on hand-crafted components in query-based detectors for temporal action detection (TAD). Despite significant advancements towards an end-to-end framework in object detection, query-based detectors have been limited in achieving full end-to-end modeling in TAD. To address this issue, we propose \mode… ▽ More In this paper, we investigate that the normalized coordinate expression is a key factor as reliance on hand-crafted components in query-based detectors for temporal action detection (TAD). Despite significant advancements towards an end-to-end framework in object detection, query-based detectors have been limited in achieving full end-to-end modeling in TAD. To address this issue, we propose \modelname{}, a full end-to-end temporal action detection transformer that integrates time-aligned coordinate expression. We reformulate coordinate expression utilizing actual timeline values, ensuring length-invariant representations from the extremely diverse video duration environment. Furthermore, our proposed adaptive query selection dynamically adjusts the number of queries based on video length, providing a suitable solution for varying video durations compared to a fixed query set. Our approach not only simplifies the TAD process by eliminating the need for hand-crafted components but also significantly improves the performance of query-based detectors. Our TE-TAD outperforms the previous query-based detectors and achieves competitive performance compared to state-of-the-art methods on popular benchmark datasets. Code is available at: https://github.com/Dotori-HJ/TE-TAD △ Less

Submitted 3 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Backpropagation-free Network for 3D Test-time Adaptation

Authors: Yanshuo Wang, Ali Cheraghian, Zeeshan Hayder, Jie Hong, Sameera Ramasinghe, Shafin Rahman, David Ahmedt-Aristizabal, Xuesong Li, Lars Petersson, Mehrtash Harandi

Abstract: Real-world systems often encounter new data over time, which leads to experiencing target domain shifts. Existing Test-Time Adaptation (TTA) methods tend to apply computationally heavy and memory-intensive backpropagation-based approaches to handle this. Here, we propose a novel method that uses a backpropagation-free approach for TTA for the specific case of 3D data. Our model uses a two-stream a… ▽ More Real-world systems often encounter new data over time, which leads to experiencing target domain shifts. Existing Test-Time Adaptation (TTA) methods tend to apply computationally heavy and memory-intensive backpropagation-based approaches to handle this. Here, we propose a novel method that uses a backpropagation-free approach for TTA for the specific case of 3D data. Our model uses a two-stream architecture to maintain knowledge about the source domain as well as complementary target-domain-specific information. The backpropagation-free property of our model helps address the well-known forgetting problem and mitigates the error accumulation issue. The proposed method also eliminates the need for the usually noisy process of pseudo-labeling and reliance on costly self-supervised training. Moreover, our method leverages subspace learning, effectively reducing the distribution variance between the two domains. Furthermore, the source-domain-specific and the target-domain-specific streams are aligned using a novel entropy-based adaptive fusion strategy. Extensive experiments on popular benchmarks demonstrate the effectiveness of our method. The code will be available at \url{https://github.com/abie-e/BFTT3D}. △ Less

Submitted 24 April, 2024; v1 submitted 27 March, 2024; originally announced March 2024.

Comments: CVPR 2024

A Recommender System for NFT Collectibles with Item Feature

Authors: Minjoo Choi, Seonmi Kim, Yejin Kim, Youngbin Lee, Joohwan Hong, Yongjae Lee

Abstract: Recommender systems have been actively studied and applied in various domains to deal with information overload. Although there are numerous studies on recommender systems for movies, music, and e-commerce, comparatively less attention has been paid to the recommender system for NFTs despite the continuous growth of the NFT market. This paper presents a recommender system for NFTs that utilizes a… ▽ More Recommender systems have been actively studied and applied in various domains to deal with information overload. Although there are numerous studies on recommender systems for movies, music, and e-commerce, comparatively less attention has been paid to the recommender system for NFTs despite the continuous growth of the NFT market. This paper presents a recommender system for NFTs that utilizes a variety of data sources, from NFT transaction records to external item features, to generate precise recommendations that cater to individual preferences. We develop a data-efficient graph-based recommender system to efficiently capture the complex relationship between each item and users and generate node(item) embeddings which incorporate both node feature information and graph structure. Furthermore, we exploit inputs beyond user-item interactions, such as image feature, text feature, and price feature. Numerical experiments verify the performance of the graph-based recommender system improves significantly after utilizing all types of item features as side information, thereby outperforming all other baselines. △ Less

Submitted 3 April, 2024; v1 submitted 27 March, 2024; originally announced March 2024.

Comments: Presented at the AAAI 2023 Bridge on AI for Financial Services (https://sites.google.com/view/aaai-ai-fin/home)

Sub-Dirac operators, spectral Einstein functionals and the noncommutative residue

Authors: Jin Hong, Yuchen Yang, Yong Wang

Abstract: In this paper, we define the spectral Einstein functional associated with the sub-Dirac operator for manifolds with boundary. A proof of the Dabrowski-Sitarz-Zalecki type theorem for spectral Einstein functions associated with the sub-Dirac operator on four-dimensional manifolds with boundary is also given. In this paper, we define the spectral Einstein functional associated with the sub-Dirac operator for manifolds with boundary. A proof of the Dabrowski-Sitarz-Zalecki type theorem for spectral Einstein functions associated with the sub-Dirac operator on four-dimensional manifolds with boundary is also given. △ Less

Submitted 30 March, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: arXiv admin note: text overlap with arXiv:2308.00833, arXiv:2312.00838

Expectations Versus Reality: Evaluating Intrusion Detection Systems in Practice

Authors: Jake Hesford, Daniel Cheng, Alan Wan, Larry Huynh, Seungho Kim, Hyoungshick Kim, Jin B. Hong

Abstract: Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere I… ▽ More Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere IoT datasets both capture IoT-related attacks, but the deep neural network performed the best when tested using the BoT_IoT dataset while HELAD performed the best when tested using the Stratosphere IoT dataset. So although we found that a deep neural network solution had the highest average F1 scores on tested datasets, it is not always the best-performing one. We further discuss difficulties in using IDS from literature and project repositories, which complicated drawing definitive conclusions regarding IDS selection. △ Less

Submitted 28 March, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: 10 pages

MSC Class: 68M25; 68M20 ACM Class: C.4; D.m

InternLM2 Technical Report

Authors: Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang , et al. (75 additional authors not shown)

Abstract: The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advancements in open-source models has been challenging. This paper introduces InternLM2, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context m… ▽ More The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advancements in open-source models has been challenging. This paper introduces InternLM2, an open-source LLM that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks, long-context modeling, and open-ended subjective evaluations through innovative pre-training and optimization techniques. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types including text, code, and long-context data. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages, exhibiting remarkable performance on the 200k ``Needle-in-a-Haystack" test. InternLM2 is further aligned using Supervised Fine-Tuning (SFT) and a novel Conditional Online Reinforcement Learning from Human Feedback (COOL RLHF) strategy that addresses conflicting human preferences and reward hacking. By releasing InternLM2 models in different training stages and model sizes, we provide the community with insights into the model's evolution. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Korean Bio-Medical Corpus (KBMC) for Medical Named Entity Recognition

Authors: Sungjoo Byun, Jiseung Hong, Sumin Park, Dongjun Jang, Jean Seo, Minseok Kim, Chaeyoung Oh, Hyopil Shin

Abstract: Named Entity Recognition (NER) plays a pivotal role in medical Natural Language Processing (NLP). Yet, there has not been an open-source medical NER dataset specifically for the Korean language. To address this, we utilized ChatGPT to assist in constructing the KBMC (Korean Bio-Medical Corpus), which we are now presenting to the public. With the KBMC dataset, we noticed an impressive 20% increase… ▽ More Named Entity Recognition (NER) plays a pivotal role in medical Natural Language Processing (NLP). Yet, there has not been an open-source medical NER dataset specifically for the Korean language. To address this, we utilized ChatGPT to assist in constructing the KBMC (Korean Bio-Medical Corpus), which we are now presenting to the public. With the KBMC dataset, we noticed an impressive 20% increase in medical NER performance compared to models trained on general Korean NER datasets. This research underscores the significant benefits and importance of using specialized tools and datasets, like ChatGPT, to enhance language processing in specialized fields such as healthcare. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Journal ref: LREC-COLING 2024

Review of Large-Scale Simulation Optimization

Authors: Weiwei Fan, L. Jeff Hong, Guangxin Jiang, Jun Luo

Abstract: Large-scale simulation optimization (SO) problems encompass both large-scale ranking-and-selection problems and high-dimensional discrete or continuous SO problems, presenting significant challenges to existing SO theories and algorithms. This paper begins by providing illustrative examples that highlight the differences between large-scale SO problems and those of a more moderate scale. Subsequen… ▽ More Large-scale simulation optimization (SO) problems encompass both large-scale ranking-and-selection problems and high-dimensional discrete or continuous SO problems, presenting significant challenges to existing SO theories and algorithms. This paper begins by providing illustrative examples that highlight the differences between large-scale SO problems and those of a more moderate scale. Subsequently, it reviews several widely employed techniques for addressing large-scale SO problems, such as divide and conquer, dimension reduction, and gradient-based algorithms. Additionally, the paper examines parallelization techniques leveraging widely accessible parallel computing environments to facilitate the resolution of large-scale SO problems. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression

Authors: Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian Bartoldson, Ajay Jaiswal, Kaidi Xu, Bhavya Kailkhura, Dan Hendrycks, Dawn Song, Zhangyang Wang, Bo Li

Abstract: Compressing high-capability Large Language Models (LLMs) has emerged as a favored strategy for resource-efficient inferences. While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the potential risks of compression in terms of safety and trustworthiness have been largely neglected. This study conducts the first, thorough evaluation o… ▽ More Compressing high-capability Large Language Models (LLMs) has emerged as a favored strategy for resource-efficient inferences. While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the potential risks of compression in terms of safety and trustworthiness have been largely neglected. This study conducts the first, thorough evaluation of three (3) leading LLMs using five (5) SoTA compression techniques across eight (8) trustworthiness dimensions. Our experiments highlight the intricate interplay between compression and trustworthiness, revealing some interesting patterns. We find that quantization is currently a more effective approach than pruning in achieving efficiency and trustworthiness simultaneously. For instance, a 4-bit quantized model retains the trustworthiness of its original counterpart, but model pruning significantly degrades trustworthiness, even at 50% sparsity. Moreover, employing quantization within a moderate bit range could unexpectedly improve certain trustworthiness dimensions such as ethics and fairness. Conversely, extreme quantization to very low bit levels (3 bits) tends to reduce trustworthiness significantly. This increased risk cannot be uncovered by looking at benign performance alone, in turn, mandating comprehensive trustworthiness evaluation in practice. These findings culminate in practical recommendations for simultaneously achieving high utility, efficiency, and trustworthiness in LLMs. Code and models are available at https://decoding-comp-trust.github.io. △ Less

Submitted 4 June, 2024; v1 submitted 17 March, 2024; originally announced March 2024.

Comments: Accepted to ICML'24

Extended Reality for Enhanced Human-Robot Collaboration: a Human-in-the-Loop Approach

Authors: Yehor Karpichev, Todd Charter, Jayden Hong, Amir M. Soufi Enayati, Homayoun Honari, Mehran Ghafarian Tamizi, Homayoun Najjaran

Abstract: The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understa… ▽ More The rise of automation has provided an opportunity to achieve higher efficiency in manufacturing processes, yet it often compromises the flexibility required to promptly respond to evolving market needs and meet the demand for customization. Human-robot collaboration attempts to tackle these challenges by combining the strength and precision of machines with human ingenuity and perceptual understanding. In this paper, we conceptualize and propose an implementation framework for an autonomous, machine learning-based manipulator that incorporates human-in-the-loop principles and leverages Extended Reality (XR) to facilitate intuitive communication and programming between humans and robots. Furthermore, the conceptual framework foresees human involvement directly in the robot learning process, resulting in higher adaptability and task generalization. The paper highlights key technologies enabling the proposed framework, emphasizing the importance of developing the digital ecosystem as a whole. Additionally, we review the existent implementation approaches of XR in human-robot collaboration, showcasing diverse perspectives and methodologies. The challenges and future outlooks are discussed, delving into the major obstacles and potential research avenues of XR for more natural human-robot interaction and integration in the industrial landscape. △ Less

Submitted 14 June, 2024; v1 submitted 21 March, 2024; originally announced March 2024.

Comments: Accepted to IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) 2024

Learning Decomposable and Debiased Representations via Attribute-Centric Information Bottlenecks

Authors: Jinyung Hong, Eun Som Jeon, Changhoon Kim, Keun Hee Park, Utkarsh Nath, Yezhou Yang, Pavan Turaga, Theodore P. Pavlic

Abstract: Biased attributes, spuriously correlated with target labels in a dataset, can problematically lead to neural networks that learn improper shortcuts for classifications and limit their capabilities for out-of-distribution (OOD) generalization. Although many debiasing approaches have been proposed to ensure correct predictions from biased datasets, few studies have considered learning latent embeddi… ▽ More Biased attributes, spuriously correlated with target labels in a dataset, can problematically lead to neural networks that learn improper shortcuts for classifications and limit their capabilities for out-of-distribution (OOD) generalization. Although many debiasing approaches have been proposed to ensure correct predictions from biased datasets, few studies have considered learning latent embedding consisting of intrinsic and biased attributes that contribute to improved performance and explain how the model pays attention to attributes. In this paper, we propose a novel debiasing framework, Debiasing Global Workspace, introducing attention-based information bottlenecks for learning compositional representations of attributes without defining specific bias types. Based on our observation that learning shape-centric representation helps robust performance on OOD datasets, we adopt those abilities to learn robust and generalizable representations of decomposable latent embeddings corresponding to intrinsic and biasing attributes. We conduct comprehensive evaluations on biased datasets, along with both quantitative and qualitative analyses, to showcase our approach's efficacy in attribute-centric representation learning and its ability to differentiate between intrinsic and bias-related features. △ Less

Submitted 21 March, 2024; originally announced March 2024.

Comments: 24 pages, 16 figures, 3 tables

Opti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments

Authors: Kurran Singh, Jungseok Hong, Nicholas R. Rypkema, John J. Leonard

Abstract: Despite recent advances in semantic Simultaneous Localization and Mapping (SLAM) for terrestrial and aerial applications, underwater semantic SLAM remains an open and largely unaddressed research problem due to the unique sensing modalities and the object classes found underwater. This paper presents an object-based semantic SLAM method for underwater environments that can identify, localize, clas… ▽ More Despite recent advances in semantic Simultaneous Localization and Mapping (SLAM) for terrestrial and aerial applications, underwater semantic SLAM remains an open and largely unaddressed research problem due to the unique sensing modalities and the object classes found underwater. This paper presents an object-based semantic SLAM method for underwater environments that can identify, localize, classify, and map a wide variety of marine objects without a priori knowledge of the object classes present in the scene. The method performs unsupervised object segmentation and object-level feature aggregation, and then uses opti-acoustic sensor fusion for object localization. Probabilistic data association is used to determine observation to landmark correspondences. Given such correspondences, the method then jointly optimizes landmark and vehicle position estimates. Indoor and outdoor underwater datasets with a wide variety of objects and challenging acoustic and lighting conditions are collected for evaluation and made publicly available. Quantitative and qualitative results show the proposed method achieves reduced trajectory error compared to baseline methods, and is able to obtain comparable map accuracy to a baseline closed-set method that requires hand-labeled data of all objects in the scene. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Automated Contrastive Learning Strategy Search for Time Series

Authors: Baoyu Jing, Yansen Wang, Guoxin Sui, Jing Hong, Jingrui He, Yuqing Yang, Dongsheng Li, Kan Ren

Abstract: In recent years, Contrastive Learning (CL) has become a predominant representation learning paradigm for time series. Most existing methods in the literature focus on manually building specific Contrastive Learning Strategies (CLS) by human heuristics for certain datasets and tasks. However, manually developing CLS usually require excessive prior knowledge about the datasets and tasks, e.g., profe… ▽ More In recent years, Contrastive Learning (CL) has become a predominant representation learning paradigm for time series. Most existing methods in the literature focus on manually building specific Contrastive Learning Strategies (CLS) by human heuristics for certain datasets and tasks. However, manually developing CLS usually require excessive prior knowledge about the datasets and tasks, e.g., professional cognition of the medical time series in healthcare, as well as huge human labor and massive experiments to determine the detailed learning configurations. In this paper, we present an Automated Machine Learning (AutoML) practice at Microsoft, which automatically learns to contrastively learn representations for various time series datasets and tasks, namely Automated Contrastive Learning (AutoCL). We first construct a principled universal search space of size over 3x1012, covering data augmentation, embedding transformation, contrastive pair construction and contrastive losses. Further, we introduce an efficient reinforcement learning algorithm, which optimizes CLS from the performance on the validation tasks, to obtain more effective CLS within the space. Experimental results on various real-world tasks and datasets demonstrate that AutoCL could automatically find the suitable CLS for a given dataset and task. From the candidate CLS found by AutoCL on several public datasets/tasks, we compose a transferable Generally Good Strategy (GGS), which has a strong performance for other datasets. We also provide empirical analysis as a guidance for future design of CLS. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Comments: Preprint. Work in progress

Parafermions with symmetry-protected non-Abelian statistics

Authors: Jian-Song Hong, Su-Qi Zhang, Xin Liu, Xiong-Jun Liu

Abstract: Non-Abelian anyons have garnered extensive attention for obeying exotic non-Abelian statistics and having potential applications to fault-tolerant quantum computing. While the prior research has predominantly focused on non-Abelian statistics without the necessity of symmetry protection, recent progresses have shown that symmetries can play essential roles and bring a notion of the symmetry-protec… ▽ More Non-Abelian anyons have garnered extensive attention for obeying exotic non-Abelian statistics and having potential applications to fault-tolerant quantum computing. While the prior research has predominantly focused on non-Abelian statistics without the necessity of symmetry protection, recent progresses have shown that symmetries can play essential roles and bring a notion of the symmetry-protected non-Abelian (SPNA) statistics. In this work, we extend the concept of SPNA statistics to strongly-correlated systems which host parafermion zero modes (PZMs). This study involves a few fundamental results proved here. First, we unveil a generic unitary symmetry mechanism that protects PZMs from local couplings. Then, with this symmetry protection, the PZMs can be categorized into two nontrivial sectors, each maintaining its own parity conservation, even though the whole system cannot be dismantled into separate subsystems due to nonlinear interactions. Finally, by leveraging the parity conservation of each sector and the general properties of the effective braiding Hamiltonian, we prove rigorously that the PZMs intrinsically obey SPNA statistics. To further confirm the results, we derive the braiding matrix at a tri-junction. We also propose a correlated quantum nanowire model that accommodates a pair of PZMs protected by mirror symmetry and satisfying the generic theory. This work shows a broad spectrum of strongly-correlated systems capable of hosting fractional SPNA quasiparticles and enriches our comprehension of fundamental quantum statistics linked to the symmetries that govern the exchange dynamics. △ Less

Submitted 21 March, 2024; v1 submitted 14 March, 2024; originally announced March 2024.

Comments: 10+pages, 4 figures. Discussions are updated, typos are corrected. References are updated according to feedbacks

Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk

Authors: Zhangheng Li, Junyuan Hong, Bo Li, Zhangyang Wang

Abstract: While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information. In this paper, we reveal a new risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. We… ▽ More While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information. In this paper, we reveal a new risk, Shake-to-Leak (S2L), that fine-tuning the pre-trained models with manipulated data can amplify the existing privacy risks. We demonstrate that S2L could occur in various standard fine-tuning strategies for diffusion models, including concept-injection methods (DreamBooth and Textual Inversion) and parameter-efficient methods (LoRA and Hypernetwork), as well as their combinations. In the worst case, S2L can amplify the state-of-the-art membership inference attack (MIA) on diffusion models by $5.4\%$ (absolute difference) AUC and can increase extracted private samples from almost $0$ samples to $15.8$ samples on average per target domain. This discovery underscores that the privacy risk with diffusion models is even more severe than previously recognized. Codes are available at https://github.com/VITA-Group/Shake-to-Leak. △ Less

Submitted 22 April, 2024; v1 submitted 14 March, 2024; originally announced March 2024.

Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory

Authors: Jeongmin Hong, Sungjun Cho, Geonwoo Park, Wonhyuk Yang, Young-Ho Gong, Gwangsun Kim

Abstract: We propose overcoming the memory capacity limitation of GPUs with high-capacity Storage-Class Memory (SCM) and DRAM cache. By significantly increasing the memory capacity with SCM, the GPU can capture a larger fraction of the memory footprint than HBM for workloads that oversubscribe memory, achieving high speedups. However, the DRAM cache needs to be carefully designed to address the latency and… ▽ More We propose overcoming the memory capacity limitation of GPUs with high-capacity Storage-Class Memory (SCM) and DRAM cache. By significantly increasing the memory capacity with SCM, the GPU can capture a larger fraction of the memory footprint than HBM for workloads that oversubscribe memory, achieving high speedups. However, the DRAM cache needs to be carefully designed to address the latency and BW limitations of the SCM while minimizing cost overhead and considering GPU's characteristics. Because the massive number of GPU threads can thrash the DRAM cache, we first propose an SCM-aware DRAM cache bypass policy for GPUs that considers the multi-dimensional characteristics of memory accesses by GPUs with SCM to bypass DRAM for data with low performance utility. In addition, to reduce DRAM cache probes and increase effective DRAM BW with minimal cost, we propose a Configurable Tag Cache (CTC) that repurposes part of the L2 cache to cache DRAM cacheline tags. The L2 capacity used for the CTC can be adjusted by users for adaptability. Furthermore, to minimize DRAM cache probe traffic from CTC misses, our Aggregated Metadata-In-Last-column (AMIL) DRAM cache organization co-locates all DRAM cacheline tags in a single column within a row. The AMIL also retains the full ECC protection, unlike prior DRAM cache's Tag-And-Data (TAD) organization. Additionally, we propose SCM throttling to curtail power and exploiting SCM's SLC/MLC modes to adapt to workload's memory footprint. While our techniques can be used for different DRAM and SCM devices, we focus on a Heterogeneous Memory Stack (HMS) organization that stacks SCM dies on top of DRAM dies for high performance. Compared to HBM, HMS improves performance by up to 12.5x (2.9x overall) and reduces energy by up to 89.3% (48.1% overall). Compared to prior works, we reduce DRAM cache probe and SCM write traffic by 91-93% and 57-75%, respectively. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Comments: Published in 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA'24)

Long-time weak convergence analysis of a semi-discrete scheme for stochastic Maxwell equations

Authors: Chuchu Chen, Jialin Hong, Ge Liang

Abstract: It is known from the monograph [1, Chapter 5] that the weak convergence analysis of numerical schemes for stochastic Maxwell equations is an unsolved problem. This paper aims to fill the gap by establishing the long-time weak convergence analysis of the semi-implicit Euler scheme for stochastic Maxwell equations. Based on analyzing the regularity of transformed Kolmogorov equation associated to st… ▽ More It is known from the monograph [1, Chapter 5] that the weak convergence analysis of numerical schemes for stochastic Maxwell equations is an unsolved problem. This paper aims to fill the gap by establishing the long-time weak convergence analysis of the semi-implicit Euler scheme for stochastic Maxwell equations. Based on analyzing the regularity of transformed Kolmogorov equation associated to stochastic Maxwell equations and constructing a proper continuous adapted auxiliary process for the semi-implicit scheme, we present the long-time weak convergence analysis for this scheme and prove that the weak convergence order is one, which is twice the strong convergence order. As applications of this result, we obtain the convergence order of the numerical invariant measure, the strong law of large numbers and central limit theorem related to the numerical solution, and the error estimate of the multi-level Monte Carlo estimator. As far as we know, this is the first result on the weak convergence order for stochastic Maxwell equations. △ Less

Submitted 14 March, 2024; originally announced March 2024.

Electrically Tunable Spin Exchange Splitting in Graphene Hybrid Heterostructure

Authors: Dongwon Shin, Hyeonbeom Kim, Sung Ju Hong, Sehwan Song, Yeongju Choi, Youngkuk Kim, Sungkyun Park, Dongseok Suh, Woo Seok Choi

Abstract: Graphene, with spin and valley degrees of freedom, fosters unexpected physical and chemical properties for the realization of next-generation quantum devices. However, the spin symmetry of graphene is rather robustly protected, hampering manipulation of the spin degrees of freedom for the application of spintronic devices such as electric gate tunable spin filters. We demonstrate that a hybrid het… ▽ More Graphene, with spin and valley degrees of freedom, fosters unexpected physical and chemical properties for the realization of next-generation quantum devices. However, the spin symmetry of graphene is rather robustly protected, hampering manipulation of the spin degrees of freedom for the application of spintronic devices such as electric gate tunable spin filters. We demonstrate that a hybrid heterostructure composed of graphene and LaCoO3 epitaxial thin film exhibits an electrically tunable spin exchange splitting. The large and adjustable spin exchange splitting of 155.9 - 306.5 meV was obtained by the characteristic shifts in both the spin symmetry broken quantum Hall states and the Shubnikov-de-Haas oscillations. Strong hybridization induced charge transfer across the hybrid heterointerface has been identified for the observed spin exchange splitting. The substantial and facile controllability of the spin exchange splitting provides an opportunity for spintronics applications with the electrically-tunable spin polarization in hybrid heterostructures. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Comments: 20 pages, 5 figures, 1 table

Journal ref: Adv. Funct. Mater. 34, 2311287 (2023)

REPAIR: Rank Correlation and Noisy Pair Half-replacing with Memory for Noisy Correspondence

Authors: Ruochen Zheng, Jiahao Hong, Changxin Gao, Nong Sang

Abstract: The presence of noise in acquired data invariably leads to performance degradation in cross-modal matching. Unfortunately, obtaining precise annotations in the multimodal field is expensive, which has prompted some methods to tackle the mismatched data pair issue in cross-modal matching contexts, termed as noisy correspondence. However, most of these existing noisy correspondence methods exhibit t… ▽ More The presence of noise in acquired data invariably leads to performance degradation in cross-modal matching. Unfortunately, obtaining precise annotations in the multimodal field is expensive, which has prompted some methods to tackle the mismatched data pair issue in cross-modal matching contexts, termed as noisy correspondence. However, most of these existing noisy correspondence methods exhibit the following limitations: a) the problem of self-reinforcing error accumulation, and b) improper handling of noisy data pair. To tackle the two problems, we propose a generalized framework termed as Rank corrElation and noisy Pair hAlf-replacing wIth memoRy (REPAIR), which benefits from maintaining a memory bank for features of matched pairs. Specifically, we calculate the distances between the features in the memory bank and those of the target pair for each respective modality, and use the rank correlation of these two sets of distances to estimate the soft correspondence label of the target pair. Estimating soft correspondence based on memory bank features rather than using a similarity network can avoid the accumulation of errors due to incorrect network identifications. For pairs that are completely mismatched, REPAIR searches the memory bank for the most matching feature to replace one feature of one modality, instead of using the original pair directly or merely discarding the mismatched pair. We conduct experiments on three cross-modal datasets, i.e., Flickr30K, MSCOCO, and CC152K, proving the effectiveness and robustness of our REPAIR on synthetic and real-world noise. △ Less

Submitted 13 March, 2024; originally announced March 2024.

ORPO: Monolithic Preference Optimization without Reference Model

Authors: Jiwoo Hong, Noah Lee, James Thorne

Abstract: While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building… ▽ More While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building on this foundation, we introduce a straightforward and innovative reference model-free monolithic odds ratio preference optimization algorithm, ORPO, eliminating the necessity for an additional preference alignment phase. We demonstrate, both empirically and theoretically, that the odds ratio is a sensible choice for contrasting favored and disfavored styles during SFT across the diverse sizes from 125M to 7B. Specifically, fine-tuning Phi-2 (2.7B), Llama-2 (7B), and Mistral (7B) with ORPO on the UltraFeedback alone surpasses the performance of state-of-the-art language models with more than 7B and 13B parameters: achieving up to 12.20% on $\text{AlpacaEval}_{2.0}$ (Figure 1), 66.19% on IFEval (instruction-level loose, Table 6), and 7.32 in MT-Bench (Figure 12). We release code and model checkpoints for Mistral-ORPO-$α$ (7B) and Mistral-ORPO-$β$ (7B). △ Less

Submitted 14 March, 2024; v1 submitted 12 March, 2024; originally announced March 2024.

Comments: Preprint

On the Generalization Ability of Unsupervised Pretraining

Authors: Yuyang Deng, Junyuan Hong, Jiayu Zhou, Mehrdad Mahdavi

Abstract: Recent advances in unsupervised learning have shown that unsupervised pre-training, followed by fine-tuning, can improve model generalization. However, a rigorous understanding of how the representation function learned on an unlabeled dataset affects the generalization of the fine-tuned model is lacking. Existing theoretical research does not adequately account for the heterogeneity of the distri… ▽ More Recent advances in unsupervised learning have shown that unsupervised pre-training, followed by fine-tuning, can improve model generalization. However, a rigorous understanding of how the representation function learned on an unlabeled dataset affects the generalization of the fine-tuned model is lacking. Existing theoretical research does not adequately account for the heterogeneity of the distribution and tasks in pre-training and fine-tuning stage. To bridge this gap, this paper introduces a novel theoretical framework that illuminates the critical factor influencing the transferability of knowledge acquired during unsupervised pre-training to the subsequent fine-tuning phase, ultimately affecting the generalization capabilities of the fine-tuned model on downstream tasks. We apply our theoretical framework to analyze generalization bound of two distinct scenarios: Context Encoder pre-training with deep neural networks and Masked Autoencoder pre-training with deep transformers, followed by fine-tuning on a binary classification task. Finally, inspired by our findings, we propose a novel regularization method during pre-training to further enhances the generalization of fine-tuned model. Overall, our results contribute to a better understanding of unsupervised pre-training and fine-tuning paradigm, and can shed light on the design of more effective pre-training algorithms. △ Less

Submitted 11 March, 2024; originally announced March 2024.

AGAThA: Fast and Efficient GPU Acceleration of Guided Sequence Alignment for Long Read Mapping

Authors: Seongyeon Park, Junguk Hong, Jaeyong Song, Hajin Kim, Youngsok Kim, Jinho Lee

Abstract: With the advance in genome sequencing technology, the lengths of deoxyribonucleic acid (DNA) sequencing results are rapidly increasing at lower prices than ever. However, the longer lengths come at the cost of a heavy computational burden on aligning them. For example, aligning sequences to a human reference genome can take tens or even hundreds of hours. The current de facto standard approach for… ▽ More With the advance in genome sequencing technology, the lengths of deoxyribonucleic acid (DNA) sequencing results are rapidly increasing at lower prices than ever. However, the longer lengths come at the cost of a heavy computational burden on aligning them. For example, aligning sequences to a human reference genome can take tens or even hundreds of hours. The current de facto standard approach for alignment is based on the guided dynamic programming method. Although this takes a long time and could potentially benefit from high-throughput graphic processing units (GPUs), the existing GPU-accelerated approaches often compromise the algorithm's structure, due to the GPU-unfriendly nature of the computational pattern. Unfortunately, such compromise in the algorithm is not tolerable in the field, because sequence alignment is a part of complicated bioinformatics analysis pipelines. In such circumstances, we propose AGAThA, an exact and efficient GPU-based acceleration of guided sequence alignment. We diagnose and address the problems of the algorithm being unfriendly to GPUs, which comprises strided/redundant memory accesses and workload imbalances that are difficult to predict. According to the experiments on modern GPUs, AGAThA achieves 18.8$\times$ speedup against the CPU-based baseline, 9.6$\times$ against the best GPU-based baseline, and 3.6$\times$ against GPU-based algorithms with different heuristics. △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: Published at PPoPP 2024