Showing 101–150 of 2,127 results for author: Kim, B

Online Continual Learning For Interactive Instruction Following Agents

Authors: Byeonghwi Kim, Minhyuk Seo, Jonghyun Choi

Abstract: In learning an embodied agent executing daily tasks via language directives, the literature largely assumes that the agent learns all training data at the beginning. We argue that such a learning scenario is less realistic since a robotic agent is supposed to learn the world continuously as it explores and perceives it. To take a step towards a more realistic embodied agent learning scenario, we p… ▽ More In learning an embodied agent executing daily tasks via language directives, the literature largely assumes that the agent learns all training data at the beginning. We argue that such a learning scenario is less realistic since a robotic agent is supposed to learn the world continuously as it explores and perceives it. To take a step towards a more realistic embodied agent learning scenario, we propose two continual learning setups for embodied agents; learning new behaviors (Behavior Incremental Learning, Behavior-IL) and new environments (Environment Incremental Learning, Environment-IL) For the tasks, previous 'data prior' based continual learning methods maintain logits for the past tasks. However, the stored information is often insufficiently learned information and requires task boundary information, which might not always be available. Here, we propose to update them based on confidence scores without task boundary information during training (i.e., task-free) in a moving average fashion, named Confidence-Aware Moving Average (CAMA). In the proposed Behavior-IL and Environment-IL setups, our simple CAMA outperforms prior state of the art in our empirical validations by noticeable margins. The project page including codes is https://github.com/snumprlab/cl-alfred. △ Less

Submitted 12 March, 2024; v1 submitted 12 March, 2024; originally announced March 2024.

Comments: ICLR 2024 (Project page: https://bhkim94.github.io/projects/CL-ALFRED)

Performance of SK-Gd's Upgraded Real-time Supernova Monitoring System

Authors: Y. Kashiwagi, K. Abe, C. Bronner, Y. Hayato, K. Hiraide, K. Hosokawa, K. Ieki, M. Ikeda, J. Kameda, Y. Kanemura, R. Kaneshima, Y. Kataoka, S. Miki, S. Mine, M. Miura, S. Moriyama, Y. Nakano, M. Nakahata, S. Nakayama, Y. Noguchi, K. Sato, H. Sekiya, H. Shiba, K. Shimizu, M. Shiozawa , et al. (214 additional authors not shown)

Abstract: Among multi-messenger observations of the next galactic core-collapse supernova, Super-Kamiokande (SK) plays a critical role in detecting the emitted supernova neutrinos, determining the direction to the supernova (SN), and notifying the astronomical community of these observations in advance of the optical signal. On 2022, SK has increased the gadolinium dissolved in its water target (SK-Gd) and… ▽ More Among multi-messenger observations of the next galactic core-collapse supernova, Super-Kamiokande (SK) plays a critical role in detecting the emitted supernova neutrinos, determining the direction to the supernova (SN), and notifying the astronomical community of these observations in advance of the optical signal. On 2022, SK has increased the gadolinium dissolved in its water target (SK-Gd) and has achieved a Gd concentration of 0.033%, resulting in enhanced neutron detection capability, which in turn enables more accurate determination of the supernova direction. Accordingly, SK-Gd's real-time supernova monitoring system (Abe te al. 2016b) has been upgraded. SK_SN Notice, a warning system that works together with this monitoring system, was released on December 13, 2021, and is available through GCN Notices (Barthelmy et al. 2000). When the monitoring system detects an SN-like burst of events, SK_SN Notice will automatically distribute an alarm with the reconstructed direction to the supernova candidate within a few minutes. In this paper, we present a systematic study of SK-Gd's response to a simulated galactic SN. Assuming a supernova situated at 10 kpc, neutrino fluxes from six supernova models are used to characterize SK-Gd's pointing accuracy using the same tools as the online monitoring system. The pointing accuracy is found to vary from 3-7$^\circ$ depending on the models. However, if the supernova is closer than 10 kpc, SK_SN Notice can issue an alarm with three-degree accuracy, which will benefit follow-up observations by optical telescopes with large fields of view. △ Less

Submitted 13 March, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: 38 pages, 29 figures, 6 tables

Joint-Embedding Masked Autoencoder for Self-supervised Learning of Dynamic Functional Connectivity from the Human Brain

Authors: Jungwon Choi, Hyungi Lee, Byung-Hoon Kim, Juho Lee

Abstract: Graph Neural Networks (GNNs) have shown promise in learning dynamic functional connectivity for distinguishing phenotypes from human brain networks. However, obtaining extensive labeled clinical data for training is often resource-intensive, making practical application difficult. Leveraging unlabeled data thus becomes crucial for representation learning in a label-scarce setting. Although generat… ▽ More Graph Neural Networks (GNNs) have shown promise in learning dynamic functional connectivity for distinguishing phenotypes from human brain networks. However, obtaining extensive labeled clinical data for training is often resource-intensive, making practical application difficult. Leveraging unlabeled data thus becomes crucial for representation learning in a label-scarce setting. Although generative self-supervised learning techniques, especially masked autoencoders, have shown promising results in representation learning in various domains, their application to dynamic graphs for dynamic functional connectivity remains underexplored, facing challenges in capturing high-level semantic representations. Here, we introduce the Spatio-Temporal Joint Embedding Masked Autoencoder (ST-JEMA), drawing inspiration from the Joint Embedding Predictive Architecture (JEPA) in computer vision. ST-JEMA employs a JEPA-inspired strategy for reconstructing dynamic graphs, which enables the learning of higher-level semantic representations considering temporal perspectives, addressing the challenges in fMRI data representation learning. Utilizing the large-scale UK Biobank dataset for self-supervised learning, ST-JEMA shows exceptional representation learning performance on dynamic functional connectivity demonstrating superiority over previous methods in predicting phenotypes and psychiatric diagnoses across eight benchmark fMRI datasets even with limited samples and effectiveness of temporal reconstruction on missing data scenarios. These findings highlight the potential of our approach as a robust representation learning method for leveraging label-scarce fMRI data. △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: Under review

MoST: Motion Style Transformer between Diverse Action Contents

Authors: Boeun Kim, Jungho Kim, Hyung Jin Chang, Jin Young Choi

Abstract: While existing motion style transfer methods are effective between two motions with identical content, their performance significantly diminishes when transferring style between motions with different contents. This challenge lies in the lack of clear separation between content and style of a motion. To tackle this challenge, we propose a novel motion style transformer that effectively disentangle… ▽ More While existing motion style transfer methods are effective between two motions with identical content, their performance significantly diminishes when transferring style between motions with different contents. This challenge lies in the lack of clear separation between content and style of a motion. To tackle this challenge, we propose a novel motion style transformer that effectively disentangles style from content and generates a plausible motion with transferred style from a source motion. Our distinctive approach to achieving the goal of disentanglement is twofold: (1) a new architecture for motion style transformer with `part-attentive style modulator across body parts' and `Siamese encoders that encode style and content features separately'; (2) style disentanglement loss. Our method outperforms existing methods and demonstrates exceptionally high quality, particularly in motion pairs with different contents, without the need for heuristic post-processing. Codes are available at https://github.com/Boeun-Kim/MoST. △ Less

Submitted 20 March, 2024; v1 submitted 10 March, 2024; originally announced March 2024.

Comments: Accepted by CVPR 2024

WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Concepts

Authors: Yong Hyun Ahn, Hyeon Bae Kim, Seong Tae Kim

Abstract: Recent advancements in neural networks have showcased their remarkable capabilities across various domains. Despite these successes, the "black box" problem still remains. Addressing this, we propose a novel framework, WWW, that offers the 'what', 'where', and 'why' of the neural network decisions in human-understandable terms. Specifically, WWW utilizes adaptive selection for concept discovery, e… ▽ More Recent advancements in neural networks have showcased their remarkable capabilities across various domains. Despite these successes, the "black box" problem still remains. Addressing this, we propose a novel framework, WWW, that offers the 'what', 'where', and 'why' of the neural network decisions in human-understandable terms. Specifically, WWW utilizes adaptive selection for concept discovery, employing adaptive cosine similarity and thresholding techniques to effectively explain 'what'. To address the 'where' and 'why', we proposed a novel combination of neuron activation maps (NAMs) with Shapley values, generating localized concept maps and heatmaps for individual inputs. Furthermore, WWW introduces a method for predicting uncertainty, leveraging heatmap similarities to estimate 'how' reliable the prediction is. Experimental evaluations of WWW demonstrate superior performance in both quantitative and qualitative metrics, outperforming existing methods in interpretability. WWW provides a unified solution for explaining 'what', 'where', and 'why', introducing a method for localized explanations from global interpretations and offering a plug-and-play solution adaptable to various architectures. △ Less

Submitted 11 April, 2024; v1 submitted 29 February, 2024; originally announced February 2024.

Comments: CVPR 2024

The frequency of metal-enrichment of cool helium-atmosphere white dwarfs using the DESI Early Data Release

Authors: Christopher J. Manser, Boris T. Gänsicke, Paula Izquierdo, Andrew Swan, Joan Najita, C. Rockosi, Andreia Carrillo, Bokyoung Kim, Siyi Xu, Arjun Dey, J. Aguilar, S. Ahlen, R. Blum, D. Brooks, T. Claybaugh, K. Dawson, A. de la Macorra, P. Doel, E. Gaztañaga, S. Gontcho A Gontcho, K. Honscheid, R. Kehoe, A. Kremin, M. Landriau, L. Le Guillou , et al. (13 additional authors not shown)

Abstract: There is overwhelming evidence that white dwarfs host planetary systems; revealed by the presence, disruption, and accretion of planetary bodies. A lower limit on the frequency of white dwarfs that host planetary material has been estimated to be roughly 25-50 per cent; inferred from the ongoing or recent accretion of metals onto both hydrogen-atmosphere and warm helium-atmosphere white dwarfs. No… ▽ More There is overwhelming evidence that white dwarfs host planetary systems; revealed by the presence, disruption, and accretion of planetary bodies. A lower limit on the frequency of white dwarfs that host planetary material has been estimated to be roughly 25-50 per cent; inferred from the ongoing or recent accretion of metals onto both hydrogen-atmosphere and warm helium-atmosphere white dwarfs. Now with the unbiased sample of white dwarfs observed by the Dark Energy Spectroscopic Instrument (DESI) survey in their Early Data Release (EDR), we have determined the frequency of metal-enrichment around cool-helium atmosphere white dwarfs as 21 $\pm$ 3 per cent using a sample of 234 systems. This value is in good agreement with values determined from previous studies. With the current samples we cannot distinguish whether the frequency of planetary accretion varies with system age or host-star mass, but the DESI data release 1 will contain roughly an order of magnitude more white dwarfs than DESI EDR and will allow these parameters to be investigated. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Comments: 6 pages, 1 figure. Submitted to MNRAS. Comments and suggestions welcome

The DESI Early Data Release White Dwarf Catalogue

Authors: Christopher J. Manser, Paula Izquierdo, Boris T. Gänsicke, Andrew Swan, Detlev Koester, Akshay Robert, Siyi Xu, Keith Inight, Ben Amroota, N. P. Gentile Fusillo, Sergey E. Koposov, Bokyoung Kim, Arjun Dey, Carlos Allende Prieto, J. Aguilar, S. Ahlen, R. Blum, D. Brooks, T. Claybaugh, A. P. Cooper, K. Dawson, A. de la Macorra, P. Doel, J. E. Forero-Romero, E. Gaztañaga , et al. (29 additional authors not shown)

Abstract: The Early Data Release (EDR) of the Dark Energy Spectroscopic Instrument (DESI) comprises spectroscopy obtained from 2020 December 14 to 2021 June 10. White dwarfs were targeted by DESI both as calibration sources and as science targets and were selected based on Gaia photometry and astrometry. Here we present the DESI EDR white dwarf catalogue, which includes 2706 spectroscopically confirmed whit… ▽ More The Early Data Release (EDR) of the Dark Energy Spectroscopic Instrument (DESI) comprises spectroscopy obtained from 2020 December 14 to 2021 June 10. White dwarfs were targeted by DESI both as calibration sources and as science targets and were selected based on Gaia photometry and astrometry. Here we present the DESI EDR white dwarf catalogue, which includes 2706 spectroscopically confirmed white dwarfs of which approximately 1630 (roughly 60 per cent) have been spectroscopically observed for the first time, as well as 66 white dwarf binary systems. We provide spectral classifications for all white dwarfs, and discuss their distribution within the Gaia Hertzsprung-Russell diagram. We provide atmospheric parameters derived from spectroscopic and photometric fits for white dwarfs with pure hydrogen or helium photospheres, a mixture of those two, and white dwarfs displaying carbon features in their spectra. We also discuss the less abundant systems in the sample, such as those with magnetic fields, and cataclysmic variables. The DESI EDR white dwarf sample is significantly less biased than the sample observed by the Sloan Digital Sky Survey, which is skewed to bluer and therefore hotter white dwarfs, making DESI more complete and suitable for performing statistical studies of white dwarfs. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Comments: 19 pages, 17 figures, not including appendix. Submitting to MNRAS. Comments and suggestions welcome

Objective and Interpretable Breast Cosmesis Evaluation with Attention Guided Denoising Diffusion Anomaly Detection Model

Authors: Sangjoon Park, Yong Bae Kim, Jee Suk Chang, Seo Hee Choi, Hyungjin Chung, Ik Jae Lee, Hwa Kyung Byun

Abstract: As advancements in the field of breast cancer treatment continue to progress, the assessment of post-surgical cosmetic outcomes has gained increasing significance due to its substantial impact on patients' quality of life. However, evaluating breast cosmesis presents challenges due to the inherently subjective nature of expert labeling. In this study, we present a novel automated approach, Attenti… ▽ More As advancements in the field of breast cancer treatment continue to progress, the assessment of post-surgical cosmetic outcomes has gained increasing significance due to its substantial impact on patients' quality of life. However, evaluating breast cosmesis presents challenges due to the inherently subjective nature of expert labeling. In this study, we present a novel automated approach, Attention-Guided Denoising Diffusion Anomaly Detection (AG-DDAD), designed to assess breast cosmesis following surgery, addressing the limitations of conventional supervised learning and existing anomaly detection models. Our approach leverages the attention mechanism of the distillation with no label (DINO) self-supervised Vision Transformer (ViT) in combination with a diffusion model to achieve high-quality image reconstruction and precise transformation of discriminative regions. By training the diffusion model on unlabeled data predominantly with normal cosmesis, we adopt an unsupervised anomaly detection perspective to automatically score the cosmesis. Real-world data experiments demonstrate the effectiveness of our method, providing visually appealing representations and quantifiable scores for cosmesis evaluation. Compared to commonly used rule-based programs, our fully automated approach eliminates the need for manual annotations and offers objective evaluation. Moreover, our anomaly detection model exhibits state-of-the-art performance, surpassing existing models in accuracy. Going beyond the scope of breast cosmesis, our research represents a significant advancement in unsupervised anomaly detection within the medical domain, thereby paving the way for future investigations. △ Less

Submitted 28 February, 2024; originally announced February 2024.

No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization

Authors: June Yong Yang, Byeongwook Kim, Jeongin Bae, Beomseok Kwon, Gunho Park, Eunho Yang, Se Jung Kwon, Dongsoo Lee

Abstract: Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs). However, the memory footprint of the KV cache poses a critical bottleneck in LLM deployment as the cache size grows with batch size and sequence length, often surpassing even the size of the model itself. Although recent methods were proposed to s… ▽ More Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs). However, the memory footprint of the KV cache poses a critical bottleneck in LLM deployment as the cache size grows with batch size and sequence length, often surpassing even the size of the model itself. Although recent methods were proposed to select and evict unimportant KV pairs from the cache to reduce memory consumption, the potential ramifications of eviction on the generative process are yet to be thoroughly examined. In this paper, we examine the detrimental impact of cache eviction and observe that unforeseen risks arise as the information contained in the KV pairs is exhaustively discarded, resulting in safety breaches, hallucinations, and context loss. Surprisingly, we find that preserving even a small amount of information contained in the evicted KV pairs via reduced precision quantization substantially recovers the incurred degradation. On the other hand, we observe that the important KV pairs must be kept at a relatively higher precision to safeguard the generation quality. Motivated by these observations, we propose \textit{Mixed-precision KV cache}~(MiKV), a reliable cache compression method that simultaneously preserves the context details by retaining the evicted KV pairs in low-precision and ensure generation quality by keeping the important KV pairs in high-precision. Experiments on diverse benchmarks and LLM backbones show that our proposed method offers a state-of-the-art trade-off between compression ratio and performance, compared to other baselines. △ Less

Submitted 28 February, 2024; originally announced February 2024.

DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation

Authors: Sunghyeon Woo, Baeseong Park, Byeongwook Kim, Minjung Jo, Sejung Kwon, Dongsuk Jeon, Dongsoo Lee

Abstract: Training deep neural networks typically involves substantial computational costs during both forward and backward propagation. The conventional layer dropping techniques drop certain layers during training for reducing the computations burden. However, dropping layers during forward propagation adversely affects the training process by degrading accuracy. In this paper, we propose Dropping Backwar… ▽ More Training deep neural networks typically involves substantial computational costs during both forward and backward propagation. The conventional layer dropping techniques drop certain layers during training for reducing the computations burden. However, dropping layers during forward propagation adversely affects the training process by degrading accuracy. In this paper, we propose Dropping Backward Propagation (DropBP), a novel approach designed to reduce computational costs while maintaining accuracy. DropBP randomly drops layers during the backward propagation, which does not deviate forward propagation. Moreover, DropBP calculates the sensitivity of each layer to assign appropriate drop rate, thereby stabilizing the training process. DropBP is designed to enhance the efficiency of the training process with backpropagation, thereby enabling the acceleration of both full fine-tuning and parameter-efficient fine-tuning using backpropagation. Specifically, utilizing DropBP in QLoRA reduces training time by 44%, increases the convergence speed to the identical loss level by 1.5$\times$, and enables training with a 6.2$\times$ larger sequence length on a single NVIDIA-A100 80GiB GPU in LLaMA2-70B. The code is available at https://github.com/WooSunghyeon/dropbp. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Eigenstate switching of topologically ordered states using non-Hermitian perturbations

Authors: Cheol Hun Yeom, Beom Hyun Kim, Moon Jip Park

Abstract: Topologically ordered phases have robust degenerate ground states against the local perturbations, providing a promising platform for fault-tolerant quantum computation. Despite of the non-local feature of the topological order, we find that local non-Hermitian perturbations can induce the transition between the topologically ordered ground states. In this work, we study the toric code in the pres… ▽ More Topologically ordered phases have robust degenerate ground states against the local perturbations, providing a promising platform for fault-tolerant quantum computation. Despite of the non-local feature of the topological order, we find that local non-Hermitian perturbations can induce the transition between the topologically ordered ground states. In this work, we study the toric code in the presence of non-Hermitian perturbations. By controlling the non-Hermiticity, we show that non-orthogonal ground states can exhibit an eigenstate coalescence and have the spectral singularity, known as an exceptional point (EP). We explore the potential of the EPs in the control of topological order. Adiabatic encircling EPs allows for the controlled switching of eigenstates, enabling dynamic manipulation between the ground state degeneracy. Interestingly, we show a property of our scheme that arbitrary strengths of local perturbations can induce the EP and eigenstate switching. Finally, we also show the orientation-dependent behavior of non-adiabatic transitions (NAT) during the dynamic encirclement around an EP. Our work shows that control of the non-Hermiticity can serve as a promising strategy for fault-tolerant quantum information processing. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: 5 pages and supplementary materials

Parameter-Free Algorithms for Performative Regret Minimization under Decision-Dependent Distributions

Authors: Sungwoo Park, Junyeop Kwon, Byeongnoh Kim, Suhyun Chae, Jeeyong Lee, Dabeen Lee

Abstract: This paper studies performative risk minimization, a formulation of stochastic optimization under decision-dependent distributions. We consider the general case where the performative risk can be non-convex, for which we develop efficient parameter-free optimistic optimization-based methods. Our algorithms significantly improve upon the existing Lipschitz bandit-based method in many aspects. In pa… ▽ More This paper studies performative risk minimization, a formulation of stochastic optimization under decision-dependent distributions. We consider the general case where the performative risk can be non-convex, for which we develop efficient parameter-free optimistic optimization-based methods. Our algorithms significantly improve upon the existing Lipschitz bandit-based method in many aspects. In particular, our framework does not require knowledge about the sensitivity parameter of the distribution map and the Lipshitz constant of the loss function. This makes our framework practically favorable, together with the efficient optimistic optimization-based tree-search mechanism. We provide experimental results that demonstrate the numerical superiority of our algorithms over the existing method and other black-box optimistic optimization methods. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Transformable Gaussian Reward Function for Socially-Aware Navigation with Deep Reinforcement Learning

Authors: Jinyeob Kim, Sumin Kang, Sungwoo Yang, Beomjoon Kim, Jargalbaatar Yura, Donghan Kim

Abstract: Robot navigation has transitioned from prioritizing obstacle avoidance to adopting socially aware navigation strategies that accommodate human presence. As a result, the recognition of socially aware navigation within dynamic human-centric environments has gained prominence in the field of robotics. Although reinforcement learning technique has fostered the advancement of socially aware navigation… ▽ More Robot navigation has transitioned from prioritizing obstacle avoidance to adopting socially aware navigation strategies that accommodate human presence. As a result, the recognition of socially aware navigation within dynamic human-centric environments has gained prominence in the field of robotics. Although reinforcement learning technique has fostered the advancement of socially aware navigation, defining appropriate reward functions, especially in congested environments, has posed a significant challenge. These rewards, crucial in guiding robot actions, demand intricate human-crafted design due to their complex nature and inability to be automatically set. The multitude of manually designed rewards poses issues with hyperparameter redundancy, imbalance, and inadequate representation of unique object characteristics. To address these challenges, we introduce a transformable gaussian reward function (TGRF). The TGRF significantly reduces the burden of hyperparameter tuning, displays adaptability across various reward functions, and demonstrates accelerated learning rates, particularly excelling in crowded environments utilizing deep reinforcement learning (DRL). We introduce and validate TGRF through sections highlighting its conceptual background, characteristics, experiments, and real-world application, paving the way for a more effective and adaptable approach in robotics.The complete source code is available on https://github.com/JinnnK/TGRF △ Less

Submitted 6 June, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: 22 pages, 9 figures

Construction of Yemilab

Authors: K. S. Park, Y. D. Kim, K. M. Bang, H. K Park, M. H. Lee, J. H. Jang, J. H. Kim, J. So, S. H. Kim, S. B. Kim

Abstract: The Center for Underground Physics of the Institute for Basic Science (IBS) in Korea has been planning the construction of a deep underground laboratory since 2013 to search for extremely rare interactions such as dark matter and neutrinos. In September 2022, a new underground laboratory, Yemilab, was finally completed in Jeongseon, Gangwon Province, with a depth of 1,000 m and an exclusive experi… ▽ More The Center for Underground Physics of the Institute for Basic Science (IBS) in Korea has been planning the construction of a deep underground laboratory since 2013 to search for extremely rare interactions such as dark matter and neutrinos. In September 2022, a new underground laboratory, Yemilab, was finally completed in Jeongseon, Gangwon Province, with a depth of 1,000 m and an exclusive experimental area spanning 3,000 m$^3$. The tunnel is encased in limestone and accommodates 17 independent experimental spaces. Over two years, from 2023 to 2024, the Yangyang Underground Laboratory facilities will be relocated to Yemilab. Preparations are underway for the AMoRE-II, a neutrinoless double beta decay experiment, scheduled to begin in Q2 2024 at Yemilab. Additionally, Yemilab includes a cylindrical pit with a volume of approximately 6,300 m$^3$, designed as a multipurpose laboratory for next-generation experiments involving neutrinos, dark matter, and related research. This article provides a focused overview of the construction and structure of Yemilab. △ Less

Submitted 21 February, 2024; originally announced February 2024.

Comments: 12 pages, 3 figures, 1 table

Journal ref: Frontiers in Physics, vol. 12, 1323991 (2024)

InMD-X: Large Language Models for Internal Medicine Doctors

Authors: Hansle Gwon, Imjin Ahn, Hyoje Jung, Byeolhee Kim, Young-Hak Kim, Tae Joon Jun

Abstract: In this paper, we introduce InMD-X, a collection of multiple large language models specifically designed to cater to the unique characteristics and demands of Internal Medicine Doctors (IMD). InMD-X represents a groundbreaking development in natural language processing, offering a suite of language models fine-tuned for various aspects of the internal medicine field. These models encompass a wide… ▽ More In this paper, we introduce InMD-X, a collection of multiple large language models specifically designed to cater to the unique characteristics and demands of Internal Medicine Doctors (IMD). InMD-X represents a groundbreaking development in natural language processing, offering a suite of language models fine-tuned for various aspects of the internal medicine field. These models encompass a wide range of medical sub-specialties, enabling IMDs to perform more efficient and accurate research, diagnosis, and documentation. InMD-X's versatility and adaptability make it a valuable tool for improving the healthcare industry, enhancing communication between healthcare professionals, and advancing medical research. Each model within InMD-X is meticulously tailored to address specific challenges faced by IMDs, ensuring the highest level of precision and comprehensiveness in clinical text analysis and decision support. This paper provides an overview of the design, development, and evaluation of InMD-X, showcasing its potential to revolutionize the way internal medicine practitioners interact with medical data and information. We present results from extensive testing, demonstrating the effectiveness and practical utility of InMD-X in real-world medical scenarios. △ Less

Submitted 19 February, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

PEGASUS: Personalized Generative 3D Avatars with Composable Attributes

Authors: Hyunsoo Cha, Byungjun Kim, Hanbyul Joo

Abstract: We present PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources. Our generative 3D avatar enables disentangled controls to selectively alter the facial attributes (e.g., hair or nose) while preserving the identity. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar. We generate a s… ▽ More We present PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources. Our generative 3D avatar enables disentangled controls to selectively alter the facial attributes (e.g., hair or nose) while preserving the identity. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar. We generate a synthetic video collection of the target identity with varying facial attributes, where the videos are synthesized by borrowing the attributes from monocular videos of diverse identities. Then, we build a person-specific generative 3D avatar that can modify its attributes continuously while preserving its identity. Through extensive experiments, we demonstrate that our method of generating a synthetic database and creating a 3D generative avatar is the most effective in preserving identity while achieving high realism. Subsequently, we introduce a zero-shot approach to achieve the same goal of generative modeling more efficiently by leveraging a previously constructed personalized generative model. △ Less

Submitted 2 April, 2024; v1 submitted 16 February, 2024; originally announced February 2024.

Comments: Accepted at CVPR 2024, Project Page: https://snuvclab.github.io/pegasus/

Automated Classification of Body MRI Sequence Type Using Convolutional Neural Networks

Authors: Kimberly Helm, Tejas Sudharshan Mathai, Boah Kim, Pritam Mukherjee, Jianfei Liu, Ronald M. Summers

Abstract: Multi-parametric MRI of the body is routinely acquired for the identification of abnormalities and diagnosis of diseases. However, a standard naming convention for the MRI protocols and associated sequences does not exist due to wide variations in imaging practice at institutions and myriad MRI scanners from various manufacturers being used for imaging. The intensity distributions of MRI sequences… ▽ More Multi-parametric MRI of the body is routinely acquired for the identification of abnormalities and diagnosis of diseases. However, a standard naming convention for the MRI protocols and associated sequences does not exist due to wide variations in imaging practice at institutions and myriad MRI scanners from various manufacturers being used for imaging. The intensity distributions of MRI sequences differ widely as a result, and there also exists information conflicts related to the sequence type in the DICOM headers. At present, clinician oversight is necessary to ensure that the correct sequence is being read and used for diagnosis. This poses a challenge when specific series need to be considered for building a cohort for a large clinical study or for developing AI algorithms. In order to reduce clinician oversight and ensure the validity of the DICOM headers, we propose an automated method to classify the 3D MRI sequence acquired at the levels of the chest, abdomen, and pelvis. In our pilot work, our 3D DenseNet-121 model achieved an F1 score of 99.5% at differentiating 5 common MRI sequences obtained by three Siemens scanners (Aera, Verio, Biograph mMR). To the best of our knowledge, we are the first to develop an automated method for the 3D classification of MRI sequences in the chest, abdomen, and pelvis, and our work has outperformed the previous state-of-the-art MRI series classifiers. △ Less

Submitted 12 February, 2024; originally announced February 2024.

Comments: Accepted at SPIE 2024

System-level Analysis of Adversarial Attacks and Defenses on Intelligence in O-RAN based Cellular Networks

Authors: Azuka Chiejina, Brian Kim, Kaushik Chowhdury, Vijay K. Shah

Abstract: While the open architecture, open interfaces, and integration of intelligence within Open Radio Access Network technology hold the promise of transforming 5G and 6G networks, they also introduce cybersecurity vulnerabilities that hinder its widespread adoption. In this paper, we conduct a thorough system-level investigation of cyber threats, with a specific focus on machine learning (ML) intellige… ▽ More While the open architecture, open interfaces, and integration of intelligence within Open Radio Access Network technology hold the promise of transforming 5G and 6G networks, they also introduce cybersecurity vulnerabilities that hinder its widespread adoption. In this paper, we conduct a thorough system-level investigation of cyber threats, with a specific focus on machine learning (ML) intelligence components known as xApps within the O-RAN's near-real-time RAN Intelligent Controller (near-RT RIC) platform. Our study begins by developing a malicious xApp designed to execute adversarial attacks on two types of test data - spectrograms and key performance metrics (KPMs), stored in the RIC database within the near-RT RIC. To mitigate these threats, we utilize a distillation technique that involves training a teacher model at a high softmax temperature and transferring its knowledge to a student model trained at a lower softmax temperature, which is deployed as the robust ML model within xApp. We prototype an over-the-air LTE/5G O-RAN testbed to assess the impact of these attacks and the effectiveness of the distillation defense technique by leveraging an ML-based Interference Classification (InterClass) xApp as an example. We examine two versions of InterClass xApp under distinct scenarios, one based on Convolutional Neural Networks (CNNs) and another based on Deep Neural Networks (DNNs) using spectrograms and KPMs as input data respectively. Our findings reveal up to 100% and 96.3% degradation in the accuracy of both the CNN and DNN models respectively resulting in a significant decline in network performance under considered adversarial attacks. Under the strict latency constraints of the near-RT RIC closed control loop, our analysis shows that the distillation technique outperforms classical adversarial training by achieving an accuracy of up to 98.3% for mitigating such attacks. △ Less

Submitted 13 February, 2024; v1 submitted 9 February, 2024; originally announced February 2024.

Comments: his paper has been accepted for publication in ACM WiSec 2024

Minecraft-ify: Minecraft Style Image Generation with Text-guided Image Editing for In-Game Application

Authors: Bumsoo Kim, Sanghyun Byun, Yonghoon Jung, Wonseop Shin, Sareer UI Amin, Sanghyun Seo

Abstract: In this paper, we first present the character texture generation system \textit{Minecraft-ify}, specified to Minecraft video game toward in-game application. Ours can generate face-focused image for texture mapping tailored to 3D virtual character having cube manifold. While existing projects or works only generate texture, proposed system can inverse the user-provided real image, or generate aver… ▽ More In this paper, we first present the character texture generation system \textit{Minecraft-ify}, specified to Minecraft video game toward in-game application. Ours can generate face-focused image for texture mapping tailored to 3D virtual character having cube manifold. While existing projects or works only generate texture, proposed system can inverse the user-provided real image, or generate average/random appearance from learned distribution. Moreover, it can be manipulated with text-guidance using StyleGAN and StyleCLIP. These features provide a more extended user experience with enlarged freedom as a user-friendly AI-tool. Project page can be found at https://gh-bumsookim.github.io/Minecraft-ify/ △ Less

Submitted 3 March, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: 2 pages, 2 figures. Accepted as Spotlight to NeurIPS 2023 Workshop on Machine Learning for Creativity and Design

Noise Map Guidance: Inversion with Spatial Context for Real Image Editing

Authors: Hansam Cho, Jonghyun Lee, Seoung Bum Kim, Tae-Hyun Oh, Yonghyun Jeong

Abstract: Text-guided diffusion models have become a popular tool in image synthesis, known for producing high-quality and diverse images. However, their application to editing real images often encounters hurdles primarily due to the text condition deteriorating the reconstruction quality and subsequently affecting editing fidelity. Null-text Inversion (NTI) has made strides in this area, but it fails to c… ▽ More Text-guided diffusion models have become a popular tool in image synthesis, known for producing high-quality and diverse images. However, their application to editing real images often encounters hurdles primarily due to the text condition deteriorating the reconstruction quality and subsequently affecting editing fidelity. Null-text Inversion (NTI) has made strides in this area, but it fails to capture spatial context and requires computationally intensive per-timestep optimization. Addressing these challenges, we present Noise Map Guidance (NMG), an inversion method rich in a spatial context, tailored for real-image editing. Significantly, NMG achieves this without necessitating optimization, yet preserves the editing quality. Our empirical investigations highlight NMG's adaptability across various editing techniques and its robustness to variants of DDIM inversions. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: ICLR 2024

Association between Prefrontal fNIRS signals during Cognitive tasks and College scholastic ability test (CSAT) scores: Analysis using a quantum annealing approach

Authors: Yeaju Kim, Junggu Choi, Bora Kim, Yongwan Park, Jihyun Cha, Jongkwan Choi, Sanghoon Han

Abstract: Academic achievement is a critical measure of intellectual ability, prompting extensive research into cognitive tasks as potential predictors. Neuroimaging technologies, such as functional near-infrared spectroscopy (fNIRS), offer insights into brain hemodynamics, allowing understanding of the link between cognitive performance and academic achievement. Herein, we explored the association between… ▽ More Academic achievement is a critical measure of intellectual ability, prompting extensive research into cognitive tasks as potential predictors. Neuroimaging technologies, such as functional near-infrared spectroscopy (fNIRS), offer insights into brain hemodynamics, allowing understanding of the link between cognitive performance and academic achievement. Herein, we explored the association between cognitive tasks and academic achievement by analyzing prefrontal fNIRS signals. A novel quantum annealer (QA) feature selection algorithm was applied to fNIRS data to identify cognitive tasks correlated with CSAT scores. Twelve features (signal mean, median, variance, peak, number of peaks, sum of peaks, slope, minimum, kurtosis, skewness, standard deviation, and root mean square) were extracted from fNIRS signals at two time windows (10- and 60-second) to compare results from various feature variable conditions. The feature selection results from the QA-based and XGBoost regressor algorithms were compared to validate the former's performance. In a three-step validation process using multiple linear regression models, correlation coefficients between the feature variables and the CSAT scores, model fitness (adjusted R2), and model prediction error (RMSE) values were calculated. The quantum annealer demonstrated comparable performance to classical machine learning models, and specific cognitive tasks, including verbal fluency, recognition, and the Corsi block tapping task, were correlated with academic achievement. Group analyses revealed stronger associations between Tower of London and N-back tasks with higher CSAT scores. Quantum annealing algorithms have significant potential in feature selection using fNIRS data, and represents a novel research approach. Future studies should explore predictors of academic achievement and cognitive ability. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 42 pages, 11 tables

Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods

Authors: Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, Hyoung-Kyu Song

Abstract: Structured pruning of modern large language models (LLMs) has emerged as a way of decreasing their high computational needs. Width pruning reduces the size of projection weight matrices (e.g., by removing attention heads) while maintaining the number of layers. Depth pruning, in contrast, removes entire layers or blocks, while keeping the size of the remaining weights unchanged. Most current resea… ▽ More Structured pruning of modern large language models (LLMs) has emerged as a way of decreasing their high computational needs. Width pruning reduces the size of projection weight matrices (e.g., by removing attention heads) while maintaining the number of layers. Depth pruning, in contrast, removes entire layers or blocks, while keeping the size of the remaining weights unchanged. Most current research focuses on either width-only or a blend of width and depth pruning, with little comparative analysis between the two units (width vs. depth) concerning their impact on LLM inference efficiency. In this work, we show that simple depth pruning can effectively compress LLMs while achieving comparable or superior performance to recent width pruning studies. Our pruning method boosts inference speeds, especially under memory-constrained conditions that require limited batch sizes for running LLMs, where width pruning is ineffective. In retraining pruned models for quality recovery, continued pretraining on a large corpus markedly outperforms LoRA-based tuning, particularly at severe pruning ratios. We hope this work can help build compact yet capable LLMs. Code and models can be found at: https://github.com/Nota-NetsPresso/shortened-llm △ Less

Submitted 23 June, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Comments: Update (arXiv-v2): continued pretraining for severe pruning ratios, compatibility with quantization, and enhanced baselines. Preliminary work (arXiv-v1) accepted at ICLR 2024 Workshop on ME-FoMo: https://openreview.net/forum?id=18VGxuOdpu

ToonAging: Face Re-Aging upon Artistic Portrait Style Transfer

Authors: Bumsoo Kim, Abdul Muqeet, Kyuchul Lee, Sanghyun Seo

Abstract: Face re-aging is a prominent field in computer vision and graphics, with significant applications in photorealistic domains such as movies, advertising, and live streaming. Recently, the need to apply face re-aging to non-photorealistic images, like comics, illustrations, and animations, has emerged as an extension in various entertainment sectors. However, the lack of a network that can seamlessl… ▽ More Face re-aging is a prominent field in computer vision and graphics, with significant applications in photorealistic domains such as movies, advertising, and live streaming. Recently, the need to apply face re-aging to non-photorealistic images, like comics, illustrations, and animations, has emerged as an extension in various entertainment sectors. However, the lack of a network that can seamlessly edit the apparent age in NPR images has limited these tasks to a naive, sequential approach. This often results in unpleasant artifacts and a loss of facial attributes due to domain discrepancies. In this paper, we introduce a novel one-stage method for face re-aging combined with portrait style transfer, executed in a single generative step. We leverage existing face re-aging and style transfer networks, both trained within the same PR domain. Our method uniquely fuses distinct latent vectors, each responsible for managing aging-related attributes and NPR appearance. By adopting an exemplar-based approach, our method offers greater flexibility compared to domain-level fine-tuning approaches, which typically require separate training or fine-tuning for each domain. This effectively addresses the limitation of requiring paired datasets for re-aging and domain-level, data-driven approaches for stylization. Our experiments show that our model can effortlessly generate re-aged images while simultaneously transferring the style of examples, maintaining both natural appearance and controllability. △ Less

Submitted 28 May, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Comments: Accepted at CVPR 2024 AI4CC Workshop, Project Page: https://gh-bumsookim.github.io/ToonAging/

Untangle charge-order dependent bulk states from surface effects in a topological kagome metal ScV$_6$Sn$_6$

Authors: Zi-Jia Cheng, Sen Shao, Byunghoon Kim, Tyler A. Cochran, Xian P. Yang, Changjiang Yi, Yu-Xiao Jiang, Junyi Zhang, Md Shafayat Hossain, Subhajit Roychowdhury, Turgut Yilmaz, Elio Vescovo, Alexei Fedorov, Shekhar Chandra, Claudia Felser, Guoqing Chang, M. Zahid Hasan

Abstract: Kagome metals with charge density wave (CDW) order exhibit a broad spectrum of intriguing quantum phenomena. The recent discovery of the novel kagome CDW compound ScV$_6$Sn$_6$ has spurred significant interest. However, understanding the interplay between CDW and the bulk electronic structure has been obscured by a profusion of surface states and terminations in this quantum material. Here, we emp… ▽ More Kagome metals with charge density wave (CDW) order exhibit a broad spectrum of intriguing quantum phenomena. The recent discovery of the novel kagome CDW compound ScV$_6$Sn$_6$ has spurred significant interest. However, understanding the interplay between CDW and the bulk electronic structure has been obscured by a profusion of surface states and terminations in this quantum material. Here, we employ photoemission spectroscopy and potassium dosing to elucidate the complete bulk band structure of ScV$_6$Sn$_6$, revealing multiple van Hove singularities near the Fermi level. We surprisingly discover a robust spin-polarized topological Dirac surface resonance state at the M point within the two-fold van Hove singularities. Assisted by the first-principle calculations, the temperature dependence of the $k_z$- resolved ARPES spectrum provides unequivocal evidence for the proposed $\sqrt{3}$$\times$$\sqrt{3}$$\times3$ charge order over other candidates. Our work not only enhances the understanding of the CDW-dependent bulk and surface states in ScV$_6$Sn$_6$ but also establishes an essential foundation for potential manipulation of the CDW order in kagome materials. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Comments: To appear in PRB

Contrasting twisted bilayer graphene and transition metal dichalcogenides for fractional Chern insulators: an emergent gauge picture

Authors: Heqiu Li, Ying Su, Yong Baek Kim, Hae-Young Kee, Kai Sun, Shi-Zeng Lin

Abstract: The recent experimental discovery of the zero-field fractional Chern insulator (FCI) in twisted $\mathrm{MoTe_2}$ moiré superlattices has sparked immense interest in this exotic topological quantum state. The FCI has also been observed in previous experiments in magic angle twisted bilayer graphene (TBG) under a finite magnetic field of about 5 Tesla. Generally, the stabilization of FCI requires f… ▽ More The recent experimental discovery of the zero-field fractional Chern insulator (FCI) in twisted $\mathrm{MoTe_2}$ moiré superlattices has sparked immense interest in this exotic topological quantum state. The FCI has also been observed in previous experiments in magic angle twisted bilayer graphene (TBG) under a finite magnetic field of about 5 Tesla. Generally, the stabilization of FCI requires fine-tuning the topological band to satisfy certain conditions. It would still be helpful to have an intuitive picture to understand the different behaviors in twisted $\mathrm{MoTe_2}$ and TBG. Here, we compare them through the lens of emergent gauge fields. In TBG, the system can be mapped to two Dirac fermions coupled to emergent gauge fields with opposite signs. In contrast, the twisted $\mathrm{MoTe_2}$ reduces to a hole with parabolic dispersion coupled to an emergent gauge field. This contrasting gauge structure provides a new perspective on the observed difference: the zero-field FCI is stable in $\mathrm{MoTe_2}$ but absent in TBG. Based on this understanding, we will explore potential strategies for stabilizing FCI in both moiré superlattices. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Journal ref: PhysRevB.109.245131 (2024)

Amorphous Boron Nitride as a Diffusion Barrier to Cu Atoms

Authors: Onurcan Kaya, Hyeongjoon Kim, Byeongkyu Kim, Luigi Colombo, Hyeon-Jin Shin, Ivan Cole, Hyeon Suk Shin, Stephan Roche

Abstract: This study focuses on amorphous boron nitride ($\rm α$-BN) as a novel diffusion barrier for advanced semiconductor technology, particularly addressing the critical challenge of copper diffusion in back-end-of-logic (BEOL) interconnects. Owing to its ultralow dielectric constant and robust barrier properties, $\rm α$-BN is examined as an alternative to conventional low-k dielectrics. The investigat… ▽ More This study focuses on amorphous boron nitride ($\rm α$-BN) as a novel diffusion barrier for advanced semiconductor technology, particularly addressing the critical challenge of copper diffusion in back-end-of-logic (BEOL) interconnects. Owing to its ultralow dielectric constant and robust barrier properties, $\rm α$-BN is examined as an alternative to conventional low-k dielectrics. The investigation primarily employs theoretical modeling, using a Gaussian Approximation Potential, to simulate and understand the atomic-level interactions and barrier mechanisms of $\rm α$-BN. This machine learning-based approach allows for realistic simulations of its amorphous structure, enabling the exploration of the impact of different film morphologies on barrier efficacy. Complementing the theoretical study, experimental analyses are conducted on Plasma-Enhanced Chemical Vapor Deposition (PECVD) grown $\rm α$-BN films, evaluating their effectiveness in preventing copper diffusion in silicon-based substrates. The results from both the theoretical and experimental investigations highlight the potential of $\rm α$-BN as a highly effective diffusion barrier, suitable for integration in nanoelectronics. This research not only proposes $\rm α$-BN as a promising candidate for BEOL interconnects but also demonstrates the synergy of advanced computational models and experimental methods in material innovation for semiconductor applications. △ Less

Submitted 2 February, 2024; originally announced February 2024.

Comments: 16 pages, 9 figures

Scale Equalization for Multi-Level Feature Fusion

Authors: Bum Jun Kim, Sang Woo Kim

Abstract: Deep neural networks have exhibited remarkable performance in a variety of computer vision fields, especially in semantic segmentation tasks. Their success is often attributed to multi-level feature fusion, which enables them to understand both global and local information from an image. However, we found that multi-level features from parallel branches are on different scales. The scale disequili… ▽ More Deep neural networks have exhibited remarkable performance in a variety of computer vision fields, especially in semantic segmentation tasks. Their success is often attributed to multi-level feature fusion, which enables them to understand both global and local information from an image. However, we found that multi-level features from parallel branches are on different scales. The scale disequilibrium is a universal and unwanted flaw that leads to detrimental gradient descent, thereby degrading performance in semantic segmentation. We discover that scale disequilibrium is caused by bilinear upsampling, which is supported by both theoretical and empirical evidence. Based on this observation, we propose injecting scale equalizers to achieve scale equilibrium across multi-level features after bilinear upsampling. Our proposed scale equalizers are easy to implement, applicable to any architecture, hyperparameter-free, implementable without requiring extra computational cost, and guarantee scale equilibrium for any dataset. Experiments showed that adopting scale equalizers consistently improved the mIoU index across various target datasets, including ADE20K, PASCAL VOC 2012, and Cityscapes, as well as various decoder choices, including UPerHead, PSPHead, ASPPHead, SepASPPHead, and FCNHead. △ Less

Submitted 2 February, 2024; originally announced February 2024.

Comments: 10 pages, 3 figures

Enhanced fringe-to-phase framework using deep learning

Authors: Won-Hoe Kim, Bongjoong Kim, Hyung-Gun Chi, Jae-Sang Hyun

Abstract: In Fringe Projection Profilometry (FPP), achieving robust and accurate 3D reconstruction with a limited number of fringe patterns remains a challenge in structured light 3D imaging. Conventional methods require a set of fringe images, but using only one or two patterns complicates phase recovery and unwrapping. In this study, we introduce SFNet, a symmetric fusion network that transforms two fring… ▽ More In Fringe Projection Profilometry (FPP), achieving robust and accurate 3D reconstruction with a limited number of fringe patterns remains a challenge in structured light 3D imaging. Conventional methods require a set of fringe images, but using only one or two patterns complicates phase recovery and unwrapping. In this study, we introduce SFNet, a symmetric fusion network that transforms two fringe images into an absolute phase. To enhance output reliability, Our framework predicts refined phases by incorporating information from fringe images of a different frequency than those used as input. This allows us to achieve high accuracy with just two images. Comparative experiments and ablation studies validate the effectiveness of our proposed method. The dataset and code are publicly accessible on our project page https://wonhoe-kim.github.io/SFNet. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 35 pages, 13 figures, 6 tables

Task-Oriented Diffusion Model Compression

Authors: Geonung Kim, Beomsu Kim, Eunhyeok Park, Sunghyun Cho

Abstract: As recent advancements in large-scale Text-to-Image (T2I) diffusion models have yielded remarkable high-quality image generation, diverse downstream Image-to-Image (I2I) applications have emerged. Despite the impressive results achieved by these I2I models, their practical utility is hampered by their large model size and the computational burden of the iterative denoising process. In this paper,… ▽ More As recent advancements in large-scale Text-to-Image (T2I) diffusion models have yielded remarkable high-quality image generation, diverse downstream Image-to-Image (I2I) applications have emerged. Despite the impressive results achieved by these I2I models, their practical utility is hampered by their large model size and the computational burden of the iterative denoising process. In this paper, we explore the compression potential of these I2I models in a task-oriented manner and introduce a novel method for reducing both model size and the number of timesteps. Through extensive experiments, we observe key insights and use our empirical knowledge to develop practical solutions that aim for near-optimal results with minimal exploration costs. We validate the effectiveness of our method by applying it to InstructPix2Pix for image editing and StableSR for image restoration. Our approach achieves satisfactory output quality with 39.2% and 56.4% reduction in model footprint and 81.4% and 68.7% decrease in latency to InstructPix2Pix and StableSR, respectively. △ Less

Submitted 30 January, 2024; originally announced January 2024.

Quantum-Secure Hybrid Blockchain System for DID-based Verifiable Random Function with NTRU Linkable Ring Signature

Authors: Bong Gon Kim, Dennis Wong, Yoon Seok Yang

Abstract: In this study, we present a secure smart contract-based Verifiable Random Function (VRF) model, addressing the shortcomings of existing systems. As quantum computing emerges, conventional public key cryptography faces potential vulnerabilities. To enhance our VRF's robustness, we employ post-quantum Ring-LWE encryption for generating pseudo-random sequences. Given the computational intensity of th… ▽ More In this study, we present a secure smart contract-based Verifiable Random Function (VRF) model, addressing the shortcomings of existing systems. As quantum computing emerges, conventional public key cryptography faces potential vulnerabilities. To enhance our VRF's robustness, we employ post-quantum Ring-LWE encryption for generating pseudo-random sequences. Given the computational intensity of this approach and associated on-chain gas costs, we propose a hybrid architecture of VRF system where on-chain and off-chain can communicate in a scalable and secure way. To ensure the validity and integrity of the off-chain computations (e.g., Ring-LWE encryption), we employ a quantum-secure linkable ring signature scheme on NTRU lattice and also delegated key generation (DKG) with a secure key encapsulation mechanism (KEM). Our decentralized VRF employs multi-party computation (MPC) with blockchain-based decentralized identifiers (DID), ensuring the collective efforts of enhanced randomness and security. We show the security and privacy advantages of our proposed VRF model with the approximated estimation of overall temporal and spatial complexities. We also evaluate our VRF MPC model's entropy and outline its Solidity smart contract integration. This research also provides a method to produce and verify the VRF output's proof, optimal for scenarios necessitating randomness and validation. Lastly, using NIST SP800-22 test suite for randomness, we demonstrate the commendable result with a 97.73% overall pass rate on 11 standard tests and 0.5459 of average p-value for the total 176 tests. △ Less

Submitted 30 January, 2024; originally announced January 2024.

Comments: 25 pages, 5 figures, 2023 International Journal on Cryptography and Information Security (IJCIS). arXiv admin note: text overlap with arXiv:2311.11734

Journal ref: Volume 13, Number 4, December 2023

Implicit Neural Representation for Physics-driven Actuated Soft Bodies

Authors: Lingchen Yang, Byungsoo Kim, Gaspard Zoss, Baran Gözcü, Markus Gross, Barbara Solenthaler

Abstract: Active soft bodies can affect their shape through an internal actuation mechanism that induces a deformation. Similar to recent work, this paper utilizes a differentiable, quasi-static, and physics-based simulation layer to optimize for actuation signals parameterized by neural networks. Our key contribution is a general and implicit formulation to control active soft bodies by defining a function… ▽ More Active soft bodies can affect their shape through an internal actuation mechanism that induces a deformation. Similar to recent work, this paper utilizes a differentiable, quasi-static, and physics-based simulation layer to optimize for actuation signals parameterized by neural networks. Our key contribution is a general and implicit formulation to control active soft bodies by defining a function that enables a continuous mapping from a spatial point in the material space to the actuation value. This property allows us to capture the signal's dominant frequencies, making the method discretization agnostic and widely applicable. We extend our implicit model to mandible kinematics for the particular case of facial animation and show that we can reliably reproduce facial expressions captured with high-quality capture systems. We apply the method to volumetric soft bodies, human poses, and facial expressions, demonstrating artist-friendly properties, such as simple control over the latent space and resolution invariance at test time. △ Less

Submitted 26 January, 2024; originally announced January 2024.

Comments: Accepted to SIGGRAPH 2022. Project page: https://studios.disneyresearch.com/2022/07/24/implicit-neural-representation-for-physics-driven-actuated-soft-bodies/ Video: https://www.youtube.com/watch?v=9EERe_CTazk

Intelli-Z: Toward Intelligible Zero-Shot TTS

Authors: Sunghee Jung, Won Jang, Jaesam Yoon, Bongwan Kim

Abstract: Although numerous recent studies have suggested new frameworks for zero-shot TTS using large-scale, real-world data, studies that focus on the intelligibility of zero-shot TTS are relatively scarce. Zero-shot TTS demands additional efforts to ensure clear pronunciation and speech quality due to its inherent requirement of replacing a core parameter (speaker embedding or acoustic prompt) with a new… ▽ More Although numerous recent studies have suggested new frameworks for zero-shot TTS using large-scale, real-world data, studies that focus on the intelligibility of zero-shot TTS are relatively scarce. Zero-shot TTS demands additional efforts to ensure clear pronunciation and speech quality due to its inherent requirement of replacing a core parameter (speaker embedding or acoustic prompt) with a new one at the inference stage. In this study, we propose a zero-shot TTS model focused on intelligibility, which we refer to as Intelli-Z. Intelli-Z learns speaker embeddings by using multi-speaker TTS as its teacher and is trained with a cycle-consistency loss to include mismatched text-speech pairs for training. Additionally, it selectively aggregates speaker embeddings along the temporal dimension to minimize the interference of the text content of reference speech at the inference stage. We substantiate the effectiveness of the proposed methods with an ablation study. The Mean Opinion Score (MOS) increases by 9% for unseen speakers when the first two methods are applied, and it further improves by 16% when selective temporal aggregation is applied. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Twisted Lattice Gauge Theory: Membrane Operators, Three-loop Braiding and Topological Charge

Authors: Joe Huxford, Dung Xuan Nguyen, Yong Baek Kim

Abstract: 3+1 dimensional topological phases can support loop-like excitations in addition to point-like ones, allowing for non-trivial loop-loop and point-loop braiding statistics not permitted to point-like excitations alone. Furthermore, these loop-like excitations can be linked together, changing their properties. In particular, this can lead to distinct three-loop braiding, involving two loops undergoi… ▽ More 3+1 dimensional topological phases can support loop-like excitations in addition to point-like ones, allowing for non-trivial loop-loop and point-loop braiding statistics not permitted to point-like excitations alone. Furthermore, these loop-like excitations can be linked together, changing their properties. In particular, this can lead to distinct three-loop braiding, involving two loops undergoing an exchange process while linked to a third loop. In this work, we investigate the loop-like excitations in a 3+1d Hamiltonian realization of Dijkgraaf-Witten theory through direct construction of their membrane operators, for a general finite Abelian group and 4-cocycle twist. Using these membrane operators, we find the braiding relations and fusion rules for the loop-like excitations, including those linked to another loop-like excitation. Furthermore, we use these membrane operators to construct projection operators that measure the topological charge and show that the number of distinct topological charges measured by the 2-torus matches the ground state degeneracy of the model on the 3-torus, explicitly confirming a general expectation for topological phases. This direct construction of the membrane operators sheds significant light on the key properties of the loop-like excitations in 3+1 dimensional topological phases. △ Less

Submitted 25 January, 2024; v1 submitted 23 January, 2024; originally announced January 2024.

Comments: 35 pages + 29 pages supplemental material, 37 figures, v2: arXiv metadata corrected

Few-Shot Learning for Chronic Disease Management: Leveraging Large Language Models and Multi-Prompt Engineering with Medical Knowledge Injection

Authors: Haoxin Liu, Wenli Zhang, Jiaheng Xie, Buomsoo Kim, Zhu Zhang, Yidong Chai

Abstract: This study harnesses state-of-the-art AI technology for chronic disease management, specifically in detecting various mental disorders through user-generated textual content. Existing studies typically rely on fully supervised machine learning, which presents challenges such as the labor-intensive manual process of annotating extensive training data for each disease and the need to design speciali… ▽ More This study harnesses state-of-the-art AI technology for chronic disease management, specifically in detecting various mental disorders through user-generated textual content. Existing studies typically rely on fully supervised machine learning, which presents challenges such as the labor-intensive manual process of annotating extensive training data for each disease and the need to design specialized deep learning architectures for each problem. To address such challenges, we propose a novel framework that leverages advanced AI techniques, including large language models and multi-prompt engineering. Specifically, we address two key technical challenges in data-driven chronic disease management: (1) developing personalized prompts to represent each user's uniqueness and (2) incorporating medical knowledge into prompts to provide context for chronic disease detection, instruct learning objectives, and operationalize prediction goals. We evaluate our method using four mental disorders, which are prevalent chronic diseases worldwide, as research cases. On the depression detection task, our method (F1 = 0.975~0.978) significantly outperforms traditional supervised learning paradigms, including feature engineering (F1 = 0.760) and architecture engineering (F1 = 0.756). Meanwhile, our approach demonstrates success in few-shot learning, i.e., requiring only a minimal number of training examples to detect chronic diseases based on user-generated textual content (i.e., only 2, 10, or 100 subjects). Moreover, our method can be generalized to other mental disorder detection tasks, including anorexia, pathological gambling, and self-harm (F1 = 0.919~0.978). △ Less

Submitted 16 January, 2024; originally announced January 2024.

MSC Class: K.5 ACM Class: I.2.7; H.4.m

GALA: Generating Animatable Layered Assets from a Single Scan

Authors: Taeksoo Kim, Byungjun Kim, Shunsuke Saito, Hanbyul Joo

Abstract: We present GALA, a framework that takes as input a single-layer clothed 3D human mesh and decomposes it into complete multi-layered 3D assets. The outputs can then be combined with other assets to create novel clothed human avatars with any pose. Existing reconstruction approaches often treat clothed humans as a single-layer of geometry and overlook the inherent compositionality of humans with hai… ▽ More We present GALA, a framework that takes as input a single-layer clothed 3D human mesh and decomposes it into complete multi-layered 3D assets. The outputs can then be combined with other assets to create novel clothed human avatars with any pose. Existing reconstruction approaches often treat clothed humans as a single-layer of geometry and overlook the inherent compositionality of humans with hairstyles, clothing, and accessories, thereby limiting the utility of the meshes for downstream applications. Decomposing a single-layer mesh into separate layers is a challenging task because it requires the synthesis of plausible geometry and texture for the severely occluded regions. Moreover, even with successful decomposition, meshes are not normalized in terms of poses and body shapes, failing coherent composition with novel identities and poses. To address these challenges, we propose to leverage the general knowledge of a pretrained 2D diffusion model as geometry and appearance prior for humans and other assets. We first separate the input mesh using the 3D surface segmentation extracted from multi-view 2D segmentations. Then we synthesize the missing geometry of different layers in both posed and canonical spaces using a novel pose-guided Score Distillation Sampling (SDS) loss. Once we complete inpainting high-fidelity 3D geometry, we also apply the same SDS loss to its texture to obtain the complete appearance including the initially occluded regions. Through a series of decomposition steps, we obtain multiple layers of 3D assets in a shared canonical space normalized in terms of poses and human shapes, hence supporting effortless composition to novel identities and reanimation with novel poses. Our experiments demonstrate the effectiveness of our approach for decomposition, canonicalization, and composition tasks compared to existing solutions. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: The project page is available at https://snuvclab.github.io/gala/

Boosting Unknown-number Speaker Separation with Transformer Decoder-based Attractor

Authors: Younglo Lee, Shukjae Choi, Byeong-Yeol Kim, Zhong-Qiu Wang, Shinji Watanabe

Abstract: We propose a novel speech separation model designed to separate mixtures with an unknown number of speakers. The proposed model stacks 1) a dual-path processing block that can model spectro-temporal patterns, 2) a transformer decoder-based attractor (TDA) calculation module that can deal with an unknown number of speakers, and 3) triple-path processing blocks that can model inter-speaker relations… ▽ More We propose a novel speech separation model designed to separate mixtures with an unknown number of speakers. The proposed model stacks 1) a dual-path processing block that can model spectro-temporal patterns, 2) a transformer decoder-based attractor (TDA) calculation module that can deal with an unknown number of speakers, and 3) triple-path processing blocks that can model inter-speaker relations. Given a fixed, small set of learned speaker queries and the mixture embedding produced by the dual-path blocks, TDA infers the relations of these queries and generates an attractor vector for each speaker. The estimated attractors are then combined with the mixture embedding by feature-wise linear modulation conditioning, creating a speaker dimension. The mixture embedding, conditioned with speaker information produced by TDA, is fed to the final triple-path blocks, which augment the dual-path blocks with an additional pathway dedicated to inter-speaker processing. The proposed approach outperforms the previous best reported in the literature, achieving 24.0 and 23.7 dB SI-SDR improvement (SI-SDRi) on WSJ0-2 and 3mix respectively, with a single model trained to separate 2- and 3-speaker mixtures. The proposed model also exhibits strong performance and generalizability at counting sources and separating mixtures with up to 5 speakers. △ Less

Submitted 22 January, 2024; originally announced January 2024.

Comments: 5 pages, 4 figures, accepted by ICASSP 2024

Finite Temperature Dynamics in 0-Flux and $π$-Flux Quantum Spin Ice: Self-Consistent Exclusive Boson Approach

Authors: Félix Desrochers, Yong Baek Kim

Abstract: Quantum spin ice (QSI) is an emblematic three-dimensional $U(1)$ quantum spin liquid (QSL) on the pyrochlore lattice that hosts gapless photon-like modes and spinon excitations. Despite its notable status and the current rise of strong material candidates Ce$_2$(Zr, Sn, Hf)$_2$O$_7$, there are still only a few analytical approaches to model the low-energy behavior of QSI. These analytical methods… ▽ More Quantum spin ice (QSI) is an emblematic three-dimensional $U(1)$ quantum spin liquid (QSL) on the pyrochlore lattice that hosts gapless photon-like modes and spinon excitations. Despite its notable status and the current rise of strong material candidates Ce$_2$(Zr, Sn, Hf)$_2$O$_7$, there are still only a few analytical approaches to model the low-energy behavior of QSI. These analytical methods are essential to gain insight into the physical interpretation of measurements. We here introduce the self-consistent exclusive boson representation (SCEBR) to model emergent spinon excitations in QSI. By treating the presence of other emergent charges in an average way, the SCEBR extends the range of validity of the exclusive boson representation previously introduced by Hao, Day, and Gingras [Hao, Day, and Gingras, Physical Review B, 90, 214430 (2014)] to numerous cases of physical relevance. We extensively benchmark the approach and provide detailed analytical expressions for the spinon dispersion, the Bogoliubov transformation that diagonalizes the system, and the dynamical spin structure factor for 0- and $π$-flux QSI. Finite temperature properties are further investigated to highlight essential differences between the thermodynamic behavior of the 0- and $π$-flux phases. We notably show that the SCEBR predicts a reduction of the spinon bandwidth with increasing temperature, consistent with previous quantum Monte-Carlo results, through suppression of spinon hopping by thermal occupation. The SCEBR thus provides a powerful analytical tool to interpret experiments on current and future candidate material that has several advantages over other widely used methods. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Comments: 13+9 pages, 8 figures

Journal ref: Physical Review B, 109(14), 144410 (2024)

Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis

Authors: Jonghyun Lee, Hansam Cho, Youngjoon Yoo, Seoung Bum Kim, Yonghyun Jeong

Abstract: Addressing the limitations of text as a source of accurate layout representation in text-conditional diffusion models, many works incorporate additional signals to condition certain attributes within a generated image. Although successful, previous works do not account for the specific localization of said attributes extended into the three dimensional plane. In this context, we present a conditio… ▽ More Addressing the limitations of text as a source of accurate layout representation in text-conditional diffusion models, many works incorporate additional signals to condition certain attributes within a generated image. Although successful, previous works do not account for the specific localization of said attributes extended into the three dimensional plane. In this context, we present a conditional diffusion model that integrates control over three-dimensional object placement with disentangled representations of global stylistic semantics from multiple exemplar images. Specifically, we first introduce \textit{depth disentanglement training} to leverage the relative depth of objects as an estimator, allowing the model to identify the absolute positions of unseen objects through the use of synthetic image triplets. We also introduce \textit{soft guidance}, a method for imposing global semantics onto targeted regions without the use of any additional localization cues. Our integrated framework, \textsc{Compose and Conquer (CnC)}, unifies these techniques to localize multiple conditions in a disentangled manner. We demonstrate that our approach allows perception of objects at varying depths while offering a versatile framework for composing localized objects with different global semantics. Code: https://github.com/tomtom1103/compose-and-conquer/ △ Less

Submitted 17 January, 2024; originally announced January 2024.

Comments: ICLR 2024

Error correctable efficient quantum homomorphic encryption using Calderbank-Shor-Steane codes

Authors: IlKwon Sohn, Boseon Kim, Kwangil Bae, Wonhyuk Lee

Abstract: To achieve fault-tolerant and secure cloud quantum computing, integrating quantum error correction codes and quantum homomorphic encryption schemes is essential. However, significant overhead challenges incurred in these schemes necessitate their efficiency. This study developed an efficient quantum homomorphic encryption based on quantum error correction codes. The proposed encryption scheme used… ▽ More To achieve fault-tolerant and secure cloud quantum computing, integrating quantum error correction codes and quantum homomorphic encryption schemes is essential. However, significant overhead challenges incurred in these schemes necessitate their efficiency. This study developed an efficient quantum homomorphic encryption based on quantum error correction codes. The proposed encryption scheme used a single encoding process to accomplish encryption and encoding. Using a longer quantum error-correcting code simultaneously improved the security and error correction capabilities. Based on the permutation key, the number of combinations exhibited an exponential growth of $n^n$ compared with the conventional order of $2^n$. △ Less

Submitted 27 January, 2024; v1 submitted 15 January, 2024; originally announced January 2024.

Comments: 6 pages, 5 figures

Background study of the AMoRE-pilot experiment

Authors: A. Agrawal, V. V. Alenkov, P. Aryal, J. Beyer, B. Bhandari, R. S. Boiko, K. Boonin, O. Buzanov, C. R. Byeon, N. Chanthima, M. K. Cheoun, J. S. Choe, Seonho Choi, S. Choudhury, J. S. Chung, F. A. Danevich, M. Djamal, D. Drung, C. Enss, A. Fleischmann, A. M. Gangapshev, L. Gastaldo, Yu. M. Gavrilyuk, A. M. Gezhaev, O. Gileva , et al. (83 additional authors not shown)

Abstract: We report a study on the background of the Advanced Molybdenum-Based Rare process Experiment (AMoRE), a search for neutrinoless double beta decay (\znbb) of $^{100}$Mo. The pilot stage of the experiment was conducted using $\sim$1.9 kg of \CAMOO~ crystals at the Yangyang Underground Laboratory, South Korea, from 2015 to 2018. We compared the measured $β/γ$ energy spectra in three experimental conf… ▽ More We report a study on the background of the Advanced Molybdenum-Based Rare process Experiment (AMoRE), a search for neutrinoless double beta decay (\znbb) of $^{100}$Mo. The pilot stage of the experiment was conducted using $\sim$1.9 kg of \CAMOO~ crystals at the Yangyang Underground Laboratory, South Korea, from 2015 to 2018. We compared the measured $β/γ$ energy spectra in three experimental configurations with the results of Monte Carlo simulations and identified the background sources in each configuration. We replaced several detector components and enhanced the neutron shielding to lower the background level between configurations. A limit on the half-life of $0νββ$ decay of $^{100}$Mo was found at $T_{1/2}^{0ν} \ge 3.0\times 10^{23}$ years at 90\% confidence level, based on the measured background and its modeling. Further reduction of the background rate in the AMoRE-I and AMoRE-II are discussed. △ Less

Submitted 7 April, 2024; v1 submitted 15 January, 2024; originally announced January 2024.

Nonproportionality of NaI(Tl) Scintillation Detector for Dark Matter Search Experiments

Authors: S. M. Lee, G. Adhikari, N. Carlin, J. Y. Cho, J. J. Choi, S. Choi, A. C. Ezeribe, L. E. Fran. a, C. Ha, I. S. Hahn, S. J. Hollick, E. J. Jeon, H. W. Joo, W. G. Kang, M. Kauer, B. H. Kim, H. J. Kim, J. Kim, K. W. Kim, S. H. Kim, S. K. Kim, S. W. Kim, W. K. Kim, Y. D. Kim, Y. H. Kim , et al. (37 additional authors not shown)

Abstract: We present a comprehensive study of the nonproportionality of NaI(Tl) scintillation detectors within the context of dark matter search experiments. Our investigation, which integrates COSINE-100 data with supplementary $γ$ spectroscopy, measures light yields across diverse energy levels from full-energy $γ$ peaks produced by the decays of various isotopes. These $γ$ peaks of interest were produced… ▽ More We present a comprehensive study of the nonproportionality of NaI(Tl) scintillation detectors within the context of dark matter search experiments. Our investigation, which integrates COSINE-100 data with supplementary $γ$ spectroscopy, measures light yields across diverse energy levels from full-energy $γ$ peaks produced by the decays of various isotopes. These $γ$ peaks of interest were produced by decays supported by both long and short-lived isotopes. Analyzing peaks from decays supported only by short-lived isotopes presented a unique challenge due to their limited statistics and overlapping energies, which was overcome by long-term data collection and a time-dependent analysis. A key achievement is the direct measurement of the 0.87 keV light yield, resulting from the cascade following electron capture decay of $^{22}$Na from internal contamination. This measurement, previously accessible only indirectly, deepens our understanding of NaI(Tl) scintillator behavior in the region of interest for dark matter searches. This study holds substantial implications for background modeling and the interpretation of dark matter signals in NaI(Tl) experiments. △ Less

Submitted 10 May, 2024; v1 submitted 14 January, 2024; originally announced January 2024.

Comments: 12 pages, 7 figures

Journal ref: Eur. Phys. J. C 84 (2024) 484

Universal Time-Series Representation Learning: A Survey

Authors: Patara Trirat, Yooju Shin, Junhyeok Kang, Youngeun Nam, Jihye Na, Minyoung Bae, Joeun Kim, Byunghyun Kim, Jae-Gil Lee

Abstract: Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform… ▽ More Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time series is crucial for understanding the complex dynamics of particular phenomena and enabling informed decisions. With the learned representations, we can perform numerous downstream analyses more effectively. Among several approaches, deep learning has demonstrated remarkable performance in extracting hidden patterns and features from time-series data without manual feature engineering. This survey first presents a novel taxonomy based on three fundamental elements in designing state-of-the-art universal representation learning methods for time series. According to the proposed taxonomy, we comprehensively review existing studies and discuss their intuitions and insights into how these methods enhance the quality of learned representations. Finally, as a guideline for future studies, we summarize commonly used experimental setups and datasets and discuss several promising research directions. An up-to-date corresponding resource is available at https://github.com/itouchz/awesome-deep-time-series-representations. △ Less

Submitted 27 August, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Comments: 41 pages, 7 figures

Discovery of a topological exciton insulator with tunable momentum order

Authors: Md Shafayat Hossain, Tyler A. Cochran, Yu-Xiao Jiang, Songbo Zhang, Huangyu Wu, Xiaoxiong Liu, Xiquan Zheng, Byunghoon Kim, Guangming Cheng, Qi Zhang, Maksim Litskevich, Junyi Zhang, Zi-Jia Cheng, Jinjin Liu, Jia-Xin Yin, Xian P. Yang, Jonathan Denlinger, Massimo Tallarida, Ji Dai, Elio Vescovo, Anil Rajapitamahuni, Hu Miao, Nan Yao, Yingying Peng, Yugui Yao , et al. (4 additional authors not shown)

Abstract: Topology and correlations are fundamental concepts in modern physics, but their simultaneous occurrence within a single quantum phase is exceptionally rare. In this study, we present the discovery of such a phase of matter in Ta2Pd3Te5, a semimetal where the Coulomb interaction between electrons and holes leads to the spontaneous formation of excitonic bound states below T=100 K. Our spectroscopy… ▽ More Topology and correlations are fundamental concepts in modern physics, but their simultaneous occurrence within a single quantum phase is exceptionally rare. In this study, we present the discovery of such a phase of matter in Ta2Pd3Te5, a semimetal where the Coulomb interaction between electrons and holes leads to the spontaneous formation of excitonic bound states below T=100 K. Our spectroscopy unveils the development of an insulating gap stemming from the condensation of these excitons, thus giving rise to a highly sought-after correlated quantum phase known as the excitonic insulator. Remarkably, our scanning tunneling microscopy measurements reveal the presence of gapless boundary modes in the excitonic insulator state. Their magnetic field response and our theoretical calculations suggest a topological origin of these modes, rendering Ta2Pd3Te5 as the first experimentally identified topological excitonic insulator in a three-dimensional material not masked by any structural phase transition. Furthermore, our study uncovers a secondary excitonic instability below T=5 K, which differs from the primary one in having finite momentum. We observe unprecedented tunability of its wavevector by an external magnetic field. These findings unlock a frontier in the study of novel correlated topological phases of matter and their tunability. △ Less

Submitted 25 December, 2023; originally announced December 2023.

Report number: Journal submission on 7th December 23

Large-scale Graph Representation Learning of Dynamic Brain Connectome with Transformers

Authors: Byung-Hoon Kim, Jungwon Choi, EungGu Yun, Kyungsang Kim, Xiang Li, Juho Lee

Abstract: Graph Transformers have recently been successful in various graph representation learning tasks, providing a number of advantages over message-passing Graph Neural Networks. Utilizing Graph Transformers for learning the representation of the brain functional connectivity network is also gaining interest. However, studies to date have underlooked the temporal dynamics of functional connectivity, wh… ▽ More Graph Transformers have recently been successful in various graph representation learning tasks, providing a number of advantages over message-passing Graph Neural Networks. Utilizing Graph Transformers for learning the representation of the brain functional connectivity network is also gaining interest. However, studies to date have underlooked the temporal dynamics of functional connectivity, which fluctuates over time. Here, we propose a method for learning the representation of dynamic functional connectivity with Graph Transformers. Specifically, we define the connectome embedding, which holds the position, structure, and time information of the functional connectivity graph, and use Transformers to learn its representation across time. We perform experiments with over 50,000 resting-state fMRI samples obtained from three datasets, which is the largest number of fMRI data used in studies by far. The experimental results show that our proposed method outperforms other competitive baselines in gender classification and age regression tasks based on the functional connectivity extracted from the fMRI data. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Comments: NeurIPS 2023 Temporal Graph Learning Workshop

Solar neutrino measurements using the full data period of Super-Kamiokande-IV

Authors: Super-Kamiokande Collaboration, :, K. Abe, C. Bronner, Y. Hayato, K. Hiraide, K. Hosokawa, K. Ieki, M. Ikeda, S. Imaizumi, K. Iyogi, J. Kameda, Y. Kanemura, R. Kaneshima, Y. Kashiwagi, Y. Kataoka, Y. Kato, Y. Kishimoto, S. Miki, S. Mine, M. Miura, T. Mochizuki, S. Moriyama, Y. Nagao, M. Nakahata , et al. (305 additional authors not shown)

Abstract: An analysis of solar neutrino data from the fourth phase of Super-Kamiokande~(SK-IV) from October 2008 to May 2018 is performed and the results are presented. The observation time of the data set of SK-IV corresponds to $2970$~days and the total live time for all four phases is $5805$~days. For more precise solar neutrino measurements, several improvements are applied in this analysis: lowering th… ▽ More An analysis of solar neutrino data from the fourth phase of Super-Kamiokande~(SK-IV) from October 2008 to May 2018 is performed and the results are presented. The observation time of the data set of SK-IV corresponds to $2970$~days and the total live time for all four phases is $5805$~days. For more precise solar neutrino measurements, several improvements are applied in this analysis: lowering the data acquisition threshold in May 2015, further reduction of the spallation background using neutron clustering events, precise energy reconstruction considering the time variation of the PMT gain. The observed number of solar neutrino events in $3.49$--$19.49$ MeV electron kinetic energy region during SK-IV is $65,443^{+390}_{-388}\,(\mathrm{stat.})\pm 925\,(\mathrm{syst.})$ events. Corresponding $\mathrm{^{8}B}$ solar neutrino flux is $(2.314 \pm 0.014\, \rm{(stat.)} \pm 0.040 \, \rm{(syst.)}) \times 10^{6}~\mathrm{cm^{-2}\,s^{-1}}$, assuming a pure electron-neutrino flavor component without neutrino oscillations. The flux combined with all SK phases up to SK-IV is $(2.336 \pm 0.011\, \rm{(stat.)} \pm 0.043 \, \rm{(syst.)}) \times 10^{6}~\mathrm{cm^{-2}\,s^{-1}}$. Based on the neutrino oscillation analysis from all solar experiments, including the SK $5805$~days data set, the best-fit neutrino oscillation parameters are $\rm{sin^{2} θ_{12,\,solar}} = 0.306 \pm 0.013 $ and $Δm^{2}_{21,\,\mathrm{solar}} = (6.10^{+ 0.95}_{-0.81}) \times 10^{-5}~\rm{eV}^{2}$, with a deviation of about 1.5$σ$ from the $Δm^{2}_{21}$ parameter obtained by KamLAND. The best-fit neutrino oscillation parameters obtained from all solar experiments and KamLAND are $\sin^{2} θ_{12,\,\mathrm{global}} = 0.307 \pm 0.012 $ and $Δm^{2}_{21,\,\mathrm{global}} = (7.50^{+ 0.19}_{-0.18}) \times 10^{-5}~\rm{eV}^{2}$. △ Less

Submitted 20 February, 2024; v1 submitted 20 December, 2023; originally announced December 2023.

Comments: 47 pages, 61 figures

Journal ref: Phys. Rev. D 109, 092001 (2024)

UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection

Authors: Bumsoo Kim, Taeho Choi, Jaewoo Kang, Hyunwoo J. Kim

Abstract: Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where… ▽ More Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where the objects are first detected and the interactions are predicted sequentially by pairing the objects. This is a major bottleneck in HOI detection inference time. To tackle this problem, we propose UnionDet, a one-stage meta-architecture for HOI detection powered by a novel union-level detector that eliminates this additional inference stage by directly capturing the region of interaction. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time 4x~14x while outperforming state-of-the-art methods on two public datasets: V-COCO and HICO-DET. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: ECCV 2020

Misalign, Contrast then Distill: Rethinking Misalignments in Language-Image Pretraining

Authors: Bumsoo Kim, Yeonsik Jo, Jinhyung Kim, Seung Hwan Kim

Abstract: Contrastive Language-Image Pretraining has emerged as a prominent approach for training vision and text encoders with uncurated image-text pairs from the web. To enhance data-efficiency, recent efforts have introduced additional supervision terms that involve random-augmented views of the image. However, since the image augmentation process is unaware of its text counterpart, this procedure could… ▽ More Contrastive Language-Image Pretraining has emerged as a prominent approach for training vision and text encoders with uncurated image-text pairs from the web. To enhance data-efficiency, recent efforts have introduced additional supervision terms that involve random-augmented views of the image. However, since the image augmentation process is unaware of its text counterpart, this procedure could cause various degrees of image-text misalignments during training. Prior methods either disregarded this discrepancy or introduced external models to mitigate the impact of misalignments during training. In contrast, we propose a novel metric learning approach that capitalizes on these misalignments as an additional training source, which we term "Misalign, Contrast then Distill (MCD)". Unlike previous methods that treat augmented images and their text counterparts as simple positive pairs, MCD predicts the continuous scales of misalignment caused by the augmentation. Our extensive experimental results show that our proposed MCD achieves state-of-the-art transferability in multiple classification and retrieval downstream datasets. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: ICCV 2023

Expediting Contrastive Language-Image Pretraining via Self-distilled Encoders

Authors: Bumsoo Kim, Jinhyung Kim, Yeonsik Jo, Seung Hwan Kim

Abstract: Recent advances in vision language pretraining (VLP) have been largely attributed to the large-scale data collected from the web. However, uncurated dataset contains weakly correlated image-text pairs, causing data inefficiency. To address the issue, knowledge distillation have been explored at the expense of extra image and text momentum encoders to generate teaching signals for misaligned image-… ▽ More Recent advances in vision language pretraining (VLP) have been largely attributed to the large-scale data collected from the web. However, uncurated dataset contains weakly correlated image-text pairs, causing data inefficiency. To address the issue, knowledge distillation have been explored at the expense of extra image and text momentum encoders to generate teaching signals for misaligned image-text pairs. In this paper, our goal is to resolve the misalignment problem with an efficient distillation framework. To this end, we propose ECLIPSE: Expediting Contrastive Language-Image Pretraining with Self-distilled Encoders. ECLIPSE features a distinctive distillation architecture wherein a shared text encoder is utilized between an online image encoder and a momentum image encoder. This strategic design choice enables the distillation to operate within a unified projected space of text embedding, resulting in better performance. Based on the unified text embedding space, ECLIPSE compensates for the additional computational cost of the momentum image encoder by expediting the online image encoder. Through our extensive experiments, we validate that there is a sweet spot between expedition and distillation where the partial view from the expedited online image encoder interacts complementarily with the momentum teacher. As a result, ECLIPSE outperforms its counterparts while achieving substantial acceleration in inference speed. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: AAAI 2024

Origin of chirality in transition-metal dichalcogenides

Authors: Kwangrae Kim, Hyun-Woo J. Kim, Seunghyeok Ha, Hoon Kim, Jin-Kwang Kim, Jaehwon Kim, Hyunsung Kim, Junyoung Kwon, Jihoon Seol, Saegyeol Jung, Changyoung Kim, Ahmet Alatas, Ayman Said, Michael Merz, Matthieu Le Tacon, Jin Mo Bok, Ki-Seok Kim, B. J. Kim

Abstract: Chirality is a ubiquitous phenomenon in which a symmetry between left- and right-handed objects is broken, examples in nature ranging from subatomic particles and molecules to living organisms. In particle physics, the weak force is responsible for the symmetry breaking and parity violation in beta decay, but in condensed matter systems interactions that lead to chirality remain poorly understood.… ▽ More Chirality is a ubiquitous phenomenon in which a symmetry between left- and right-handed objects is broken, examples in nature ranging from subatomic particles and molecules to living organisms. In particle physics, the weak force is responsible for the symmetry breaking and parity violation in beta decay, but in condensed matter systems interactions that lead to chirality remain poorly understood. Here, we unravel the mechanism of chiral charge density wave formation in the transition-metal dichalcogenide 1T-TiSe2. Using representation analysis, we show that charge density modulations and ionic displacements, which transform as a continuous scalar field and a vector field on a discrete lattice, respectively, follow different irreducible representations of the space group, despite the fact that they propagate with the same wave-vectors and are strongly coupled to each other. This charge-lattice symmetry frustration is resolved by further breaking of all symmetries not common to both sectors through induced lattice distortions, thus leading to chirality. Our theory is verified using Raman spectroscopy and inelastic x-ray scattering, which reveal that all but translation symmetries are broken at a level not resolved by state-of-the-art diffraction techniques. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 10 pages, 3 figures, 1 table

TRACTOR: Traffic Analysis and Classification Tool for Open RAN

Authors: Joshua Groen, Mauro Belgiovine, Utku Demir, Brian Kim, Kaushik Chowdhury

Abstract: 5G and beyond cellular networks promise remarkable advancements in bandwidth, latency, and connectivity. The emergence of Open Radio Access Network (O-RAN) represents a pivotal direction for the evolution of cellular networks, inherently supporting machine learning (ML) for network operation control. Within this framework, RAN Intelligence Controllers (RICs) from one provider can employ ML models… ▽ More 5G and beyond cellular networks promise remarkable advancements in bandwidth, latency, and connectivity. The emergence of Open Radio Access Network (O-RAN) represents a pivotal direction for the evolution of cellular networks, inherently supporting machine learning (ML) for network operation control. Within this framework, RAN Intelligence Controllers (RICs) from one provider can employ ML models developed by third-party vendors through the acquisition of key performance indicators (KPIs) from geographically distant base stations or user equipment (UE). Yet, the development of ML models hinges on the availability of realistic and robust datasets. In this study, we embark on a two-fold journey. First, we collect a comprehensive 5G dataset, harnessing real-world cell phones across diverse applications, locations, and mobility scenarios. Next, we replicate this traffic within a full-stack srsRAN-based O-RAN framework on Colosseum, the world's largest radio frequency (RF) emulator. This process yields a robust and O-RAN compliant KPI dataset mirroring real-world conditions. We illustrate how such a dataset can fuel the training of ML models and facilitate the deployment of xApps for traffic slice classification by introducing a CNN based classifier that achieves accuracy $>95\%$ offline and $92\%$ online. To accelerate research in this domain, we provide open-source access to our toolchain and supplementary utilities, empowering the broader research community to expedite the creation of realistic and O-RAN compliant datasets. △ Less

Submitted 12 December, 2023; originally announced December 2023.

Comments: 6 pages, 5 figures, 2 tables, submitted to ICC 2024