Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Showing 1–50 of 68 results for author: Yuan, R

Searching in archive cs. Search in all archives.
.
  1. arXiv:2406.04321  [pdf, other

    cs.CV cs.LG cs.MM cs.SD

    VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling

    Authors: Zeyue Tian, Zhaoyang Liu, Ruibin Yuan, Jiahao Pan, Xiaoqiang Huang, Qifeng Liu, Xu Tan, Qifeng Chen, Wei Xue, Yike Guo

    Abstract: In this work, we systematically study music generation conditioned solely on the video. First, we present a large-scale dataset comprising 190K video-music pairs, including various genres such as movie trailers, advertisements, and documentaries. Furthermore, we propose VidMuse, a simple framework for generating music aligned with video inputs. VidMuse stands out by producing high-fidelity music t… ▽ More

    Submitted 6 June, 2024; originally announced June 2024.

    Comments: The code and datasets will be available at https://github.com/ZeyueT/VidMuse/

  2. arXiv:2406.00507  [pdf, other

    cs.CL cs.AI

    Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization

    Authors: Shichao Sun, Ruifeng Yuan, Ziqiang Cao, Wenjie Li, Pengfei Liu

    Abstract: Large language models (LLMs) have demonstrated the capacity to improve summary quality by mirroring a human-like iterative process of critique and refinement starting from the initial draft. Two strategies are designed to perform this iterative process: Prompt Chaining and Stepwise Prompt. Prompt chaining orchestrates the drafting, critiquing, and refining phases through a series of three discrete… ▽ More

    Submitted 1 June, 2024; originally announced June 2024.

    Comments: Accepted to Findings of ACL 2024

  3. arXiv:2405.19334  [pdf, other

    cs.AI cs.CL cs.CV cs.MM cs.SD

    LLMs Meet Multimodal Generation and Editing: A Survey

    Authors: Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Runtao Liu, Ruibin Yuan, Yazhou Xing, Wenhai Wang, Jifeng Dai, Yong Zhang, Wei Xue, Qifeng Liu, Yike Guo, Qifeng Chen

    Abstract: With the recent advancement in large language models (LLMs), there is a growing interest in combining LLMs with multimodal learning. Previous surveys of multimodal large language models (MLLMs) mainly focus on multimodal understanding. This survey elaborates on multimodal generation and editing across various domains, comprising image, video, 3D, and audio. Specifically, we summarize the notable a… ▽ More

    Submitted 9 June, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

    Comments: 52 Pages with 16 Figures, 12 Tables, and 545 References. GitHub Repository at: https://github.com/YingqingHe/Awesome-LLMs-meet-Multimodal-Generation

  4. arXiv:2405.19327  [pdf, other

    cs.CL cs.AI cs.LG

    MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

    Authors: Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yinghao Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Benetos, Huan Yang, Junting Zhou, Kaijing Ma, Minghao Liu, Morry Niu , et al. (20 additional authors not shown)

    Abstract: Large Language Models (LLMs) have made great strides in recent years to achieve unprecedented performance across different tasks. However, due to commercial interest, the most competitive models like GPT, Gemini, and Claude have been gated behind proprietary interfaces without disclosing the training details. Recently, many institutions have open-sourced several strong LLMs like LLaMA-3, comparabl… ▽ More

    Submitted 2 June, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

    Comments: https://map-neo.github.io/

  5. arXiv:2404.18081  [pdf, other

    cs.SD cs.AI cs.CL cs.LG cs.MM eess.AS

    ComposerX: Multi-Agent Symbolic Music Composition with LLMs

    Authors: Qixin Deng, Qikai Yang, Ruibin Yuan, Yipeng Huang, Yi Wang, Xubo Liu, Zeyue Tian, Jiahao Pan, Ge Zhang, Hanfeng Lin, Yizhi Li, Yinghao Ma, Jie Fu, Chenghua Lin, Emmanouil Benetos, Wenwu Wang, Guangyu Xia, Wei Xue, Yike Guo

    Abstract: Music composition represents the creative side of humanity, and itself is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints. While demonstrating impressive capabilities in STEM subjects, current LLMs easily fail in this task, generating ill-written music even when equipped with modern techniques like In-Context-Learning and C… ▽ More

    Submitted 30 April, 2024; v1 submitted 28 April, 2024; originally announced April 2024.

  6. arXiv:2404.07525  [pdf, other

    cs.LG

    Enhancing Policy Gradient with the Polyak Step-Size Adaption

    Authors: Yunxiang Li, Rui Yuan, Chen Fan, Mark Schmidt, Samuel Horváth, Robert M. Gower, Martin Takáč

    Abstract: Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other RL algorithms, its practical application is often hindered by sensitivity to hyper-parameters, particularly the step-size. In this paper, we introduce the integration of the Polyak step-size in RL, which automatically a… ▽ More

    Submitted 11 April, 2024; originally announced April 2024.

  7. arXiv:2404.06393  [pdf, other

    cs.SD cs.AI eess.AS

    MuPT: A Generative Symbolic Music Pretrained Transformer

    Authors: Xingwei Qu, Yuelin Bai, Yinghao Ma, Ziya Zhou, Ka Man Lo, Jiaheng Liu, Ruibin Yuan, Lejun Min, Xueling Liu, Tianyu Zhang, Xinrun Du, Shuyue Guo, Yiming Liang, Yizhi Li, Shangda Wu, Junting Zhou, Tianyu Zheng, Ziyang Ma, Fengze Han, Wei Xue, Gus Xia, Emmanouil Benetos, Xiang Yue, Chenghua Lin, Xu Tan , et al. (4 additional authors not shown)

    Abstract: In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music. While the prevalent use of MIDI in music modeling is well-established, our findings suggest that LLMs are inherently more compatible with ABC Notation, which aligns more closely with their design and strengths, thereby enhancing the model's performance in musical composition. To address the chal… ▽ More

    Submitted 10 April, 2024; v1 submitted 9 April, 2024; originally announced April 2024.

  8. arXiv:2404.04167  [pdf, other

    cs.CL cs.AI

    Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model

    Authors: Xinrun Du, Zhouliang Yu, Songyang Gao, Ding Pan, Yuyang Cheng, Ziyang Ma, Ruibin Yuan, Xingwei Qu, Jiaheng Liu, Tianyu Zheng, Xinchen Luo, Guorui Zhou, Binhang Yuan, Wenhu Chen, Jie Fu, Ge Zhang

    Abstract: In this study, we introduce CT-LLM, a 2B large language model (LLM) that illustrates a pivotal shift towards prioritizing the Chinese language in developing LLMs. Uniquely initiated from scratch, CT-LLM diverges from the conventional methodology by primarily incorporating Chinese textual data, utilizing an extensive corpus of 1,200 billion tokens, including 800 billion Chinese tokens, 300 billion… ▽ More

    Submitted 9 April, 2024; v1 submitted 5 April, 2024; originally announced April 2024.

  9. arXiv:2404.01204  [pdf, other

    cs.CL

    The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis

    Authors: Chen Yang, Junzhuo Li, Xinyao Niu, Xinrun Du, Songyang Gao, Haoran Zhang, Zhaoliang Chen, Xingwei Qu, Ruibin Yuan, Yizhi Li, Jiaheng Liu, Stephen W. Huang, Shawn Yue, Wenhu Chen, Jie Fu, Ge Zhang

    Abstract: Uncovering early-stage metrics that reflect final model performance is one core principle for large-scale pretraining. The existing scaling law demonstrates the power-law correlation between pretraining loss and training flops, which serves as an important indicator of the current training state for large language models. However, this principle only focuses on the model's compression properties o… ▽ More

    Submitted 1 April, 2024; originally announced April 2024.

  10. arXiv:2404.00610  [pdf, other

    cs.CL

    RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation

    Authors: Chi-Min Chan, Chunpu Xu, Ruibin Yuan, Hongyin Luo, Wei Xue, Yike Guo, Jie Fu

    Abstract: Large Language Models (LLMs) exhibit remarkable capabilities but are prone to generating inaccurate or hallucinatory responses. This limitation stems from their reliance on vast pretraining datasets, making them susceptible to errors in unseen scenarios. To tackle these challenges, Retrieval-Augmented Generation (RAG) addresses this by incorporating external, relevant documents into the response g… ▽ More

    Submitted 31 March, 2024; originally announced April 2024.

  11. arXiv:2403.18058  [pdf, other

    cs.CL cs.AI

    COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

    Authors: Yuelin Bai, Xinrun Du, Yiming Liang, Yonggang Jin, Ziqiang Liu, Junting Zhou, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Wenhu Chen, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

    Abstract: Recently, there have been significant advancements in large language models (LLMs), particularly focused on the English language. These advancements have enabled these LLMs to understand and execute complex instructions with unprecedented accuracy and fluency. However, despite these advancements, there remains a noticeable gap in the development of Chinese instruction tuning. The unique linguistic… ▽ More

    Submitted 26 March, 2024; originally announced March 2024.

  12. arXiv:2403.16331  [pdf, other

    cs.SD cs.LG eess.AS

    Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models

    Authors: Hanzhi Yin, Gang Cheng, Christian J. Steinmetz, Ruibin Yuan, Richard M. Stern, Roger B. Dannenberg

    Abstract: We describe a novel approach for developing realistic digital models of dynamic range compressors for digital audio production by analyzing their analog prototypes. While realistic digital dynamic compressors are potentially useful for many applications, the design process is challenging because the compressors operate nonlinearly over long time scales. Our approach is based on the structured stat… ▽ More

    Submitted 24 March, 2024; originally announced March 2024.

  13. arXiv:2403.11953  [pdf, other

    eess.IV cs.CV

    Advancing COVID-19 Detection in 3D CT Scans

    Authors: Qingqiu Li, Runtian Yuan, Junlin Hou, Jilan Xu, Yuejie Zhang, Rui Feng, Hao Chen

    Abstract: To make a more accurate diagnosis of COVID-19, we propose a straightforward yet effective model. Firstly, we analyse the characteristics of 3D CT scans and remove the non-lung parts, facilitating the model to focus on lesion-related areas and reducing computational cost. We use ResNeSt50 as the strong feature extractor, initializing it with pretrained weights which have COVID-19-specific prior kno… ▽ More

    Submitted 18 March, 2024; originally announced March 2024.

  14. arXiv:2403.11498  [pdf, other

    eess.IV cs.CV

    Domain Adaptation Using Pseudo Labels for COVID-19 Detection

    Authors: Runtian Yuan, Qingqiu Li, Junlin Hou, Jilan Xu, Yuejie Zhang, Rui Feng, Hao Chen

    Abstract: In response to the need for rapid and accurate COVID-19 diagnosis during the global pandemic, we present a two-stage framework that leverages pseudo labels for domain adaptation to enhance the detection of COVID-19 from CT scans. By utilizing annotated data from one domain and non-annotated data from another, the model overcomes the challenge of data scarcity and variability, common in emergent he… ▽ More

    Submitted 18 March, 2024; originally announced March 2024.

  15. arXiv:2403.09294  [pdf, other

    cs.CV cs.CL

    Anatomical Structure-Guided Medical Vision-Language Pre-training

    Authors: Qingqiu Li, Xiaohan Yan, Jilan Xu, Runtian Yuan, Yuejie Zhang, Rui Feng, Quanli Shen, Xiaobo Zhang, Shujun Wang

    Abstract: Learning medical visual representations through vision-language pre-training has reached remarkable progress. Despite the promising performance, it still faces challenges, i.e., local alignment lacks interpretability and clinical relevance, and the insufficient internal and external representation learning of image-report pairs. To address these issues, we propose an Anatomical Structure-Guided (A… ▽ More

    Submitted 14 March, 2024; originally announced March 2024.

  16. arXiv:2403.02622  [pdf, other

    cs.LG cs.AI cs.RO

    World Models for Autonomous Driving: An Initial Survey

    Authors: Yanchen Guan, Haicheng Liao, Zhenning Li, Jia Hu, Runze Yuan, Yunjian Li, Guohui Zhang, Chengzhong Xu

    Abstract: In the rapidly evolving landscape of autonomous driving, the capability to accurately predict future events and assess their implications is paramount for both safety and efficiency, critically aiding the decision-making process. World models have emerged as a transformative approach, enabling autonomous driving systems to synthesize and interpret vast amounts of sensor data, thereby predicting po… ▽ More

    Submitted 7 May, 2024; v1 submitted 4 March, 2024; originally announced March 2024.

  17. arXiv:2402.17098  [pdf, other

    cs.CV

    In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking

    Authors: Peng Gao, Shi-Min Li, Feng Gao, Fei Wang, Ru-Yue Yuan, Hamido Fujita

    Abstract: Deep learning-based methods monopolize the latest research in the field of thermal infrared (TIR) object tracking. However, relying solely on deep learning models to obtain better tracking results requires carefully selecting feature information that is beneficial to representing the target object and designing a reasonable template update strategy, which undoubtedly increases the difficulty of mo… ▽ More

    Submitted 26 February, 2024; originally announced February 2024.

  18. arXiv:2402.16570  [pdf, other

    cs.CV cs.LG

    Searching a Lightweight Network Architecture for Thermal Infrared Pedestrian Tracking

    Authors: Peng Gao, Xiao Liu, Yu Wang, Ru-Yue Yuan

    Abstract: Manually-designed network architectures for thermal infrared pedestrian tracking (TIR-PT) require substantial effort from human experts. Neural networks with ResNet backbones are popular for TIR-PT. However, TIR-PT is a tracking task and more challenging than classification and detection. This paper makes an early attempt to search an optimal network architecture for TIR-PT automatically, employin… ▽ More

    Submitted 26 February, 2024; originally announced February 2024.

  19. arXiv:2402.16153  [pdf, other

    cs.SD cs.AI cs.CL cs.LG cs.MM eess.AS

    ChatMusician: Understanding and Generating Music Intrinsically with LLM

    Authors: Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Ziyang Ma, Liumeng Xue, Ziyu Wang, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Pengfei Li, Jingcheng Wu, Chenghua Lin, Qifeng Liu , et al. (10 additional authors not shown)

    Abstract: While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative language. We introduce ChatMusician, an open-source LLM that integrates intrinsic musical abilities. It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the… ▽ More

    Submitted 25 February, 2024; originally announced February 2024.

    Comments: GitHub: https://shanghaicannon.github.io/ChatMusician/

  20. arXiv:2402.14309  [pdf, other

    cs.CV

    YOLO-TLA: An Efficient and Lightweight Small Object Detection Model based on YOLOv5

    Authors: Peng Gao, Chun-Lin Ji, Tao Yu, Ru-Yue Yuan

    Abstract: Object detection, a crucial aspect of computer vision, has seen significant advancements in accuracy and robustness. Despite these advancements, practical applications still face notable challenges, primarily the inaccurate detection or missed detection of small objects. In this paper, we propose YOLO-TLA, an advanced object detection model building on YOLOv5. We first introduce an additional dete… ▽ More

    Submitted 22 February, 2024; originally announced February 2024.

    Comments: 11 pages, 11 figures, 7 tables

  21. arXiv:2402.14304   

    cs.RO cs.AI cs.CV

    Vision-Language Navigation with Embodied Intelligence: A Survey

    Authors: Peng Gao, Peng Wang, Feng Gao, Fei Wang, Ruyue Yuan

    Abstract: As a long-term vision in the field of artificial intelligence, the core goal of embodied intelligence is to improve the perception, understanding, and interaction capabilities of agents and the environment. Vision-language navigation (VLN), as a critical research path to achieve embodied intelligence, focuses on exploring how agents use natural language to communicate effectively with humans, rece… ▽ More

    Submitted 15 March, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

    Comments: The pictures in Figures 2, 4, and 5 are used without authorization, and the literatures in Table 1 have been cited improperly

  22. arXiv:2402.14236  [pdf, other

    cs.LG cs.AI cs.AR

    Automated Design and Optimization of Distributed Filtering Circuits via Reinforcement Learning

    Authors: Peng Gao, Tao Yu, Fei Wang, Ru-Yue Yuan

    Abstract: Designing distributed filtering circuits (DFCs) is complex and time-consuming, with the circuit performance relying heavily on the expertise and experience of electronics engineers. However, manual design methods tend to have exceedingly low-efficiency. This study proposes a novel end-to-end automated method for fabricating circuits to improve the design of DFCs. The proposed method harnesses rein… ▽ More

    Submitted 21 February, 2024; originally announced February 2024.

    Comments: 13 pages, 7 figures, 4 tables

  23. arXiv:2402.13109  [pdf, other

    cs.CL cs.AI

    CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models

    Authors: Yizhi LI, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Stephen W. Huang, Chenghua Lin, Jie Fu

    Abstract: The advancement of large language models (LLMs) has enhanced the ability to generalize across a wide range of unseen natural language processing (NLP) tasks through instruction-following. Yet, their effectiveness often diminishes in low-resource languages like Chinese, exacerbated by biased evaluations from data leakage, casting doubt on their true generalizability to new linguistic territories. I… ▽ More

    Submitted 4 June, 2024; v1 submitted 20 February, 2024; originally announced February 2024.

    Comments: Camera-ready version for ACL 2024. Project page at https://yizhilll.github.io/CIF-Bench/

  24. arXiv:2402.12226  [pdf, other

    cs.CL cs.AI cs.CV cs.LG

    AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling

    Authors: Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yugang Jiang, Xipeng Qiu

    Abstract: We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music. AnyGPT can be trained stably without any alterations to the current large language model (LLM) architecture or training paradigms. Instead, it relies exclusively on data-level preprocessing, facilitating the… ▽ More

    Submitted 7 March, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

    Comments: 28 pages, 16 figures, under review, work in progress

  25. arXiv:2401.11944  [pdf, other

    cs.CL cs.AI cs.CV

    CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark

    Authors: Ge Zhang, Xinrun Du, Bei Chen, Yiming Liang, Tongxu Luo, Tianyu Zheng, Kang Zhu, Yuyang Cheng, Chunpu Xu, Shuyue Guo, Haoran Zhang, Xingwei Qu, Junjie Wang, Ruibin Yuan, Yizhi Li, Zekun Wang, Yudong Liu, Yu-Hsuan Tsai, Fengji Zhang, Chenghua Lin, Wenhao Huang, Wenhu Chen, Jie Fu

    Abstract: As the capabilities of large multimodal models (LMMs) continue to advance, evaluating the performance of LMMs emerges as an increasing need. Additionally, there is an even larger gap in evaluating the advanced knowledge and reasoning abilities of LMMs in non-English contexts such as Chinese. We introduce CMMMU, a new Chinese Massive Multi-discipline Multimodal Understanding benchmark designed to e… ▽ More

    Submitted 18 March, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

  26. Joint Beam Direction Control and Radio Resource Allocation in Dynamic Multi-beam LEO Satellite Networks

    Authors: Shuo Yuan, Yaohua Sun, Mugen Peng, Renzhi Yuan

    Abstract: Multi-beam low earth orbit (LEO) satellites are emerging as key components in beyond 5G and 6G to provide global coverage and high data rate. To fully unleash the potential of LEO satellite communication, resource management plays a key role. However, the uneven distribution of users, the coupling of multi-dimensional resources, complex inter-beam interference, and time-varying network topologies… ▽ More

    Submitted 17 January, 2024; originally announced January 2024.

    Comments: Accepted by IEEE Transactions on Vehicular Technology

  27. Core-periphery Detection Based on Masked Bayesian Non-negative Matrix Factorization

    Authors: Zhonghao Wang, Ru Yuan, Jiaye Fu, Ka-Chun Wong, Chengbin Peng

    Abstract: Core-periphery structure is an essential mesoscale feature in complex networks. Previous researches mostly focus on discriminative approaches while in this work, we propose a generative model called masked Bayesian non-negative matrix factorization. We build the model using two pair affiliation matrices to indicate core-periphery pair associations and using a mask matrix to highlight connections t… ▽ More

    Submitted 16 January, 2024; originally announced January 2024.

    Comments: 12 pages, 11 figures. IEEE Transactions on Computational Social Systems(TCSS), 2024, early access

    Journal ref: IEEE Transactions on Computational Social Systems

  28. arXiv:2401.04518  [pdf, other

    cs.CL cs.AI

    The Critique of Critique

    Authors: Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu

    Abstract: Critique, as a natural language description for assessing the quality of model-generated content, has played a vital role in the training, evaluation, and refinement of LLMs. However, a systematic method to evaluate the quality of critique is lacking. In this paper, we pioneer the critique of critique, termed MetaCritique, which builds specific quantification criteria. To achieve a reliable evalua… ▽ More

    Submitted 1 June, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

    Comments: Accepted to Findings of ACL 2024

  29. arXiv:2312.17257  [pdf, other

    cs.CL cs.AI

    Evolving Large Language Model Assistant with Long-Term Conditional Memory

    Authors: Ruifeng Yuan, Shichao Sun, Zili Wang, Ziqiang Cao, Wenjie Li

    Abstract: With the rapid development of large language models, AI assistants like ChatGPT have widely entered people's works and lives. In this paper, we present an evolving large language model assistant that utilizes verbal long-term memory. It focuses on preserving the knowledge and experience from the history dialogue between the user and AI assistant, which can be applied to future dialogue for generat… ▽ More

    Submitted 21 December, 2023; originally announced December 2023.

  30. arXiv:2311.17532  [pdf, other

    cs.CV

    Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation

    Authors: Xingqun Qi, Jiahao Pan, Peng Li, Ruibin Yuan, Xiaowei Chi, Mengfei Li, Wenhan Luo, Wei Xue, Shanghang Zhang, Qifeng Liu, Yike Guo

    Abstract: Generating vivid and emotional 3D co-speech gestures is crucial for virtual avatar animation in human-machine interaction applications. While the existing methods enable generating the gestures to follow a single emotion label, they overlook that long gesture sequence modeling with emotion transition is more practical in real scenes. In addition, the lack of large-scale available datasets with emo… ▽ More

    Submitted 27 March, 2024; v1 submitted 29 November, 2023; originally announced November 2023.

    Comments: Accepted by CVPR 2024

  31. arXiv:2311.16502  [pdf, other

    cs.CL cs.AI cs.CV

    MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

    Authors: Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, Wenhu Chen

    Abstract: We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and… ▽ More

    Submitted 13 June, 2024; v1 submitted 27 November, 2023; originally announced November 2023.

    Comments: CVPR 2024 Oral

  32. arXiv:2311.00399  [pdf, other

    cs.CV cs.CL

    Enhanced Knowledge Injection for Radiology Report Generation

    Authors: Qingqiu Li, Jilan Xu, Runtian Yuan, Mohan Chen, Yuejie Zhang, Rui Feng, Xiaobo Zhang, Shang Gao

    Abstract: Automatic generation of radiology reports holds crucial clinical value, as it can alleviate substantial workload on radiologists and remind less experienced ones of potential anomalies. Despite the remarkable performance of various image captioning methods in the natural image field, generating accurate reports for medical images still faces challenges, i.e., disparities in visual and textual data… ▽ More

    Submitted 1 November, 2023; originally announced November 2023.

    Comments: Accepted by BIBM 2023

  33. arXiv:2310.14215  [pdf, other

    cs.IR cs.AI

    Item-Graph2vec: a Efficient and Effective Approach using Item Co-occurrence Graph Embedding for Collaborative Filtering

    Authors: Ruilin Yuan, Leya Li, Yuanzhe Cai

    Abstract: Current item-item collaborative filtering algorithms based on artificial neural network, such as Item2vec, have become ubiquitous and are widely applied in the modern recommender system. However, these approaches do not apply to the large-scale item-based recommendation system because of their extremely long training time. To overcome the shortcoming that current algorithms have high training time… ▽ More

    Submitted 22 October, 2023; originally announced October 2023.

  34. arXiv:2309.12200  [pdf, other

    eess.SP cs.LG cs.SI

    A Variational Auto-Encoder Enabled Multi-Band Channel Prediction Scheme for Indoor Localization

    Authors: Ruihao Yuan, Kaixuan Huang, Pan Yang, Shunqing Zhang

    Abstract: Indoor localization is getting increasing demands for various cutting-edged technologies, like Virtual/Augmented reality and smart home. Traditional model-based localization suffers from significant computational overhead, so fingerprint localization is getting increasing attention, which needs lower computation cost after the fingerprint database is built. However, the accuracy of indoor localiza… ▽ More

    Submitted 19 September, 2023; originally announced September 2023.

  35. arXiv:2309.09012  [pdf, other

    eess.SY cs.IT

    Modelling Irrational Behaviour of Residential End Users using Non-Stationary Gaussian Processes

    Authors: Nam Trong Dinh, Sahand Karimi-Arpanahi, Rui Yuan, S. Ali Pourmousavi, Mingyu Guo, Jon A. R. Liisberg, Julian Lemos-Vinasco

    Abstract: Demand response (DR) plays a critical role in ensuring efficient electricity consumption and optimal use of network assets. Yet, existing DR models often overlook a crucial element, the irrational behaviour of electricity end users. In this work, we propose a price-responsive model that incorporates key aspects of end-user irrationality, specifically loss aversion, time inconsistency, and bounded… ▽ More

    Submitted 26 March, 2024; v1 submitted 16 September, 2023; originally announced September 2023.

    Comments: This manuscript has been accepted for publication in IEEE Transactions on Smart Grid

  36. A Novel Temporal Multi-Gate Mixture-of-Experts Approach for Vehicle Trajectory and Driving Intention Prediction

    Authors: Renteng Yuan, Mohamed Abdel-Aty, Qiaojun Xiang, Zijin Wang, Ou Zheng

    Abstract: Accurate Vehicle Trajectory Prediction is critical for automated vehicles and advanced driver assistance systems. Vehicle trajectory prediction consists of two essential tasks, i.e., longitudinal position prediction and lateral position prediction. There is a significant correlation between driving intentions and vehicle motion. In existing work, the three tasks are often conducted separately with… ▽ More

    Submitted 1 August, 2023; originally announced August 2023.

    Journal ref: IEEE Transactions on Intelligent Vehicles

  37. arXiv:2307.15625  [pdf

    stat.ML cs.LG

    A Comparative Analysis of Machine Learning Methods for Lane Change Intention Recognition Using Vehicle Trajectory Data

    Authors: Renteng Yuan

    Abstract: Accurately detecting and predicting lane change (LC)processes can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes and compares different machine learning methods' performance to recognize LC intention from high-dimensionality time series data. To validate the performance of… ▽ More

    Submitted 28 July, 2023; originally announced July 2023.

    Comments: arXiv admin note: text overlap with arXiv:2304.13732

  38. arXiv:2307.12326  [pdf, other

    cs.RO

    Scale jump-aware pose graph relaxation for monocular SLAM with re-initializations

    Authors: Runze Yuan, Ran Cheng, Lige Liu, Tao Sun, Laurent Kneip

    Abstract: Pose graph relaxation has become an indispensable addition to SLAM enabling efficient global registration of sensor reference frames under the objective of satisfying pair-wise relative transformation constraints. The latter may be given by incremental motion estimation or global place recognition. While the latter case enables loop closures and drift compensation, care has to be taken in the mono… ▽ More

    Submitted 23 July, 2023; originally announced July 2023.

    Comments: 8 pages, 23 figures, International Conference on Intelligent Robots and Systems 2023

  39. arXiv:2307.05161  [pdf, other

    cs.SD cs.AI cs.LG eess.AS

    On the Effectiveness of Speech Self-supervised Learning for Music

    Authors: Yinghao Ma, Ruibin Yuan, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Ruibo Liu, Gus Xia, Roger Dannenberg, Yike Guo, Jie Fu

    Abstract: Self-supervised learning (SSL) has shown promising results in various speech and natural language processing applications. However, its efficacy in music information retrieval (MIR) still remains largely unexplored. While previous SSL models pre-trained on music recordings may have been mostly closed-sourced, recent speech models such as wav2vec2.0 have shown promise in music modelling. Neverthele… ▽ More

    Submitted 11 July, 2023; originally announced July 2023.

  40. arXiv:2306.17103  [pdf, other

    cs.CL cs.SD eess.AS

    LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT

    Authors: Le Zhuo, Ruibin Yuan, Jiahao Pan, Yinghao Ma, Yizhi LI, Ge Zhang, Si Liu, Roger Dannenberg, Jie Fu, Chenghua Lin, Emmanouil Benetos, Wenhu Chen, Wei Xue, Yike Guo

    Abstract: We introduce LyricWhiz, a robust, multilingual, and zero-shot automatic lyrics transcription method achieving state-of-the-art performance on various lyrics transcription datasets, even in challenging genres such as rock and metal. Our novel, training-free approach utilizes Whisper, a weakly supervised robust speech recognition model, and GPT-4, today's most performant chat-based large language mo… ▽ More

    Submitted 21 November, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

    Comments: 9 pages, 2 figures, 5 tables, accepted by ISMIR 2023

  41. arXiv:2306.10548  [pdf, other

    cs.SD cs.AI cs.LG eess.AS

    MARBLE: Music Audio Representation Benchmark for Universal Evaluation

    Authors: Ruibin Yuan, Yinghao Ma, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Roger Dannenberg, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu

    Abstract: In the era of extensive intersection between art and Artificial Intelligence (AI), such as image generation and fiction co-creation, AI for music remains relatively nascent, particularly in music understanding. This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark. To address this issue… ▽ More

    Submitted 23 November, 2023; v1 submitted 18 June, 2023; originally announced June 2023.

    Comments: camera-ready version for NeurIPS 2023

  42. arXiv:2306.00107  [pdf, other

    cs.SD cs.AI cs.CL cs.LG eess.AS

    MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

    Authors: Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Zili Wang, Yike Guo, Jie Fu

    Abstract: Self-supervised learning (SSL) has recently emerged as a promising paradigm for training generalisable models on large-scale data in the fields of vision, text, and speech. Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored. This is partially due to the distinctive challenges associated with modelling musical knowledge, part… ▽ More

    Submitted 22 April, 2024; v1 submitted 31 May, 2023; originally announced June 2023.

    Comments: accepted by ICLR 2024

  43. arXiv:2305.14994  [pdf, other

    cs.CL

    RefGPT: Dialogue Generation of GPT, by GPT, and for GPT

    Authors: Dongjie Yang, Ruifeng Yuan, Yuantao Fan, Yifei Yang, Zili Wang, Shusen Wang, Hai Zhao

    Abstract: Large Language Models (LLMs) have attained the impressive capability to resolve a wide range of NLP tasks by fine-tuning high-quality instruction data. However, collecting human-written data of high quality, especially multi-turn dialogues, is expensive and unattainable for most people. Though previous studies have used powerful LLMs to generate the dialogues automatically, they all suffer from ge… ▽ More

    Submitted 18 October, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

    Comments: Accepted by EMNLP 2023

  44. arXiv:2304.13732  [pdf

    cs.LG

    Lane Change Intention Recognition and Vehicle Status Prediction for Autonomous Vehicles

    Authors: Renteng Yuan, Mohamed Abdel-Aty, Xin Gu, Ou Zheng, Qiaojun Xiang

    Abstract: Accurately detecting and predicting lane change (LC)processes of human-driven vehicles can help autonomous vehicles better understand their surrounding environment, recognize potential safety hazards, and improve traffic safety. This paper focuses on LC processes, first developing a temporal convolutional network with an attention mechanism (TCN-ATM) model to recognize LC intention. Considering th… ▽ More

    Submitted 19 July, 2023; v1 submitted 25 April, 2023; originally announced April 2023.

  45. arXiv:2304.09743  [pdf, other

    cs.DS

    The Price of Explainability for Clustering

    Authors: Anupam Gupta, Madhusudhan Reddy Pittu, Ola Svensson, Rachel Yuan

    Abstract: Given a set of points in $d$-dimensional space, an explainable clustering is one where the clusters are specified by a tree of axis-aligned threshold cuts. Dasgupta et al. (ICML 2020) posed the question of the price of explainability: the worst-case ratio between the cost of the best explainable clusterings to that of the best clusterings. We show that the price of explainability for $k$-medians… ▽ More

    Submitted 19 April, 2023; originally announced April 2023.

  46. arXiv:2304.07987  [pdf, other

    cs.CL cs.AI

    Chinese Open Instruction Generalist: A Preliminary Release

    Authors: Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, Jie Fu

    Abstract: Instruction tuning is widely recognized as a key technique for building generalist language models, which has attracted the attention of researchers and the public with the release of InstructGPT~\citep{ouyang2022training} and ChatGPT\footnote{\url{https://chat.openai.com/}}. Despite impressive progress in English-oriented large-scale language models (LLMs), it is still under-explored whether Engl… ▽ More

    Submitted 24 April, 2023; v1 submitted 17 April, 2023; originally announced April 2023.

  47. arXiv:2304.00944  [pdf, other

    cs.CR cs.CY

    Lessons in VCR Repair: Compliance of Android App Developers with the California Consumer Privacy Act (CCPA)

    Authors: Nikita Samarin, Shayna Kothari, Zaina Siyed, Oscar Bjorkman, Reena Yuan, Primal Wijesekera, Noura Alomar, Jordan Fischer, Chris Hoofnagle, Serge Egelman

    Abstract: The California Consumer Privacy Act (CCPA) provides California residents with a range of enhanced privacy protections and rights. Our research investigated the extent to which Android app developers comply with the provisions of the CCPA that require them to provide consumers with accurate privacy notices and respond to "verifiable consumer requests" (VCRs) by disclosing personal information that… ▽ More

    Submitted 3 April, 2023; originally announced April 2023.

    Comments: Appears in Issue 3 of 23rd Privacy Enhancing Technologies Symposium (PETS 23)

  48. arXiv:2303.08366  [pdf, other

    quant-ph cs.HC

    VENUS: A Geometrical Representation for Quantum State Visualization

    Authors: Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang

    Abstract: Visualizations have played a crucial role in helping quantum computing users explore quantum states in various quantum computing applications. Among them, Bloch Sphere is the widely-used visualization for showing quantum states, which leverages angles to represent quantum amplitudes. However, it cannot support the visualization of quantum entanglement and superposition, the two essential propertie… ▽ More

    Submitted 23 April, 2023; v1 submitted 15 March, 2023; originally announced March 2023.

  49. arXiv:2302.12490  [pdf, other

    cs.CL

    Improving Sentence Similarity Estimation for Unsupervised Extractive Summarization

    Authors: Shichao Sun, Ruifeng Yuan, Wenjie Li, Sujian Li

    Abstract: Unsupervised extractive summarization aims to extract salient sentences from a document as the summary without labeled data. Recent literatures mostly research how to leverage sentence similarity to rank sentences in the order of salience. However, sentence similarity estimation using pre-trained language models mostly takes little account of document-level information and has a weak correlation w… ▽ More

    Submitted 24 February, 2023; originally announced February 2023.

    Comments: Accepted by ICASSP 2023

  50. arXiv:2301.13139  [pdf, other

    stat.ML cs.LG math.OC math.ST

    A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence

    Authors: Carlo Alfano, Rui Yuan, Patrick Rebeschini

    Abstract: Modern policy optimization methods in reinforcement learning, such as TRPO and PPO, owe their success to the use of parameterized policies. However, while theoretical guarantees have been established for this class of algorithms, especially in the tabular setting, the use of general parameterization schemes remains mostly unjustified. In this work, we introduce a novel framework for policy optimiz… ▽ More

    Submitted 13 February, 2024; v1 submitted 30 January, 2023; originally announced January 2023.

    Comments: Post-conference updates