🤖PhD Candidate
he/his/him
I am a final-year PhD candidate in Computer Science at UCLA, working with Prof. Wei Wang. I’m also a machine learning scientist on the Genentech Prescient Design Language Model team, working on LLM agents for scientific discovery and LLM post-training with Dr. Keunwoo Choi, Dr. Stephen Ra, and Prof. Kyunghyun Cho. I’m a recipient of the J.P. Morgan Chase AI PhD Fellowship and an Amazon Fellow.
I’ve worked at Amazon AGI, USC (working with Prof. Nanyun (Violet) Peng and Prof. Muhao Chen), The Chinese University of Hong Kong (working with Prof. Helen Meng), UC Santa Cruz (working with Prof. Marilyn Walker) and MIT (working with Dr. Abel Sanchez and Prof. John R. Williams). I earned my bachelor’s degree in Computing from The Hong Kong Polytechnic University, advised by Prof. Qin Lu and Prof. Jiannong Cao and studied at the University of Maryland.
My Research
I develop machine learning (ML) systems inspired by scientific data and expert tasks, equipping large language models (LLMs) with the intuition and knowledge of domain experts. My research introduces machine learning innovations and insights to enable a comprehensive spectrum of expertise acquisition, from explicit to implicit knowledge and from individual decision-making to the automation of complex expert workflows. Specifically, I focus on:
Extracting explicit knowledge from unstructured data in low-resource scenarios: dataset (ACL'23), library (NAACL'21), data-efficient (ACL'23) and parameter-efficient (INTERSPEECH'23) methods, indirect supervision (ACL'23, EMNLP-F'22), cross-document (ACL-F'23), data synthesis/augmentation for zero-shot scenarios (AAAI'24, ACL'24)
Capturing implicit expert intuition: LLMs’ clinical decision-making benchmark (preprint 24), rich supervision to model decision sequences (AAAI'25), conveying intuition with a decoding-free paradigm (NeurIPS ENLSP'24)
Compositional, project-level reasoning and automation: KG-inspired reasoning (preprint 24), cross-modality (NeurIPS'24), drug discovery agents, scientific workflow agent platform, material design (preprint 24)
Fairness and safety of generative LLMs: unsupervised bias mitigation (NAACL'24), attacking LLM with data poisoning (NAACL'24), ownership protection (NAACL'24), LLMs’ clinical bias analysis (preprint 24)
Empowered expert applications: clinical diagnosis (preprint 24, preprint 24, preprint 24), health outcome prediction (AAAI'25), clinical event extraction (ACL'23), biomedical and scientific QAs (preprint 24, NeurIPS'24, NAACL'24, ACL'23), computational social science (EMNLP'24, AAAI'24), political event forecasting (preprint 24), dialogue state tracking (INTERSPEECH'23), knowledge structure/graph construction (EMNLP-F'21, AKBC'22)
New preprint on new output paradigm for LLMs
Our new preprint explores a new paradigm for expressing LLMs’ decisions without decoding concrete words. We formally define the paradigm and introduce a comprehensive evaluation set covering a few to thousands of candidates. We found that this new output paradigm can outperform full decoding while being 40x faster.
New preprint using KG structure as inspiration for reasoning
Our new preprint simulates the thinking process of the knowledge graph constructors and utilizes the KG structure as inspiration for reasoning. This is especially helpful when expert-curated KG is sparse, enabling significantly better performance on biomedical KG-based QA.
Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction
We introduce MERA, a clinical diagnosis prediction model that bridges pertaining natural language knowledge with medical practice. We apply hierarchical contrastive learning on a disease candidate ranking list to alleviate the large decision space issue. With concept memorization through fine-tuning, we bridge the natural language clinical knowledge with medical codes.
Inferring from Logits: Exploring Best Practices for Decoding-Free Generative Candidate Selection
Existing works have been using decoding-free candidate selection methods to obtain candidate probability from initial output logits over vocabulary. Though these estimation methods are widely used, they are not systematically evaluated, especially on end tasks. We introduce an evaluation of a comprehensive collection of decoding-free candidate selection approaches.
CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Making
We introduce CliBench, a novel benchmark offering a comprehensive and realistic assessment of LLMs' capabilities in clinical diagnosis. This benchmark not only covers diagnosis from a diverse range of medical cases across various specialties but also incorporates tasks of clinical significance: treatment procedure identification, lab test ordering and medication prescriptions.
GIVE: Structured Reasoning with Knowledge Graph Inspired Veracity Extrapolation
GIVE is a novel reasoning framework that integrates the parametric and non-parametric memories to enhance both knowledge retrieval and faithful reasoning processes on very sparse knowledge graphs. By leveraging the external structured knowledge to inspire LLM to model the interconnections among relevant concepts, our method facilitates a more logical and step-wise reasoning approach akin to human problem-solving, rather than gold answer retrieval.
GraphVis: Boosting LLMs with Visual Knowledge Graph Integration
GraphVis conserves the intricate graph structure through the visual modality to enhance the comprehension of KGs with the aid of Large Vision Language Models (LVLMs). Our approach incorporates a unique curriculum fine-tuning scheme which first instructs LVLMs to recognize basic graphical features from the images, and subsequently incorporates reasoning on QA tasks with the visual graphs.
Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach
We propose a computational model to infer users' susceptibility levels given their activities. Since user's susceptibility is a key indicator for their reposting behavior, we utilize the supervision from the observable sharing behavior to infer the underlying susceptibility tendency. Building upon such large-scale susceptibility labeling, we further conduct a comprehensive analysis of how different social factors relate to susceptibility.
CLIMB: A Benchmark of Clinical Bias in Large Language Models
We introduce a pioneering comprehensive benchmark to evaluate both intrinsic (within LLMs) and extrinsic (on downstream tasks) bias in LLMs for clinical decision tasks. Our experiments across popular and medically adapted LLMs, particularly from the Mistral and LLaMA families, unveil prevalent behaviors with both intrinsic and extrinsic bias. This work underscores the critical need to mitigate clinical bias and sets a new standard for future evaluations of LLMs' clinical bias.
MIRAI: Evaluating LLM Agents for Event Forecasting
We introduce MIRAI, a novel benchmark designed to systematically evaluate LLM agents as temporal forecasters in the context of international events. Our benchmark features an agentic environment with tools for accessing an extensive database of historical, structured events and textual news articles.
Mitigating Bias for Question Answering Models by Tracking Bias Influence
We propose BMBI, an approach to mitigate the bias of multiple-choice QA models. Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance by observing its influence on another instance. We then use the bias level detected as an optimization objective to form a multi-task learning setting in addition to the original QA task.
Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models
Our studies demonstrate that an attacker can inject backdoors by issuing very few malicious instructions among thousands of gathered data and control model behavior through data poisoning. Through such instruction attacks, the attacker can achieve over 90% attack success rate across four commonly used NLP datasets, and cause persistent backdoors that are easily transferred to 15 diverse datasets zero-shot.
Instructional Fingerprinting of Large Language Models
We present a pilot study on LLM fingerprinting as a form of very lightweight instruction tuning. Model publisher specifies a confidential private key and implants it as an instruction backdoor that causes the LLM to generate specific text when the key is present. Results on 11 popularly-used LLMs showed that this approach prevents publisher overclaim, maintains robustness against fingerprint guessing and parameter-efficient training, and supports multi-stage fingerprinting akin to MIT License.
STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models
We propose STAR, a structure-to-text data generation method for complicated structure prediction tasks that first generates complicated event structures (Y) and then generates input passages (X), all with Large Language Models. We further reduce errors and improve data quality through self-reflection error identification and self-refinement with iterative revision. We show that the data generated by STAR significantly improves the performance of low-resource event extraction and relation extraction tasks, even surpassing the effectiveness of human-curated data.
DICE: Data-Efficient Clinical Event Extraction with Generative Models
Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning
We use soft prompt tokens to learn task properties, incorporate segment information and reiterate the task before predicting value. Our method drastically reduces the number of parameters needed to less than 0.5% of prior works while achieving better low-resource dialogue state tracking performance.
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning
Experience
Prescient Design, Genentech, Roche
Machine Learning Scientist
Since Jun 2024, New York City, NY
Amazon Alexa AI
Applied Scientist Intern with Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Kai-Wei Chang, Nanyun Peng and Tagyoung Chung
Jun - Sep 2022, Sunnyvale, CAAmazon Alexa AI
Applied Scientist Intern with Jiun-Yu Kao, Shuyang Gao, Arpit Gupta, Di Jin, Nanyun Peng and Tagyoung Chung
Jun - Sep 2021, Remote
Education
University of California, Los Angeles
The Hong Kong Polytechnic University
Bachelor of Science in Computing (First Class Honours)
2014 - 2018, Hong Kong
Best Thesis Award, commencement speaker, advised by Prof. Qin Lu and Prof. Jiannong Cao
University of Maryland, College Park
Exchange Student
2016, College Park, MD
Awards
Teaching
Services
Lead Organizer Organizer: Area Chair:- ACL (2025)
- ACL Rolling Review (2025, 2024, 2023, 2022, 2021), NeurIPS (2024), ICLR (2025), ACL (2024, 2023, 2022), EMNLP (2024, 2023, 2022, 2021, 2019), NAACL (2025, 2024, 2022), COLM (2024), AISTATS (2025), KDD (2023), TheWebConf (2024), EACL (2023), NLPCC (2023, 2022)
- IEEE/ACM Transactions on Audio, Speech, and Language Processing (since 2023)
- NeurIPS workshop on Efficient Natural Language and Speech Processing (2024)
- AAAI Spring Symposium on Clinical Foundation Models (2024)
- EMNLP workshop on Deep Learning for Low-resources NLP workshop (2019)
- SoCal NLP Symposium (2023, 2022)