seeking good explanations with machine learning


senior researcher at microsoft research (deep learning group) ; phd from berkeley (with prof. bin yu)

  Location via proxy:   
[Report a bug]   [Manage cookies]                

Research

Some areas I'm currently excited about. If you want to chat about research or are interested in interning at MSR, feel free to reach out over email :)
🔎 Interpretability methods, especially LLM interpretability.

augmented imodels - build a transparent model using LLMs
attention steering - mechanistically guide LLMs by emphasizing specific input spans
explanation penalization - regularize explanations to align models with prior knowledge
adaptive wavelet distillation - replace neural nets with transparent wavelet models
🧠 Semantic brain mapping, mostly using fMRI responses to language.

explanation-mediated validation - causally test fMRI explanations with LLM-generated stimuli
qa encoding models - model fMRI language responses to verbal theories using LLM annotations
summarize & score explanations - generate natural-language explanations of fMRI encoding models
💊 Clinical decision rules, can we improve them with data?

greedy tree sums - build accurate, compact tree-based clinical models
clinical self-verification - improve LLM-based clinical information extraction with self-verirication
clinical rule vetting - test a clinical decision instrument for evaluating intra-abdominal injury
clinical rule bias assessment - evaluate biases in the development of popular clinical decision instruments
Note: I put a lot of my code into the imodels and imodelsX packages.

year title authors tags paper code misc
'25 Systematic Bias in Clinical Decision Instrument Development obra, singh, et al. 🔎💊 medrxiv
'25 Analyzing patient perspectives with llms kornblith*, singh* et al. 💊🌀 nature scientific reports
'25 Simplifying DINO via Coding Rate Regularization wu et al. 🌀 arxiv
'25 Towards Understanding Graphical Perception in Large Multimodal Models zhang et al. 🌀 arxiv
'25 Vector-ICL: In-context Learning with Continuous Vector Representations zhuang et al. 🔎🌀 iclr
'24 Crafting Interpretable Embeddings by Asking LLMs Questions benara*, singh*, morris, antonello, stoica, huth, & gao 🧠🔎🌀 neurips
'24 Generative causal testing to bridge data-driven models and scientific theories in language neuroscience antonello*, singh*, jain, hsu, gao, yu, & huth 🧠🔎🌀 arxiv
'24 Interpretable Language Modeling via Induction-head Ngram Models kim*, mantena*, et al. 🧠🔎🌀 arxiv
'24 Rethinking Interpretability in the Era of Large Language Models singh, inala, galley, caruana, & gao 🔎🌀 arxiv
'24 Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning chen et al. 🔎🌀 COLING
'24 Learning a Decision Tree Algorithm with Transformers zhuang et al. 🔎🌀🌳 tmlr
'24 Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steering zhang*, yu*, et al. 🔎🌀 arxiv
'24 Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs zhang et al. 🔎🌀 iclr
'24 Attribute Structuring Improves LLM-Based Evaluation of Clinical Text Summaries gero et al. 🔎🌀 ml4h findings
'23 Tree Prompting morris*, singh*, rush, gao, & deng 🔎🌀🌳 emnlp
'23 Augmenting Interpretable Models with LLMs during Training singh, askari, caruana, & gao 🔎🌀🌳 nature communications
'23 Explaining black box text modules in natural language with language models singh*, hsu*, antonello, jain, huth, yu & gao 🔎🌀 neurips workshop
'23 Self-Verification Improves Few-Shot Clinical Information Extraction gero*, singh*, cheng, naumann, galley, gao, & poon 🔎🌀💊 icml workshop
'22 Explaining patterns in data with language models via interpretable autoprompting singh*, morris*, aneja, rush, & gao 🔎🌀 emnlp workshop
'22 Stress testing a clinical decision instrument performance for intra-abdominal injury kornblith*, singh* et al. 🔎🌳💊 PLOS digital health
'22 Fast interpretable greedy-tree sums (FIGS) tan*, singh*, nasseri, agarwal, & yu 🔎🌳 pnas
'22 Hierarchical shrinkage for trees agarwal*, tan*, ronen, singh, & yu 🔎🌳 icml (spotlight)
'22 VeridicalFlow: a python package for building trustworthy data science pipelines with PCS duncan*, kapoor*, agarwal*, singh*, & yu 💻🔍 joss
'21 imodels: a python package for fitting interpretable models singh*, nasseri*, et al. 💻🔍🌳 joss
'21 Adaptive wavelet distillation from neural networks through interpretations ha, singh, et al. 🔍🌀🌳 neurips
'21 Matched sample selection with GANs for mitigating attribute confounding singh, balakrishnan, & perona 🌀 cvpr workshop
'21 Revisiting complexity and the bias-variance tradeoff dwivedi*, singh*, yu & wainwright 🌀 jmlr
'20 Curating a COVID-19 data repository and forecasting county-level death counts in the United States altieri et al. 🔎🦠 hdsr
'20 Transformation importance with applications to cosmology singh*, ha*, lanusse, boehm, liu & yu 🔎🌀🌌 iclr workshop (spotlight)
'20 Interpretations are useful: penalizing explanations to align neural networks with prior knowledge rieger, singh, murdoch & yu 🔎🌀 icml
'19 Hierarchical interpretations for neural network predictions Singh*, Murdoch*, & Yu 🔍🌀 ICLR
'19 interpretable machine learning: definitions, methods, and applications Murdoch*, Singh*, et al. 🔍🌳🌀 pnas
'19 disentangled attribution curves for interpreting random forests and boosted trees devlin, singh, murdoch & yu 🔍🌳 arxiv
'18 large scale image segmentation with structured loss based deep learning for connectome reconstruction Funke*, Tschopp*, et al. 🧠🌀 TPAMI
'18 linearization of excitatory synaptic integration at no extra cost Morel, Singh, & Levy 🧠 J Comp Neuro
'17 a consensus layer V pyramidal neuron can sustain interpulse-interval coding Singh & Levy 🧠 Plos One
'17 a constrained, weighted-l1 minimization approach for joint discovery of heterogeneous neural connectivity graphs Singh, Wang, & Qi 🧠 neurips Workshop ,

resources + posts



Notes in machine learning / neuroscience.

Mini personal projects. There's also some dumb stuff here.

experience

  • microsoft research
    Summer '22 - Present

    generating explanations with large language models for science & medicine, working in the deep learning group

  • healthcare ai
    Summer '21-Summer '22, Spring '20, Summer '19

    worked at paige ai as a research scientist developing deep learning models for pathology -- also worked at pacmed ai in 2019 and response4life covid forecasting in 2020

  • phd at uc berkeley
    Fall '17-Spring '22

    researched interpretable machine learning, especially for neural networks and in scientific applications, working in the bin yu group

  • big tech internships
    Summer '17, Summer '20

    worked on unsupervised semantic segmentation at meta ai, and worked on causal fairness benchmarking in computer vision at aws

  • undergrad
    summer '14- spring '17

    worked on graphical models under Yanjun Qi and on biophysical modeling under William Levy -- also spent 3 summers doing research at HHMI Janelia under Srini Turaga


I've been lucky to work with many amazing people & help advise some incredible students

Advisors / managers
Scientific collaborators