Matt Kusner

I am an Associate Professor in Machine Learning at University College London.

I'm interested in improving our fundamental understanding of ML (e.g., models, optimizers), and to use this understanding to create reliable, trustworthy machine learning algorithms with guarantees. This includes algorithms that preserve privacy, measure fairness, regulate model use, and provide guarantees for distribution shifts.

I am also a member of the European Laboratory for Learning and Intelligent Systems. Previously I was an Associate Professor at the University of Oxford. I was in the first group of Research Fellows at The Alan Turing Institute, and I did my Ph.D. with Kilian Weinberger. I am married to the wonderful Sonia Rego, and am a guardian of the fantastic Karl & Nora.

Current graduate students (alumni):

Yuchen (Caroline) Zhu (with Ricardo Silva)

Jean Kaddour (with Ricardo Silva)

Gbètondji Dovonon (with Michael Bronstein)

Leo Richter

Preprints

Setting the Record Straight on Transformer Oversmoothing
Gbètondji Dovonon, Michael Bronstein, Matt J. Kusner
paper

Papers

Proxy Methods for Domain Adaptation
Katherine Tsai, Stephen Pfohl, Olawale Salaudeen, Nicole Chiou, Matt J. Kusner, Alexander D'Amour, Sanmi Koyejo, Arthur Gretton
*=authors contributing equally, listed in alphabetical order
The International Conference on Artificial Intelligence and Statistics (AISTATS), 2024.
paper

No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models
Jean Kaddour*, Oscar Key*, Piotr Nawrot, Pasquale Minervini, Matt J. Kusner
*=authors contributing equally, listed in alphabetical order
Neural Information Processing Systems (NeurIPS), 2023.
paper

Adapting to Latent Subgroup Shifts via Concepts and Proxies
Ibrahim Alabdulmohsin*, Nicole Chiou*, Alexander D’Amour*, Arthur Gretton*, Sanmi Koyejo*, Matt J. Kusner*, Stephen R. Pfohl*, Olawale Salaudeen*, Jessica Schrouff*, Katherine Tsai*
*=authors contributing equally, listed in alphabetical order
The International Conference on Artificial Intelligence and Statistics (AISTATS), 2023.
paper | poster

DAG Learning on the Permutahedron
Valentina Zantedeschi, Luca Franceschi, Jean Kaddour, Matt J. Kusner, Vlad Niculae
The International Conference on Learning Representations (ICLR), 2023.
paper

Stochastic Causal Programming for Bounding Treatment Effects
Kirtan Padh, Jakob Zeitler, David Watson, Matt J. Kusner, Ricardo Silva, Niki Kilbertus
Conference on Causal Learning and Reasoning (CLeaR), 2023. Oral Presentation.
paper

When Do Flat Minima Optimizers Work?
Jean Kaddour, Linqing Liu, Ricardo Silva, Matt J. Kusner
Neural Information Processing Systems (NeurIPS), 2022.
paper

Local Latent Space Bayesian Optimization over Structured Inputs
Natalie Maus, Haydn T. Jones, Juston S. Moore, Matt J. Kusner, John Bradshaw, Jacob R. Gardner
Neural Information Processing Systems (NeurIPS), 2022.
paper

Causal Inference with Treatment Measurement Error: A Nonparametric Instrumental Variable Approach
Yuchen Zhu, Limor Gultchin, Arthur Gretton, Matt J. Kusner, Ricardo Silva
The Conference on Uncertainty in Artificial Intelligence (UAI), 2022. Oral Presentation.
paper

MPC-Friendly Commitments for Publicly Verifiable Covert Security
Nitin Agrawal, James Bell, Adrià Gascón, Matt J. Kusner
The Conference on Computer and Communications Security (CCS), 2021.
paper

Causal Effect Inference for Structured Treatments
Jean Kaddour, Yuchen Zhu, Qi Liu, Matt J. Kusner, Ricardo Silva
Neural Information Processing Systems (NeurIPS), 2021.
paper | code

Unsupervised Point Cloud Pre-Training via View-Point Occlusion, Completion
Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, Matt J. Kusner
The International Conference on Computer Vision (ICCV), 2021.
paper | code

Learning Binary Decision Trees by Argmin Differentiation
Valentina Zantedeschi, Matt J. Kusner, Vlad Niculae
The International Conference on Machine Learning (ICML), 2021.
paper | code

Operationalizing Complex Causes: A Pragmatic View of Mediation
Limor Gultchin, David Watson, Matt J. Kusner, Ricardo Silva
The International Conference on Machine Learning (ICML), 2021.
paper | code

Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction
Afsaneh Mastouri* Yuchen Zhu*, Limor Gultchin, Anna Korba, Ricardo Silva, Matt J. Kusner, Arthur Gretton', Krikamol Muandet'
*,'=authors contributing equally
The International Conference on Machine Learning (ICML), 2021.
paper | code

Counterfactual Data Augmentation for Neural Machine Translation
Qi Liu, Matt J. Kusner, Phil Blunsom
North American Chapter of the Association for Computational Linguistics (NAACL), 2021.
paper

A Class of Algorithms for General Instrumental Variable Models
Niki Kilbertus, Matt J. Kusner, Ricardo Silva
Neural Information Processing Systems (NeurIPS), 2020.
paper | code

Barking up the right tree: an approach to search over molecule synthesis DAGs
John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, José Miguel Hernández-Lobato
Neural Information Processing Systems (NeurIPS), 2020. Spotlight Presentation.
paper | code

The long road to fairer algorithms
Matt J. Kusner, Joshua R. Loftus
Nature (Comment), 2020.
paper

Differentiable Causal Backdoor Discovery
Limor Gultchin, Matt J. Kusner, Varun Kanade, Ricardo Silva
The International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
paper | code

A Model to Search for Synthesizable Molecules
John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, José Miguel Hernández-Lobato
Neural Information Processing Systems (NeurIPS), 2019.
paper | code

The Sensitivity of Counterfactual Fairness to Unmeasured Confounding
Niki Kilbertus, Philip J. Ball, Matt J. Kusner, Adrian Weller, Ricardo Silva
The Conference on Uncertainty in Artificial Intelligence (UAI), 2019.
paper | code

Making Decisions that Reduce Discriminatory Impact
Matt J. Kusner, Chris Russell, Joshua R. Loftus, Ricardo Silva
The International Conference on Machine Learning (ICML), 2019.
paper | code | slides

QUOTIENT: Two-Party Secure Neural Network Training and Prediction
Nitin Agrawal*, Ali Shahin Shamsabadi*, Matt J. Kusner, Adrià Gascón
*=authors contributing equally
The Conference on Computer and Communications Security (CCS), 2019.
paper

A Generative Model For Electron Paths
John Bradshaw, Matt J. Kusner, Brooks Paige, Marwin H. S. Segler, José Miguel Hernández-Lobato
The International Conference on Learning Representations (ICLR), 2019.
paper | code

TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service
Amartya Sanyal, Matt J. Kusner, Adrià Gascón, Varun Kanade
The International Conference on Machine Learning (ICML), 2018.
paper | code

Blind Justice: Fairness with Encrypted Sensitive Attributes
Niki Kilbertus, Adrià Gascón, Matt J. Kusner, Michael Veale, Krishna Gummadi, Adrian Weller
The International Conference on Machine Learning (ICML), 2018.
paper | code

Learning a Generative Model for Validity in Complex Discrete Structures
David Janz, Jos van der Westhuizen, Brooks Paige, Matt J. Kusner, José Miguel Hernández-Lobato
The International Conference on Learning Representations (ICLR), 2018.
paper

When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness
Chris Russell*, Matt J. Kusner*, Joshua R. Loftus, Ricardo Silva
Neural Information Processing Systems (NeurIPS), 2017.
paper

Counterfactual Fairness
Matt J. Kusner*, Joshua R. Loftus*, Chris Russell*, Ricardo Silva
Neural Information Processing Systems (NeurIPS), 2017. Oral Presentation.
paper | code | talk | poster

Grammar Variational Autoencoder
Matt J. Kusner*, Brooks Paige*, José Miguel Hernández-Lobato
The International Conference on Machine Learning (ICML), 2017.
paper | code | talk

Learning in the Real World: Constraints on Cost, Space, and Privacy
Matt J. Kusner
Ph.D. Thesis, 2016.
paper

Supervised Word Mover's Distance
Gao Huang*, Chuan Guo*, Matt J. Kusner, Yu Sun, Kilian Q. Weinberger, Fei Sha
Neural Information Processing Systems (NeurIPS), 2016. Oral Presentation.
paper | code | talk

Private Causal Inference
Matt J. Kusner, Yu Sun, Karthik Sridharan, Kilian Q. Weinberger
The International Conference on Artificial Intelligence and Statistics (AISTATS), 2016. Oral Presentation.
paper | code

Fast Distributed k-Center Clustering with Outliers on Massive Data
Gustavo Malkomes, Matt J. Kusner, Wenlin Chen, Kilian Q. Weinberger, Benjamin Moseley
Neural Information Processing Systems (NeurIPS), 2015.
paper

From Word Embeddings To Document Distances
Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger
The International Conference on Machine Learning (ICML), 2015.
paper | code | talk | slides | poster

Differentially Private Bayesian Optimization
Matt J. Kusner, Jacob R. Gardner, Roman Garnett, Kilian Q. Weinberger
The International Conference on Machine Learning (ICML), 2015.
paper | talk | slides | poster

Classifier Cascades and Trees for Minimizing Feature Evaluation Cost
Zhixiang (Eddie) Xu, Matt J. Kusner, Kilian Q. Weinberger, Minmin Chen, Olivier Chapelle
The Journal of Machine Learning Research (JMLR), 2014.
paper

Feature-Cost Sensitive Learning with Submodular Trees of Classifiers
Matt J. Kusner, Wenlin Chen, Quan Zhou, Zhixiang (Eddie) Xu, Kilian Q. Weinberger, Yixin Chen
AAAI Conference on Artificial Intelligence (AAAI), 2014.
paper | code | poster

Stochastic Neighbor Compression
Matt J. Kusner, Stephen Tyree, Kilian Q. Weinberger, Kunal Agrawal
The International Conference on Machine Learning (ICML), 2014.
paper | code | slides | poster

Bayesian Optimization with Inequality Constraints
Jacob R. Gardner, Matt J. Kusner, Zhixiang (Eddie) Xu, Kilian Q. Weinberger, John P. Cunningham
The International Conference on Machine Learning (ICML), 2014.
paper

Anytime Feature Learning
Zhixiang (Eddie) Xu, Matt J. Kusner, Gao Huang, Kilian Q. Weinberger
The International Conference on Machine Learning (ICML), 2013.
paper

Cost-Sensitive Tree of Classifiers
Zhixiang (Eddie) Xu, Matt J. Kusner, Kilian Q. Weinberger, Minmin Chen
The International Conference on Machine Learning (ICML), 2013.
paper

Technical Reports

A Survey on Contextual Embeddings
Qi Liu, Matt J. Kusner, Phil Blunsom
paper

Cumulo: A Dataset for Learning Cloud Classes
Valentina Zantedeschi, Fabrizio Falasca, Alyson Douglas, Richard Strange, Matt J. Kusner, Duncan Watson-Parris
NeurIPS Workshop Tackling Climate Change with Machine Learning, 2019. Best Paper Award.
paper

Causal Reasoning for Algorithmic Fairness
Joshua R. Loftus, Chris Russell, Matt J. Kusner, Ricardo Silva
paper

Deep Manifold Traversal: Changing Labels with Convolutional Features
Jacob R. Gardner*, Paul Upchurch*, Matt J. Kusner, Yixuan Li, Kilian Q. Weinberger, Kavita Bala, John E. Hopcroft
paper