Graham Neubig

I am an Associate Professor at the Carnegie Mellon University Language Technology Institute in the School of Computer Science, and work with a bunch of great students in my lab NeuLab. I am also chief scientist at All Hands AI, where we are building AI agents for software development.

Prospective Students/Visitors: Please see the contact info page for more info.

Research

My research focuses on machine learning and natural language processing. In particular, I am interested in basic research and applications of large language models, with a particular focus on question answering, code generation, multilingual processing, and evaluation/interpretability. You can find more details, research page or publications page, and reference my third-person bio for talk introductions, etc..

I like developing, and have created open-source software which I have summarized here: software page. I also post class and tutorial notes teaching page in the hope that people will find them useful. I also talk about research on Twitter occasionally.

What's New?

2024-10-16: Gave a talk at Berkeley and Stanford on "Agents for Software Development".
2023-04-01: Gave a talk at the CMU LTI Large Language Model Seminar on "Large Language Models: a Birds-eye View".
2023-02-27: Gave an updated talk at UIUC and UPenn on "Is My NLP Model Working? The Answer is Harder Than You Think".
2022-11-3: Gave a talk at ODSC West on "Is My NLP Model Working? The Answer is Harder Than You Think".
2022-7-13: Gave three invited talks at NAACL workshops on "Can we Automatically Create Language-learning Textbooks?", "Unlocking Resources for Under-resourced Languages", and "GlobalBench: A Benchmark for Global Progress in Natural Language Processing"
2021-7-21: Gave an invited talk at the Lisbon Machine Learning School on "How Can We Know What and When Language Models Know?".
2020-11-20: Gave an invited talk at the SustaiNLP workshop on "More is Less: Nonparametric Models and Efficiency".
2020-9-21: Gave an invited talk at the New York Circle of Translators meeting on "Artificial Intelligence, Machine Translation, and Future Linguists Like You".
2020-4-26: Gave an invited talk at the ICLR workshop AfricaNLP on "The Low-resource NLP Toolbox, 2020 Version".
2019-6-6: Gave an invited talk at NAACL workshop NeuralGen on "What can Statistical Machine Translation Teach Neural Models about Generation?".
2019-5-6: I will be giving two invited talks at ICLR 2019 workshops on "What can Statistical Machine Translation Teach Neural Machine Translation about Structured Prediction?" at the structured prediction workshop, and "Learning about Language with Normalizing Flows" at the deep generative models workshop.
2019-4-25: Our work on computer assisted interpretation was covered in Slator
2018-11-1: I posted my slides from my talk at the Black Box NLP Workshop on Learning with Latent Linguistic Structure
2018-10-24: I posted my slides from my talk at UT Austin on Towards Open-domain Generation of Programs from Natural Language
2018-6-18: We released CoNaLa, a large dataset for natural language to code generation. Check it out!
2018-6-1:NeuLab had 6 papers, 1 tutorial, and 1 invited talk at NAACL! See the details on my lab site.
2018-4-12: Gave a talk at the NYU Text as Data Seminar Series on "What Can Neural Networks Teach us about Language?".
2017-12-9: Gave a keynote talk at the How to Code a Paper workshop at NIPS, on "Simple and Efficient Learning in Dynamic Networks".
2017-12-4: Check out our 2 papers at NIPS 2017!
2017-11-18: Gave a keynote talk at the Ann Arbor Deep Learning Event a2-dlearn, on "What Can Neural Networks Teach us about Language?".
2017-7-30: NeuLab members have a total of 10 papers at ACL2017 and Workshops! Come to our presentations and check out the papers.
2017-7-26: I gave a talk at the Lisbon Machine Learning School on Simple and Effective Learning with Dynamic Neural Networks.
2017-5-24: I gave a talk at the Machine Translation Marathon in the Americas on Softmax Alternatives for Neural MT.
2017-3-12: We'll be presenting three papers at EACL on cross-lingual word embeddings for language modeling, real-time translation with neural machine translation, and learning about syntax with recurrent neural network grammars. Adhi's paper on RNNGs won the Outstanding Paper Award, congratulations!
2017-3-5: I wrote a tutorial on neural machine translation and sequence to sequence models, which should be good for beginners who are just getting into the field.
2017-1-16: We wrote a technical report on DyNet, our neural network toolkit. We describe the inner workings, and do speed benchmarks against other popular toolkits such as TensorFlow, Theano, and Chainer. Take a look!
2017-1-5: We'll be holding the First Workshop on Neural Machine Translation at ACL2017. Lots of great speakers and hopefully an interesting panel discussion, so please join if you're interested!
2016-11-1: Our paper on learning lexicons from speech will receive the Best Short Paper Award at EMNLP!
2016-11-1: Four papers accepted to EMNLP on combining neural and n-gram language models, incorporating lexicons into neural machine translation, controlling output length in encoder-decoders, and learning lexicons from parallel speech.
2016-10-20: Our Japanese-English system for the Workshop on Asian Translation achieved the best results on the task!
2016-9-12: One paper accepted to Speech Communication on learning persuasive dialog systems using framing.
2016-8-10: One paper on continuous-space rule selection for syntax-based machine translation to be presented at ACL!
2016-6-28: I'll be giving an invited talk on automatic programming at the AIST artificial intelligence seminar.
2016-6-13: Two papers to be presented at NAACL-HLT on morphological inflection generation, and segment-based active learning for machine translation!
2016-5-30: I'll be giving an invited talk on language modeling at the TITECH/Ochadai NLP seminar.
2016-3-12: I'll be giving an invited talk at the IPSJ-ONE event sponsored by the Information Processing Society of Japan.
2016-3-6: A book (in Japanese) that I co-authored with Masato Hagiwara and Yoh Okuno on "Natural Language Processing: Fundamentals and Techniques" will be going on sale!
2016-3-3: Our Computational Linguistics survey on optimization for machine translation has been put online. Take a look if you want to know more about machine learning techniques for MT.
2016-2-16: I wrote a chapter on machine translation for the new Iwanami data science book series on natural language processing.
2016-2-11: I gave an invited talk at the Karlsruhe Institute of Technology on simultaneous speech translation.
2016-1-15: Paper accepted to the IEEE Transactions on Audio Speech and Language Processing on using the modulation spectrum for speech synthesis!
2016-1-13: Presented 3 papers at IWSDS on dialog-based deception detection, active learning for constructing dialog systems, and LSTM-based dialog state tracking.
2015-12-13: Presented 4 papers at ASRU on user-adaptive dialogue, sentence compression using LSTMs, affective communication, and our speech recognition system for the MGB challenge!
2015-12-2: Presented 4 papers at IWSLT on parser self-training for machine translation, learning lexicons from phoneme transcriptions, translation of prosodic emphasis, and the NAIST speech recognition system.
2015-11-9: Presented 2 papers at ASE on automatically generating pseudo-code from source code, and the corresponding tool, which can be found here!
2015-11-7: Giving a talk on Simultaneous Speech Translation at several universities: CMU, JHU, USC, and UMD.
2015-10-16: Our system at the 2nd workshop on Asian translation got 1st place in all 4 main language pairs! See our paper for details, and an analysis of why neural network reranking helps machine translation.
2015-10-8: A paper was accepted to TACL on semantic parsing of ambiguous input through paraphrasing and verification.
2015-10-1: Started a 4 months as a visiting scholar at CMU, working with Chris Dyer.