Natural Language Processing Section, Department of Computer Science, University of Copenhagen

CopeNLU is a Natural Language Processing research group led by Isabelle Augenstein and Pepa Atanasova with a focus on researching methods for tasks that require a deep understanding of language, as opposed to shallow processing. We are affiliated with the Natural Language Processing Section, as well as with the Pioneer Centre for AI, at the Department of Computer Science, University of Copenhagen. We are interested in core methodology research on, among others, learning with limited training data and explainable AI; as well as applications thereof to tasks such as fact checking, gender bias detection and question answering. Our group is partly funded by an ERC Starting Grant on Explainable and Robust Automatic Fact Checking, as well as a Sapere Aude Research Leader fellowship on `Learning to Explain Attitudes on Social Media’.

Interests

Natural Language Understanding
Learning with Limited Labelled Data
Explainable AI
Fact Checking
Question Answering
Gender Bias Detection

News

More Talks

5 Papers Accepted to EMNLP 2024

12 Nov 2024

The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)

5 papers by CopeNLU authors are accepted to appear at EMNLP 2024, on topics including factuality and probing for bias.

PhD fellowship on Interpretable Machine Learning available

8 Nov 2024

PhD Fellowship Call

PhD fellowship on Interpretable Machine Learning available. The successfull candidate will be supervised by Pepa Atanasova and Isabelle Augenstein.

Project Project

Pepa has been appointed as a Tenure-Track Assistant Professor

1 Sep 2024

Pepa appointed as a Tenure-Track Assistant Professor

Starting in September 2024, Pepa is taking on a new role as Tenure-Track Assistant Professor in the NLP Section at the University of Copenhagen.

Project Project

Participate in research on explainable fact checking

1 Jun 2024

We are recruiting professional fact checkers to take part in an interview and/or a survey about their experiences of fact checking and fact checking technologies.

Project Project

Oustanding paper award at EACL 2024

17 Mar 2024

The 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024)

Our paper on measuring the fragility of natural language inference models has won an outstanding paper award at EACL 2024!

PDF

PhD and postdoc positions available at Pioneer Centre for AI

1 Mar 2024

Pioneer Centre PhD and Postdoc Call

A PhD and two postdoc positions on natural language understanding are available. The positions are funded by the Pioneer Centre for AI.

Project Project

5 Papers Accepted to EMNLP 2023

6 Dec 2023

The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)

5 papers by CopeNLU authors are accepted to appear at EMNLP 2023, ranging from explainability to language modelling.

PhD position available in context of ERC Starting Grant project ExplainYourself

7 Nov 2023

ERC StG PhD Fellowship Call

A PhD position on explainable natural language understanding is available in CopeNLU. The positions is funded by the ERC Starting Grant project ExplainYourself, and applications are due by 1 February 2024.

Project Project

ExplainYourself Project Kick-Off

1 Sep 2023

ERC Starting Grant ExplainYourself

On 1 September 2023, the ERC Starting Grant project ExplainYourself on ‘Explainable and Robust Automatic Fact Checking’ is kicking off, and two new PhD students are joining our group.

Project Project

4 Papers Accepted to ACL 2023

22 May 2023

61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)

4 papers by CopeNLU authors are accepted to appear at ACL 2023, on faithfulness of explanations, measuring intersectional biases, event extraction and few-shot stance detection.

People

Isabelle Augenstein

Full Professor

Isabelle’s main research interests are natural language understanding, explainability and learning with limited training data.

Pepa Atanasova

Assistant Professor

Pepa’s research interests include the development, diagnostics, and application of explainability and interpretability techniques for NLP models.

Erik Arakelyan

PhD Student

Erik’s main research interests are question answering and explainability.

Nadav Borenstein

PhD Student

Nadav’s research interests include improving the trustworthiness and usefulness of deep models in the NLP domain.

Arnav Arora

PhD Student

Arnav’s research interests include equitable ML, mitigating online harms, and the intersection of NLP and Computational Social Science.

Haeun Yu

PhD Student

Haeun’s main research interests include enhancing explainability in fact-checking and transparency of knowledge-enhanced LM.

Jingyi Sun

PhD Student

Jingyi Sun’s research interests include explainability, fact-checking, and question answering.

Lovisa Hagström

PhD Intern

Lovisa is a visiting PhD student from Chalmers University of Technology. Her research interests include language model controllability, interpretability and retrieval-augmented methods.

Siddhesh Pawar

PhD Student

Siddhesh Pawar’s research interests include multilingual models, fairness and accountability in NLP systems.

Dustin Wright

Postdoc

Dustin is a DDSA postdoctoral fellow, working on scientific natural language understanding and faithful text generation.

Ercong Nie

PhD Intern

Ercong’s research interests include multilingual NLP, efficient methods for NLP, human-inspired NLP, and interpretability.

Steffen Eger

Visiting Professor

Steffen Eger is Heisenberg and BMBF group leader at University of Mannheim, Germany, working mainly on text generation and its evaluation. He was visiting CopeNLU in Winter/Spring 2024.

Ran Zhang

PhD Intern

Ran Zhang is a PhD student at Mannheim University, working on text generation and its evaluation. She was visiting CopeNLU in Winter/Spring 2024.

Christoph Leitner

PhD Intern

Christoph Leitner is a PhD student at Mannheim University, working on explainable text generation metrics. He was visiting CopeNLU in Winter/Spring 2024.

Greta Warren

Postdoc

Greta’s research interests include user-centred explainability, fact-checking, and human-AI interaction.

Shuzhou Yuan

PhD Intern

Shuzhou Yuan is a PhD student at Karlsruhe Institute of Technology and Dresden University of Technology, working on the application of graph neural networks for natural language processing. He was visiting CopeNLU in Spring 2024.

Sekh Mainul Islam

PhD Student

Sekh’s research interests include explainability in fact checking and improving robustness and trustworthiness in NLP models.

Sarah Masud

Postdoc

Sarah broadly works in the area of computational social systems with a focus on news narrative and hate speech modelling. Her PhD at IIIT-Delhi was supported by fellowships from Google and PMRF.

Zain Muhammad Mujahid

PhD Student

Zain’s main research interests include disinformation detection, fact-checking, and factual text generation.

Ryan Cotterell

Lecturer

This is Ryan. He’s a lecturer at the University of Cambridge and a frequent collaborator of the CopeNLU group.

Sara Vera Marjanovic

PhD Student

Sara’s research interests include explainable IR and NLP models, identifying biases in large text datasets, as well as working with social media data. She is a member of the DIKU ML section and IR group and co-advised by Isabelle.

Amalie Brogaard Pauli

PhD Student

Amalie’s research focuses on detecting persuasive and misleading text. She is a PhD student at Aarhus University and co-advised by Isabelle.

Lucas Resck

PhD Student

Lucas is an ELLIS PhD student at the University of Cambridge, supervised by Anna Corhonen and co-supervised by Isabelle. His research interests include machine learning, NLP and explainability.

Ahmad Dawar Hakimi

PhD Student

Dawar is an ELLIS PhD student at LMU Munich, supervised by Hinrich Schütze and co-supervised by Isabelle. His research interests include mechanistic interpretability, summarisation and factuality of LLMs.

Karolina Stanczak

PhD Student

Karolina’s research interests include gender-biased language detection and statistical methods. She successfully defended her PhD thesis in January 2024, and is now a postdoctoral researcher at Mila, Montreal.

Marta Marchiori Manerba

PhD Intern

Marta’s research interests are explainability and fairness, with a focus on abusive language detection. She is a second-year PhD student at the University of Pisa and was visiting CopeNLU in Spring/Summer 2023.

Kevin Roitero

Visiting Researcher

Kevin Roitero is an Assistant Professor (RTD/a) at the University of Udine, Italy. His research interests include Artificial Intelligence, Deep Learning, Crowdsourcing, and Information Retrieval. He was visiting CopeNLU in Spring/Summer 2023.

Nils Rethmeier

PhD Student

Nils researches low-resource learning and unsupervised learning as well as explainability. He successfully defended his PhD thesis in August 2023.

Amelie Wührl

PhD Intern

Amelie is a PhD Student at the University of Stuttgart, interested in fact-checking, specifically for medical claims and content that people chose to share on social media. She was visiting CopeNLU in Spring 2023 to work on scientific fact checking.

Lucie-Aimée Kaffee

Postdoc

Lucie’s research interests include supporting lower-resourced language communities (including Wikipedia and Wikidata) with NLP, and multilingual knowledge graphs. She was a postdoctoral researcher in the CopeNLU group from 2021-2023 and is now a postdoctoral researcher at Hasso-Plattner-Institut.

Oscar Kjell

Postdoc

Oscar researches automating the assessment of psychological constructs using representation learning. He is an international postdoc affiliated with Lund University as well the University of Copenhagen and Stony Brook University.

Andreas Nugaard Holm

PhD Student

Andreas’ main research areas are representation learning and domain adaptation, with a focus on scientific texts.

Thea Rolskov Sloth

Intern

Thea is a MSc Student in Cognitive Science at Aarhus University, visiting CopeNLU in Autumn 2022 to work on gender bias detection.

Miryam de Lhoneux

Postdoc

Miryam’s main research interests are syntactic parsing, multilingual NLP and interpretability. She is an assistant profesor at KU Leuven, and was previously an international postdoc affiliated with KU Leuven as well as the University of Copenhagen.

Nodens Koren

PhD Student

Nodens is interested in interpretability and bio-inspired models.

Klim Zaporojets

PhD Intern

Klim is a PhD Student at Ghent University. He was visiting CopeNLU in Spring 2022 to work on entity linking.

Sagnik Ray Choudhury

Postdoc

Sagnik is a postdoctoral researcher at the University of Michigan. He was a postdoctoral researcher in CopeNLU from 2020 to 2022, working on question answering and interpretability of neural black-box models.

Shailza Jolly

PhD Intern

Shailza is a PhD Student at the Technical University of Kaiserslautern, and a research assistant at DFKI. She was visiting CopeNLU in Winter/Spring 2021 to work on interpretability.

Yova Kementchedjhieva

PhD Student

Yova was a PhD student in the CoAStaL NLP group researching low-resource and cross-lingual learning, co-advised by Isabelle. She is now a postdoc in the same group.

Ana Valeria González

PhD Student

Ana is a senior data scientist at Halfspace, Copenhagen. Prior to this, she was a PhD student in the CoAStaL NLP group from 2017-2021 co-advised by Isabelle, working on question answering.

Andrea Lekkas

PhD Student

Andrea’s main research interests are multilingual learning and language modelling. He was a PhD student at the University of Southern Denmark and co-advised by Isabelle.

Johannes Bjerva

Postdoc

Johannes is an associate professor at Aalborg University Copenhagen. He was a postdoc in CopeNLU from 2017 to 2020, researching multi-lingual and multi-task learning.

Liesbeth Allein

PhD Intern

Liesbeth is a PhD Student at KU Leuven, and was visiting CopeNLU in Spring 2020 to work on fact checking.

Wei Zhao

PhD Intern

Wei Zhao is a postdoctoral researcher at Heidelberg University, and was visiting CopeNLU in Winter 2019 to work on low-resource natural language generation.

Mareike Hartmann

PhD Student

Mareike was a member of the CoAStaL NLP group, researching disinformation detection and co-advised by Isabelle, and is now a researcher at DFKI.

Farhad Nooralahzadeh

PhD Intern

Farhad is a currently postdoc at the University of Zurich, and was visiting CopeNLU in Spring 2019 to work on domain adaptation for information extraction.

Zhong Xuan

Research Intern

Zhong Xuan is a data scientist at Grab, Singapore, and was visiting CopeNLU in Summer 2019 to work on relation extraction and knowledge base population.

Luna De Bruyne

PhD Intern

Luna is a PhD Student at Ghent University, and was visiting CopeNLU in Spring 2019 to work on emotion detection.

Giannis Bekoulis

PhD Intern

Giannis is a principal data scientist at Clarivate, and was visiting CopeNLU as a PhD Intern in Spring 2019 to work on joint information extraction.

Sune Debel

Research Assistant

Now a machine learning engineer at Omhu.

Featured Publications

Factuality Challenges in the Era of Large Language Models

The emergence of tools based on large language models (LLMs), like OpenAI’s ChatGPT and Google’s Gemini, has garnered immense public attention due to their advanced natural language generation capabilities. These remarkably natural-sounding tools have the potential to be highly useful across various tasks. However, they also tend to produce false, erroneous, or misleading content – commonly referred to as hallucinations. Additionally, LLMs can be misused to generate convincing yet false content and profiles on a large scale, posing a substantial societal challenge by potentially deceiving users and spreading inaccurate information. This makes fact-checking increasingly important. Despite their issues with factual accuracy, LLMs have shown proficiency in various subtasks that support fact-checking, which is essential for ensuring factually accurate responses. In light of these concerns, we explore the issues related to factuality in LLMs and their impact on fact-checking. We identify key challenges, imminent threats, and possible solutions to these factuality issues. We also thoroughly examine these challenges, existing solutions, and potential prospects for fact-checking. By analysing the factuality constraints within LLMs and their impact on fact-checking, we aim to contribute to a path towards maintaining accuracy at a time of confluence of generative AI and misinformation.

Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, Eduard Hovy, Heng Ji, Filippo Menczer, Ruben Miguez, Preslav Nakov, Dietram Scheufele, Shivam Sharma, Giovanni Zagni

In Nature Machine Intelligence, 2024

PDF Project Project

Recent Publications

More Publications

Survey of Cultural Awareness in Language Models: Text and Beyond

Large-scale deployment of large language models (LLMs) in various applications, such as chatbots and virtual assistants, requires LLMs …

Siddhesh Milind Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, Isabelle Augenstein

PDF Project

FLARE: Faithful Logic-Aided Reasoning and Exploration

Modern Question Answering (QA) and Reasoning approaches based on Large Language Models (LLMs) commonly use prompting techniques, such …

Erik Arakelyan, Pasquale Minervini, Pat Verga, Patrick Lewis, Isabelle Augenstein

PDF Project Project

Can Transformers Learn n-gram Language Models?

Much theoretical work has described the ability of transformer language models (LMs) to represent formal languages.However, linking …

Anej Svete, Nadav Borenstein, Mike Zhou, Isabelle Augenstein, Ryan Cotterell

PDF Project

DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Models

Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent …

Sara Vera Marjanović, Haeun Yu, Pepa Atanasova, Maria Maistro, Christina Lioma, Isabelle Augenstein

PDF Project Project

Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-Checkers

The increased use of large language models (LLMs) across a variety of real-world applications calls for mechanisms to verify the …

Yuxia Wang, Revanth Gangi Reddy, Zain Muhammad Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, Isabelle Augenstein, Iryna Gurevych, Preslav Nakov

PDF Project Project

Revealing Fine-Grained Values and Opinions in Large Language Models

Uncovering latent values and opinions in large language models (LLMs) can help identify biases and mitigate potential harm. Recently, …

Dustin Wright, Arnav Arora, Nadav Borenstein, Srishti Yadav, Serge Belongie, Isabelle Augenstein

PDF Project Project

Social Bias Probing: Fairness Benchmarking for Language Models

Large language models have been shown to encode a variety of social biases, which carries the risk of downstream harms. While the …

Marta Marchiori Manerba, Karolina Stańczak, Riccardo Guidotti, Isabelle Augenstein

PDF Project

Claim Verification in the Age of Large Language Models: A Survey

The large and ever-increasing amount of data available on the Internet coupled with the laborious task of manual claim and fact …

Alphaeus Dmonte, Roland Oruche, Marcos Zampieri, Prasad Calyam, Isabelle Augenstein

PDF Project Project

Factuality Challenges in the Era of Large Language Models

The emergence of tools based on large language models (LLMs), like OpenAI’s ChatGPT and Google’s Gemini, has garnered immense public …

PDF Project Project

Grammatical Gender's Influence on Distributional Semantics: A Causal Perspective

How much meaning influences gender assignment across languages is an active area of research in modern linguistics and cognitive …

Karolina Stańczak, Kevin Du, Adina Williams, Isabelle Augenstein, Ryan Cotterell

PDF Project Project