Eve Fleisig

PhD student at UC Berkeley | Ethical NLP

I'm a fourth-year PhD student at UC Berkeley, advised by Dan Klein. My research lies at the intersection of natural language processing (NLP) and AI ethics: how can we create NLP systems that we trust to work for all users, without perpetuating societal harms? Specifically, I have worked on measuring & mitigating harms in text generated by language models, improving evaluation, and training models that account for different user perspectives or data annotators who disagree.

I did my undergraduate work at Princeton University, advised by Christiane Fellbaum, where I worked on the psycholinguistics of bilingualism and fairness in NLP.

My research is supported by an NSF Graduate Research Fellowship.

Selected Work

Please see Google or Semantic Scholar for an up-to-date publication list.

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect
Discrimination

Eve Fleisig*, Genevieve Smith*, Madeline Bossi*, Ishita Rustagi*, Xavier Yin*, Dan Klein (EMNLP 2024).

[ ARXIV ]

Mapping Social Choice Theory to RLHF

Jessica Dai, Eve Fleisig (R2FM @ ICLR 2024).

[ ARXIV ] [ TWITTER ]

When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks

Eve Fleisig, Rediet Abebe, Dan Klein (EMNLP 2023).

[ ARXIV ] [ TWITTER ]

Incorporating Worker Perspectives into MTurk Annotation Practices for NLP

Olivia Huang, Eve Fleisig, Dan Klein (EMNLP 2023 - Outstanding Paper Award).

[ ARXIV ]

Ghostbuster: Detecting Text Ghostwritten by Large Language Models

Vivek Verma, Eve Fleisig, Nicholas Tomlin, Dan Klein

[ ARXIV ] [ TWITTER ] [ BLOG ]

Hedges and Apologies in ChatGPT Responses to African-American English

Eve Fleisig (NWAV 2023).

[ SLIDES ] [ ABSTRACT ]

First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models

Naomi Saphra, Eve Fleisig, Kyunghyun Cho, Adam Lopez

[ ARXIV ] [ TWITTER ]

Centering the Margins: Outlier-Based Identification of Harmed Populations in Toxicity Detection

Vyoma Raman*, Eve Fleisig*, Dan Klein (EMNLP 2023).

[ ARXIV ] [ TWITTER ]

FairPrism: Evaluating fairness-related harms in text generation

Eve Fleisig, Aubrie Amstutz, Chad Atalla, Su Lin Blodgett, Hal Daumé III, Alexandra Olteanu, Emily Sheng, Dan Vann, Hanna Wallach (ACL 2023).

[ PAPER ] [ DATASET ]

Mitigating Gender Bias in Machine Translation through Adversarial Learning

Eve Fleisig, Christiane Fellbaum
Outstanding Senior Thesis Award; Sigma Xi Book Award

[ ARXIV ]

Mentorship

I've mentored some wonderful undergraduate students, including Harbani Jaggi, Kayla Lee, Kashyap Murali, Mahathi Ryali, Zaina Shaik, Vivek Verma, Xavier Yin, Olivia Huang (→Citadel), and Vyoma Raman (→Stanford).

If you are an undergraduate interested in working together, please email me your resume, a description of your interests, and why you're interested in working with me.

Interested in chatting or collaborating?

Reach out to me at efleisig :at: berkeley :dot: edu

Miscellaneous

My surname is pronounced /'flʌɪsɪg/ ("fly"-sihg).

I'm a member of ACF, a volunteer-run organization that produces high-quality collegiate quizbowl tournaments.

🇦🇷¡Siempre estoy feliz de charlar con otros latinoamericanos!