poster

Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition

Authors:

Aram GalstyanAuthors Info & Claims

HT '20: Proceedings of the 31st ACM Conference on Hypertext and Social Media

Pages 231 - 232

https://doi.org/10.1145/3372923.3404804

Published: 13 July 2020 Publication History

Get Access

Abstract

In this paper, we study the bias in named entity recognition (NER) models---specifically, the difference in the ability to recognize male and female names as PERSON entity types. We evaluate NER models on a dataset containing 139 years of U.S. census baby names and find that relatively more female names, as opposed to male names, are not recognized as PERSON entities. The result of this analysis yields a new benchmark for gender bias evaluation in named entity recognition systems. The data and code for the application of this benchmark is publicly available for researchers to use.

References

[1]

Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Association for Computational Linguistics (ACL) System Demonstrations. 55--60. http://www.aclweb.org/anthology/P/P14/P14-5010

Google Scholar

[2]

Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating Gender Bias in Natural Language Processing: Literature Review. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1630--1640.

Crossref

Google Scholar

Cited By

View all

Chen ZZhang JHort MHarman MSarro F(2024)Fairness Testing: A Comprehensive Survey and Analysis of TrendsACM Transactions on Software Engineering and Methodology10.1145/365215533:5(1-59)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3652155
Kraft ASoulier E(2024)Knowledge-Enhanced Language Models Are Not Bias-Proof: Situated Knowledge and Epistemic Injustice in AIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658981(1433-1445)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658981
Le NSadat F(2024)Enhancing Neural Machine Translation of Indigenous Languages through Gender Debiasing and Named Entity Recognition2024 16th International Conference on Human System Interaction (HSI)10.1109/HSI61632.2024.10613583(1-5)Online publication date: 8-Jul-2024
https://doi.org/10.1109/HSI61632.2024.10613583
Show More Cited By

Index Terms

Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction

Recommendations

Named entity recognition and resolution in legal text
Semantic Processing of Legal Texts

Named entities in text are persons, places, companies, etc. that are explicitly mentioned in text using proper nouns. The process of finding named entities in a text and classifying them to a semantic type, is called named entity recognition. Resolution ...
Inducing Gazetteer for Chinese Named Entity Recognition Based on Local High-Frequent Strings
FITME '09: Proceedings of the 2009 Second International Conference on Future Information Technology and Management Engineering

Gazetteers, or entity dictionaries, are important for named entity recognition (NER). Although the dictionaries extracted automatically by the previous methods from a corpus, web or Wikipedia are very huge, they also misses some entities, especially the ...
Unsupervised biomedical named entity recognition

Display Omitted BM-NER is approached by an unsupervised stepwise method.Noun phrase chunking is a good approximation of boundary detection.Distributional semantics works well in classifying entities.The system performs well on clinical and biological ...

Comments

Information & Contributors

Information

Published In

HT '20: Proceedings of the 31st ACM Conference on Hypertext and Social Media

July 2020

327 pages

ISBN:9781450370981

DOI:10.1145/3372923

General Chairs:
Anastasia Salter
University of Central Florida, USA
,
John F. Barber
Washington State University Vancouver, USA
,
Program Chair:
Ujwal Gadiraju
Delft University of Technology, The Netherlands

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2020

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

Defense Advanced Research Projects Agency (DARPA)

Conference

HT '20

Sponsor:

HT '20: 31st ACM Conference on Hypertext and Social Media

July 13 - 15, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 378 of 1,158 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
540
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)4

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chen ZZhang JHort MHarman MSarro F(2024)Fairness Testing: A Comprehensive Survey and Analysis of TrendsACM Transactions on Software Engineering and Methodology10.1145/365215533:5(1-59)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3652155
Kraft ASoulier E(2024)Knowledge-Enhanced Language Models Are Not Bias-Proof: Situated Knowledge and Epistemic Injustice in AIProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658981(1433-1445)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658981
Le NSadat F(2024)Enhancing Neural Machine Translation of Indigenous Languages through Gender Debiasing and Named Entity Recognition2024 16th International Conference on Human System Interaction (HSI)10.1109/HSI61632.2024.10613583(1-5)Online publication date: 8-Jul-2024
https://doi.org/10.1109/HSI61632.2024.10613583
Nenno S(2024)Bootstrapping public entities. Domain-specific NER for public speakersCommunication Methods and Measures10.1080/19312458.2024.2388098(1-26)Online publication date: 13-Aug-2024
https://doi.org/10.1080/19312458.2024.2388098
Balluff PBoomgaarden HWaldherr A(2024)Automatically Finding Actors in Texts: A Performance Review of Multilingual Named Entity Recognition ToolsCommunication Methods and Measures10.1080/19312458.2024.2324789(1-19)Online publication date: 19-Mar-2024
https://doi.org/10.1080/19312458.2024.2324789
Xiao YLim SPollard TGhassemi M(2023)In the Name of Fairness: Assessing the Bias in Clinical Record De-identificationProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3593982(123-137)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3593982
Das SPaik J(2023)Gender tagging of named entities using retrieval‐assisted multi‐context aggregation: An unsupervised approachJournal of the Association for Information Science and Technology10.1002/asi.2473574:4(461-475)Online publication date: 27-Jan-2023
https://doi.org/10.1002/asi.24735
Hoque MGhai BElmqvist N(2022)DramatVis Personae: Visual Text Analytics for Identifying Social Biases in Creative WritingProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533526(1260-1276)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533526
Agarwal SMuku SAnand SArora C(2022)Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Bias2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV51458.2022.00395(3898-3907)Online publication date: Jan-2022
https://doi.org/10.1109/WACV51458.2022.00395
Calix RBen-Joseph JLopatina NAshley RGogia MSieniawski GBrennen A(2022)Saisiyat Is Where It Is At! Insights Into Backdoors And Debiasing Of Cross Lingual Transformers For Named Entity Recognition2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020403(2940-2949)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020403
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Named entity recognition and resolution in legal text

Inducing Gazetteer for Chinese Named Entity Recognition Based on Local High-Frequent Strings

Unsupervised biomedical named entity recognition