Share this page:

Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang, in NAACL (short), 2018.

Top-10 cited paper at NAACL 18

Poster Code

Download the full text

Abstract

In this paper, we introduce a new benchmark for co-reference resolution focused on gender bias, WinoBias. Our corpus contains Winograd-schema style sentences with entities corresponding to people referred by their occupation (e.g. the nurse, the doctor, the carpenter). We demonstrate that a rule-based, a feature-rich, and a neural coreference system all link gendered pronouns to pro-stereotypical entities with higher accuracy than anti-stereotypical entities, by an average difference of 21.1 in F1 score. Finally, we demonstrate a data-augmentation approach that, in combination with existing word-embedding debiasing techniques, removes the bias demonstrated by these systems in WinoBias without significantly affecting their performance on existing datasets.

Jieyu comes on the NLP Highlights Podcast to discuss this paper.

Bib Entry

@inproceedings{zhao2018gender,
  author = {Zhao, Jieyu and Wang, Tianlu and Yatskar, Mark and Ordonez, Vicente and Chang, Kai-Wei},
  title = {Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods},
  booktitle = {NAACL (short)},
  press_url = {https://www.stitcher.com/podcast/matt-gardner/nlp-highlights/e/55861936},
  year = {2018}
}

Related Publications

  1. Measuring Fairness of Text Classifiers via Prediction Sensitivity, ACL, 2022
  2. Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification, ACL-Finding, 2021
  3. LOGAN: Local Group Bias Detection by Clustering, EMNLP (short), 2020
  4. Towards Understanding Gender Bias in Relation Extraction, ACL, 2020
  5. Mitigating Gender Bias Amplification in Distribution by Posterior Regularization, ACL (short), 2020
  6. Mitigating Gender in Natural Language Processing: Literature Review, ACL, 2019
  7. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, EMNLP, 2017