Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification

Yada Pruksachatkun, Satyapriya Krishna, Jwala Dhamala, Rahul Gupta, and Kai-Wei Chang, in ACL-Finding, 2021.

Code

Download the full text

Abstract

Existing bias mitigation methods to reduce disparities in model outcomes across cohorts have focused on data augmentation, debiasing model embeddings, or adding fairness-based optimization objectives during training. Separately, certified word substitution robustness methods have been developed to decrease the impact of spurious features and synonym substitutions on model predictions. While their end goals are different, they both aim to encourage models to make the same prediction for certain changes in the input. In this paper, we investigate the utility of certified word substitution robustness methods to improve equality of odds and equality of opportunity on multiple text classification tasks. We observe that certified robustness methods improve fairness, and using both robustness and bias mitigation methods in training results in an improvement in both fronts.

Source Code

Bib Entry

@inproceedings{pruksachatkun2021robustness,
  title = {Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification},
  author = {Pruksachatkun, Yada and Krishna, Satyapriya and Dhamala, Jwala and Gupta, Rahul and Chang, Kai-Wei},
  booktitle = {ACL-Finding},
  year = {2021}
}

Related Publications

Measuring Fairness of Text Classifiers via Prediction Sensitivity, ACL, 2022
LOGAN: Local Group Bias Detection by Clustering, EMNLP (short), 2020
Towards Understanding Gender Bias in Relation Extraction, ACL, 2020
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization, ACL (short), 2020
Mitigating Gender in Natural Language Processing: Literature Review, ACL, 2019
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, NAACL (short), 2018
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, EMNLP, 2017