LOGAN: Local Group Bias Detection by Clustering

Jieyu Zhao and Kai-Wei Chang, in EMNLP (short), 2020.

Code

Download the full text

Abstract

Machine learning techniques have been widely used in natural language processing (NLP). However, as revealed by many recent studies, machine learning models often inherit and amplify the societal biases in data. Various metrics have been proposed to quantify biases in model predictions. In particular, several of them evaluate disparity in model performance between protected groups and advantaged groups in the test corpus. However, we argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. In fact, a model with similar aggregated performance between different groups on the entire data may behave differently on instances in a local region. To analyze and detect such local bias, we propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region and allows us to better analyze the biases in model predictions.

1/2 Existing studies on measuring bias often consider performance gap between cohorts over the entire test set. But, does it show the whole story? In our #EMNLP20 paper “LOGAN: Local Group Bias Detection by Clustering” (https://t.co/wnpL4Ern8l), joint work with @kaiwei_chang,
— Jieyu Zhao (@jieyuzhao11) November 16, 2020

Source Code

Bib Entry

@inproceedings{zhao2020logan,
  author = {Zhao, Jieyu and Chang, Kai-Wei},
  title = {LOGAN: Local Group Bias Detection by Clustering},
  booktitle = {EMNLP (short)},
  presentation_id = {https://virtual.2020.emnlp.org/paper_main.2886.html},
  year = {2020}
}

Related Publications

Measuring Fairness of Text Classifiers via Prediction Sensitivity, ACL, 2022
Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification, ACL-Finding, 2021
Towards Understanding Gender Bias in Relation Extraction, ACL, 2020
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization, ACL (short), 2020
Mitigating Gender in Natural Language Processing: Literature Review, ACL, 2019
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, NAACL (short), 2018
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, EMNLP, 2017