White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs

Kai-Wei Chang Yixin Wan, in ACL, 2025.

Best paper at TrustNLP workshop at NAACL 2024

Download the full text

Abstract

Language agency is an important aspect of evaluating social biases in texts. While several studies approached agency-related bias in human-written language, very limited research has investigated such biases in Large Language Model (LLM)-generated content. In addition, previous research often relies on string-matching techniques to identify agentic and communal words within texts, which fall short of accurately classifying language agency. We introduce the novel Language Agency Bias Evaluation (LABE) benchmark, which comprehensively evaluates biases in LLMs by analyzing agency levels attributed to different demographic groups in model generations. LABE leverages 5,400 template-based prompts, an accurate agency classifier, and corresponding bias metrics to test for gender, racial, and intersectional language agency biases in LLMs on 3 text generation tasks: biographies, professor reviews, and reference letters. To build better and more accurate automated agency classifiers, we also contribute and release the Language Agency Classification (LAC) dataset, consisting of 3,724 agentic and communal sentences. Using LABE, we unveil previously under-explored language agency social biases in 3 recent LLMs: ChatGPT, Llama3, and Mistral. We observe that: (1) For the same text category, LLM generations demonstrate higher levels of gender bias than human-written texts; (2) On most generation tasks, models show remarkably higher levels of intersectional bias than the other bias aspects. Those who are at the intersection of gender and racial minority groups – such as Black females – are consistently described by texts with lower levels of agency; (3) Among the 3 LLMs investigated, Llama3 demonstrates greatest overall bias in language agency; (4) Not only does prompt-based mitigation fail to resolve language agency bias in LLMs, but it frequently leads to the exacerbation of biases in generated texts.

Bib Entry

@inproceedings{wan2024white,
  title = {White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs},
  author = {Yixin Wan, Kai-Wei Chang},
  year = {2025},
  booktitle = {ACL}
}

Related Publications

A Meta-Evaluation of Measuring LLM Misgendering, COLM 2025, 2025
Controllable Generation via Locally Constrained Resampling, ICLR, 2025
On Localizing and Deleting Toxic Memories in Large Language Models, NAACL-Finding, 2025
Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification, EMNLP-Finding, 2024
Mitigating Bias for Question Answering Models by Tracking Bias Influence, NAACL, 2024
Are you talking to ['xem'] or ['x', 'em']? On Tokenization and Addressing Misgendering in LLMs with Pronoun Tokenization Parity, NAACL-Findings, 2024
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems, EMNLP-Finding, 2023
Kelly is a Warm Person, Joseph is a Role Model: Gender Biases in LLM-Generated Reference Letters, EMNLP-Findings, 2023
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks, ACL (short), 2023
Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness, AIES, 2023
How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?, EMNLP (Short), 2022
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations, ACL (short), 2022
Societal Biases in Language Generation: Progress and Challenges, ACL, 2021
"Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses, NAACL, 2021
BOLD: Dataset and metrics for measuring biases in open-ended language generation, FAccT, 2021
Towards Controllable Biases in Language Generation, EMNLP-Finding, 2020
The Woman Worked as a Babysitter: On Biases in Language Generation, EMNLP (short), 2019