How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?

Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang, in EMNLP (Short), 2022.

Code

Download the full text

Abstract

Text-to-image generative models have achieved unprecedented success in generating high-quality images based on natural language descriptions. However, it is shown that these models tend to favor specific social groups when prompted with neutral text descriptions (e.g., ’a photo of a lawyer’). Following Zhao et al. (2021), we study the effect on the diversity of the generated images when adding ethical intervention that supports equitable judgment (e.g., ’if all individuals can be a lawyer irrespective of their gender’) in the input prompts. To this end, we introduce an Ethical NaTural Language Interventions in Text-to-Image GENeration (ENTIGEN) benchmark dataset to evaluate the change in image generations conditional on ethical interventions across three social axes – gender, skin color, and culture. Through ENTIGEN framework, we find that the generations from minDALL.E, DALL.E-mini and Stable Diffusion cover diverse social groups while preserving the image quality. Preliminary studies indicate that a large change in the model predictions is triggered by certain phrases such as ’irrespective of gender’ in the context of gender bias in the ethical interventions. We release code and annotated data at https://github.com/Hritikbansal/entigen_emnlp.

Source Code

Bib Entry

@inproceedings{bansal2022how,
  title = {How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?},
  author = {Bansal, Hritik and Yin, Da and Monajatipoor, Masoud and Chang, Kai-Wei},
  booktitle = {EMNLP (Short)},
  year = {2022}
}

Related Publications

A Meta-Evaluation of Measuring LLM Misgendering, COLM 2025, 2025
White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs, ACL, 2025
Controllable Generation via Locally Constrained Resampling, ICLR, 2025
On Localizing and Deleting Toxic Memories in Large Language Models, NAACL-Finding, 2025
Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification, EMNLP-Finding, 2024
Mitigating Bias for Question Answering Models by Tracking Bias Influence, NAACL, 2024
Are you talking to ['xem'] or ['x', 'em']? On Tokenization and Addressing Misgendering in LLMs with Pronoun Tokenization Parity, NAACL-Findings, 2024
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems, EMNLP-Finding, 2023
Kelly is a Warm Person, Joseph is a Role Model: Gender Biases in LLM-Generated Reference Letters, EMNLP-Findings, 2023
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks, ACL (short), 2023
Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness, AIES, 2023
On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations, ACL (short), 2022
Societal Biases in Language Generation: Progress and Challenges, ACL, 2021
"Nice Try, Kiddo": Investigating Ad Hominems in Dialogue Responses, NAACL, 2021
BOLD: Dataset and metrics for measuring biases in open-ended language generation, FAccT, 2021
Towards Controllable Biases in Language Generation, EMNLP-Finding, 2020
The Woman Worked as a Babysitter: On Biases in Language Generation, EMNLP (short), 2019