Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases

Jen-tse Huang, Yuhang Yan, Linqi Liu, Yixin Wan, Wenxuan Wang, Kai-Wei Chang, and Michael R. Lyu, in EMNLP-Finding, 2025.

Code

Download the full text

Abstract

Instances such as misrepresentative images generated by AI illustrate how outputs can be factually plausible yet socially harmful. Existing fairness benchmarks conflate factual correctness and normative fairness, leading to ambiguous evaluations. This paper argues for distinguishing fact and fairness when assessing bias and introduces the Fact-or-Fair benchmark containing objective queries aligned with fact-based judgments and subjective queries aligned with fairness-based judgments. The queries draw on cognitive psychology biases and experiments across frontier models reveal different fact-fair trade-offs. The authors provide both a theoretical lens and a practical benchmark to advance responsible model.

Source Code

Bib Entry

@inproceedings{huang2025where,
  title = {Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases},
  author = {Huang, Jen-tse and Yan, Yuhang and Liu, Linqi and Wan, Yixin and Wang, Wenxuan and Chang, Kai-Wei and Lyu, Michael R.},
  booktitle = {EMNLP-Finding},
  year = {2025}
}

Related Publications

The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects, ACL, 2025
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images, NeurIPS (Datasets and Benchmarks Track), 2024
The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention, EMNLP, 2024
MACAROON: Training Vision-Language Models To Be Your Engaged Partners, EMNLP-Finding, 2024
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond, EMNLP-Findings, 2023
Resolving Ambiguities in Text-to-Image Generative Models, ACL, 2023
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding, ACL-Finding, 2023