Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases

Jen-tse Huang, Yuhang Yan, Linqi Liu, Yixin Wan, Wenxuan Wang, Kai-Wei Chang, and Michael R. Lyu, in EMNLP-Finding, 2025.

Code

Download the full text

Abstract

Instances such as misrepresentative images generated by AI illustrate how outputs can be factually plausible yet socially harmful. Existing fairness benchmarks conflate factual correctness and normative fairness, leading to ambiguous evaluations. This paper argues for distinguishing fact and fairness when assessing bias and introduces the Fact-or-Fair benchmark containing objective queries aligned with fact-based judgments and subjective queries aligned with fairness-based judgments. The queries draw on cognitive psychology biases and experiments across frontier models reveal different fact-fair trade-offs. The authors provide both a theoretical lens and a practical benchmark to advance responsible model.

Source Code

Bib Entry

@inproceedings{huang2025where,
  title = {Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases},
  author = {Huang, Jen-tse and Yan, Yuhang and Liu, Linqi and Wan, Yixin and Wang, Wenxuan and Chang, Kai-Wei and Lyu, Michael R.},
  booktitle = {EMNLP-Finding},
  year = {2025}
}