| Title: |
All Should Be Equal in the Eyes of LMs: Counterfactually Aware Fair Text Generation |
| Authors: |
Banerjee, Pragyan; Java, Abhinav; Jandial, Surgan; Shahid, Simra; Furniturewala, Shaz; Krishnamurthy, Balaji; Bhatia, Sumit |
| Source: |
Proceedings of the AAAI Conference on Artificial Intelligence; Vol. 38 No. 16: AAAI-24 Technical Tracks 16; 17673-17681 ; 2374-3468 ; 2159-5399 |
| Publisher Information: |
Association for the Advancement of Artificial Intelligence |
| Publication Year: |
2024 |
| Collection: |
Association for the Advancement of Artificial Intelligence: AAAI Publications |
| Subject Terms: |
NLP: Ethics -- Bias; Fairness; Transparency & Privacy; ML: Ethics; Bias; and Fairness; NLP: (Large) Language Models; NLP: Safety and Robustness |
| Description: |
Fairness in Language Models (LMs) remains a long-standing challenge, given the inherent biases in training data that can be perpetuated by models and affect the downstream tasks. Recent methods employ expensive retraining or attempt debiasing during inference by constraining model outputs to contrast from a reference set of biased templates/exemplars. Regardless, they don’t address the primary goal of fairness to maintain equitability across different demographic groups. In this work, we posit that inferencing LMs to generate unbiased output for one demographic under a context ensues from being aware of outputs for other demographics under the same context. To this end, we propose Counterfactually Aware Fair InferencE (CAFIE), a framework that dynamically compares the model’s understanding of diverse demographics to generate more equitable sentences. We conduct an extensive empirical evaluation using base LMs of varying sizes and across three diverse datasets and found that CAFIE outperforms strong baselines. CAFIE produces fairer text and strikes the best balance between fairness and language modeling capability. |
| Document Type: |
article in journal/newspaper |
| File Description: |
application/pdf |
| Language: |
English |
| Relation: |
https://ojs.aaai.org/index.php/AAAI/article/view/29719/31235; https://ojs.aaai.org/index.php/AAAI/article/view/29719 |
| DOI: |
10.1609/aaai.v38i16.29719 |
| Availability: |
https://ojs.aaai.org/index.php/AAAI/article/view/29719; https://doi.org/10.1609/aaai.v38i16.29719 |
| Rights: |
Copyright (c) 2024 Association for the Advancement of Artificial Intelligence |
| Accession Number: |
edsbas.5CB9DC4F |
| Database: |
BASE |