| Title: |
Optimal Fair Aggregation of Crowdsourced Noisy Labels using Demographic Parity Constraints ; Agrégation équitable optimale de labels bruités issues du crowdsourcing sous contraintes de parité démographique |
| Authors: |
Singer, Gabriel; Gruffaz, Samuel; Vo Van, Olivier; Vayatis, Nicolas; Kalogeratos, Argyris |
| Contributors: |
CB - Centre Borelli - UMR 9010 (CB); Service de Santé des Armées-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Ecole Normale Supérieure Paris-Saclay (ENS Paris Saclay)-Université Paris Cité (UPCité); SNCF; Société nationale des chemins de fer français Groupe SNCF (Société nationale SNCF); Ecole Normale Supérieure Paris-Saclay (ENS Paris Saclay) |
| Source: |
https://hal.science/hal-05484387 ; 2026. |
| Publisher Information: |
CCSD |
| Publication Year: |
2026 |
| Subject Terms: |
Crowdsourcing; Fairness; Convex optimization; Statistical Learning; [MATH]Mathematics [math]; [INFO]Computer Science [cs] |
| Description: |
As acquiring reliable ground-truth labels is usually costly, or infeasible, crowdsourcing and aggregation of noisy human annotations is the typical resort. Aggregating subjective labels, though, may amplify individual biases, particularly regarding sensitive features, raising fairness concerns. Nonetheless, fairness in crowdsourced aggregation remains largely unexplored, with no existing convergence guarantees and only limited postprocessing approaches for enforcing ε-fairness under demographic parity. We address this gap by analyzing the fairness of crowdsourced aggregation methods within the ε-fairness framework, for Majority Vote and Optimal Bayesian aggregation. In the small-crowd regime, we derive an upper bound on the fairness gap of Majority Vote in terms of the fairness gaps of the individual annotators. We further show that the fairness gap of the aggregated consensus converges exponentially fast to that of the ground-truth under interpretable conditions. Since ground-truth itself may still be unfair, we generalize a state-of-the-art multiclass fairness post-processing algorithm from the continuous to the discrete setting, which enforces strict demographic parity constraints to any aggregation rule. Experiments on synthetic and real datasets demonstrate the effectiveness of our approach and corroborate the theoretical insights. |
| Document Type: |
report |
| Language: |
English |
| Availability: |
https://hal.science/hal-05484387; https://hal.science/hal-05484387v1/document; https://hal.science/hal-05484387v1/file/icml2026-24.pdf |
| Rights: |
https://about.hal.science/hal-authorisation-v1/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.640D1931 |
| Database: |
BASE |