| Title: |
DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents |
| Authors: |
Chuang, Yun-Shiuan; Tu, Ruixuan; Dai, Chengtao; Vasani, Smit; Li, You; Yao, Binwei; Tessler, Michael Henry; Yang, Sijia; Shah, Dhavan; Hawkins, Robert; Hu, Junjie; Rogers, Timothy T. |
| Publication Year: |
2025 |
| Collection: |
ArXiv.org (Cornell University Library) |
| Subject Terms: |
Computation and Language |
| Description: |
Accurately modeling opinion change through social interactions is crucial for understanding and mitigating polarization, misinformation, and societal conflict. Recent work simulates opinion dynamics with role-playing LPL agents (RPLAs), but multi-agent simulations often display unnatural group behavior, such as premature convergence, and lack empirical benchmarks for assessing alignment with real human group interactions. We introduce DEBATE, a large-scale benchmark for evaluating the authenticity of opinion dynamics in multi-agent RPLA simulations. DEBATE contains 30,707 messages from 2,832 U.S.-based participants across 708 groups and 107 topics, with both public messages and private Likert-scale beliefs, enabling evaluation at the utterance and group levels while also supporting future individual-level analyses. We instantiate "digital twin" RPLAs with seven LLMs and evaluate them in two settings: next-message prediction and full conversation rollout, using stance-alignment and opinion-convergence metrics. In zero-shot settings, RPLA groups exhibit strong opinion convergence relative to human groups. Post-training via supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) improves stance alignment and brings group-level convergence closer to human behavior, though discrepancies in opinion change and belief updating remain. DEBATE enables rigorous benchmarking of simulated opinion dynamics and supports future research on aligning multi-agent RPLAs with realistic human interactions. The benchmark is publicly available at. |
| Document Type: |
text |
| Language: |
unknown |
| Relation: |
http://arxiv.org/abs/2510.25110 |
| Availability: |
http://arxiv.org/abs/2510.25110 |
| Accession Number: |
edsbas.4BEB00F |
| Database: |
BASE |