| Title: |
Applying Large Language Models to Enhance Dialogue and Communication Analysis for Adaptive Team Training |
| Language: |
English |
| Authors: |
Randall Spain; Wookhee Min; Vikram Kumaran; Jay Pande; Jason Saville; James Lester |
| Source: |
International Journal of Artificial Intelligence in Education. 2025 35(4):2534-2568. |
| Availability: |
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: |
Y |
| Page Count: |
35 |
| Publication Date: |
2025 |
| Sponsoring Agency: |
US Army Futures Command, Combat Capabilities Development Command Soldier Center (DEVCOM) |
| Contract Number: |
W912CG1920001 |
| Document Type: |
Journal Articles; Reports - Research |
| Descriptors: |
Adjustment (to Environment); Artificial Intelligence; Natural Language Processing; Identification; Dialogs (Language); Communication (Thought Transfer); Speech Acts; Teamwork; Data Analysis |
| DOI: |
10.1007/s40593-025-00479-5 |
| ISSN: |
1560-4292; 1560-4306 |
| Abstract: |
Adaptive training environments that can analyze team communication content and provide remediation to facilitate team coordination offer great potential for enhancing adaptive training systems for teams. However, developing computational models that can perform robust team communication analytics based on small datasets is challenging. Large language models (LLMs) hold significant promise for addressing these challenges and improving dialogue act classification performance using zero-shot and few-shot learning. This article evaluates LLMs against previous state-of-the-art methods, with an emphasis on dialogue act recognition performance for team coordination and information sharing tasks and error analysis for identifying frequently misclassified instances. Results from a small team communication dataset indicate that zero-shot LLMs, particularly GPT-4 models that have been refined through robust prompt engineering, achieve significant classification performance improvements in dialogue act recognition compared to previous state-of-the-art transformer-based models fine-tuned with team communication data. Results also showed dialogue act models based on GPT-4 outperformed the prior state-of-the-art result on a domain-transfer task. Error analysis shows that the prompt refinements, especially those aimed at clarifying confusion between dialogue acts, achieve superior recall rates for challenging dialogue act labels by effectively handling complex dialogue scenarios and ambiguities within communication data. Our transformer-based framework demonstrates its effectiveness in achieving high accuracy rates in dialogue act recognition with minimal training data, underscoring its potential to enhance team training programs by providing adaptive feedback. This approach paves the way for developing AI-enabled training systems that can adapt to the dynamic communication styles of different teams. |
| Abstractor: |
As Provided |
| Entry Date: |
2026 |
| Accession Number: |
EJ1499172 |
| Database: |
ERIC |