| Title: |
From unimodal to multimodal dynamics of verbal and nonverbal cues during unstructured conversation |
| Authors: |
Fauviaux, Tifenn; Marin, Ludovic; Parisi, Mathilde; Schmidt, Richard; Mostafaoui, Ghilès |
| Contributors: |
EuroMov - Digital Health in Motion (Euromov DHM); IMT MINES ALÈS; Institut Mines-Télécom Paris (IMT)-Institut Mines-Télécom Paris (IMT)-Université de Montpellier (UM); Equipes Traitement de l'Information et Systèmes (ETIS - UMR 8051); Ecole Nationale Supérieure de l'Electronique et de ses Applications (ENSEA)-Centre National de la Recherche Scientifique (CNRS)-CY Cergy Paris Université (CY); CY Cergy Paris Université (CY) |
| Source: |
ISSN: 1932-6203. |
| Publisher Information: |
CCSD; Public Library of Science |
| Publication Year: |
2024 |
| Collection: |
Université Paris Seine: ComUE (HAL) |
| Subject Terms: |
[SDV.MHEP]Life Sciences [q-bio]/Human health and pathology |
| Description: |
International audience ; Conversations encompass continuous exchanges of verbal and nonverbal information. Previous research has demonstrated that gestures dynamically entrain each other and that speakers tend to align their vocal properties. While gesture and speech are known to synchronize at the intrapersonal level, few studies have investigated the multimodal dynamics of gesture/speech between individuals. The present study aims to extend our comprehension of unimodal dynamics of speech and gesture to multimodal speech/gesture dynamics. We used an online dataset of 14 dyads engaged in unstructured conversation. Speech and gesture synchronization was measured with cross-wavelets at different timescales. Results supported previous research on intrapersonal speech/gesture coordination, finding synchronization at all timescales of the conversation. Extending the literature, we also found interpersonal synchronization between speech and gesture. Given that the unimodal and multimodal synchronization occurred at similar timescales, we suggest that synchronization likely depends on the vocal channel, particularly on the turn-taking dynamics of the conversation. |
| Document Type: |
article in journal/newspaper |
| Language: |
English |
| DOI: |
10.1371/journal.pone.0309831 |
| Availability: |
https://imt-mines-ales.hal.science/hal-04930186; https://imt-mines-ales.hal.science/hal-04930186v1/document; https://imt-mines-ales.hal.science/hal-04930186v1/file/journal.pone.0309831.pdf; https://doi.org/10.1371/journal.pone.0309831 |
| Rights: |
https://creativecommons.org/licenses/by/4.0/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.1F78C6E4 |
| Database: |
BASE |