| Title: |
AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices |
| Authors: |
Liu, Ji; Che, Tianshi; Zhou, Yang; Jin, Ruoming; Dai, Huaiyu; Valduriez, Patrick |
| Contributors: |
Hithink RoyalFlush Information Network Co; Auburn University (AU); Kent State University; North Carolina State University Raleigh (NC State); University of North Carolina System (UNC); Laboratorio Nacional de Computação Cientifica Rio de Janeiro (LNCC / MCT); Scientific Data Management (ZENITH); Centre Inria d'Université Côte d'Azur; Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier (LIRMM); Université de Perpignan Via Domitia (UPVD)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Université de Montpellier (UM)-Université de Montpellier Paul-Valéry (UMPV)-Université de Perpignan Via Domitia (UPVD)-Centre National de la Recherche Scientifique (CNRS)-Université de Montpellier (UM)-Université de Montpellier Paul-Valéry (UMPV); Society for Industrial and Applied Mathematics |
| Source: |
SDM 2024 - SIAM International Conference on Data Mining ; https://hal-lirmm.ccsd.cnrs.fr/lirmm-04597263 ; SDM 2024 - SIAM International Conference on Data Mining, Society for Industrial and Applied Mathematics, Apr 2024, Houston, TX, United States. pp.833-841, ⟨10.1137/1.9781611978032.95⟩ ; https://www.siam.org/conferences/cm/conference/sdm24 |
| Publisher Information: |
CCSD; Society for Industrial and Applied Mathematics |
| Publication Year: |
2024 |
| Collection: |
Université de Perpignan: HAL |
| Subject Terms: |
Federated Learning; Decentralized Machine Learning; Asynchronous Learning; Staleness-Aware Model Update; Sparse Training; [INFO]Computer Science [cs] |
| Subject Geographic: |
Houston; TX; United States |
| Description: |
International audience ; Federated Learning (FL) has achieved significant achievements re- cently, enabling collaborative model training on distributed data over edge devices. Iterative gradient or model exchanges between devices and the centralized server in the standard FL paradigm suf- fer from severe efficiency bottlenecks on the server. While enabling collaborative training without a central server, existing decentral- ized FL approaches either focus on the synchronous mechanism that deteriorates FL convergence or ignore device staleness with an asynchronous mechanism, resulting in inferior FL accuracy. In this paper, we propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in heterogeneous environments with three unique contributions. First, we propose an asynchronous FL sys- tem model with an efficient model aggregation method for improv- ing the FL convergence. Second, we propose a dynamic staleness- aware model update approach to achieve superior accuracy. Third, we propose an adaptive sparse training method to reduce commu- nication and computation costs without significant accuracy degra- dation. Extensive experimentation on four public datasets and four models demonstrates the strength of AEDFL in terms of accuracy (up to 16.3% higher), efficiency (up to 92.9% faster), and computa- tion costs (up to 42.3% lower). |
| Document Type: |
conference object |
| Language: |
English |
| DOI: |
10.1137/1.9781611978032.95 |
| Availability: |
https://hal-lirmm.ccsd.cnrs.fr/lirmm-04597263; https://hal-lirmm.ccsd.cnrs.fr/lirmm-04597263v1/document; https://hal-lirmm.ccsd.cnrs.fr/lirmm-04597263v1/file/SDM%202024.pdf; https://doi.org/10.1137/1.9781611978032.95 |
| Rights: |
https://about.hal.science/hal-authorisation-v1/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.C1D8607E |
| Database: |
BASE |