| Title: |
COGNAC: Cooperative Graph-based Networked Agent Challenges for Multi-Agent Reinforcement Learning |
| Authors: |
Sintes, Jules; Bušić, Ana |
| Contributors: |
Département d'informatique - ENS-PSL (DI-ENS); École normale supérieure - Paris (ENS-PSL); Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS); Apprentissage, graphes et optimisation distribuée (ARGO); Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL); Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre Inria de Paris; Institut National de Recherche en Informatique et en Automatique (Inria); Laboratory of Information, Network and Communication Sciences (LINCS); Institut National de Recherche en Informatique et en Automatique (Inria)-Institut Mines-Télécom Paris (IMT)-Sorbonne Université (SU); ANR-22-PETA-0004,AI-NRGY,Distributed AI-based architecture of future energy systems integrating very large amounts of distributed sources(2022); ANR-23-PEIA-0005,REDEEM,Resilient, Decentralized and Privacy-Preserving Machine Learning(2023) |
| Source: |
NeurIPS 2025 - 39th Annual Conference on Neural Information Processing Systems ; https://hal.science/hal-05488783 ; NeurIPS 2025 - 39th Annual Conference on Neural Information Processing Systems, Dec 2025, San Diego, United States |
| Publisher Information: |
CCSD |
| Publication Year: |
2025 |
| Subject Terms: |
Multi-agent; Reinforcement learning; Benchmark; Environment; Open-source; [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] |
| Subject Geographic: |
San Diego; United States |
| Description: |
International audience ; Many controlled complex systems have an inherent network structure, such as power grids, traffic light systems, or computer networks. Automatically controlling these systems is highly challenging due to their combinatorial complexity. Standard single-agent reinforcement learning (RL) approaches often struggle with the curse of dimensionality in such settings. In contrast, the multi-agent paradigm offers a promising solution by distributing decision-making, thereby addressing both algorithmic and combinatorial challenges. In this paper, we introduce COGNAC (COoperative Graph-based Networked Agent Challenges), a collection of cooperative graph-structured environments designed to facilitate experiments across different graph sizes and topologies. COGNAC bridges the gap between theoretical research in network control and practical multi-agent RL (MARL) applications by offering a flexible, scalable platform with a suite of simple yet highly challenging problems rooted in networked environments. Our benchmarks also support the development and evaluation of decentralized and distributed learning algorithms, motivated by the growing interest in more sustainable and frugal AI systems. Experiments on COGNAC show that independent actor–critic learning (IPPO) yields the highest-quality joint policies while scaling robustly to large network sizes with minimal hyperparameter tuning. Value-based independent learning (IDQL) typically needs substantially more training and is less reliable on combinatorial tasks. In contrast, standard Centralized-Training Decentralized-Execution (CTDE) methods and fully centralized training are slower to converge, less stable, and struggle to generalize to larger, more interdependent networks. These results suggest that CTDE approaches likely need extra information or inter-agent communication to fully capture the underlying network structure of each problem. |
| Document Type: |
conference object |
| Language: |
English |
| Availability: |
https://hal.science/hal-05488783; https://hal.science/hal-05488783v1/document; https://hal.science/hal-05488783v1/file/1229_COGNAC_Cooperative_Graph_%20%281%29.pdf |
| Rights: |
https://creativecommons.org/licenses/by/4.0/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.5D69BCFB |
| Database: |
BASE |