| Title: |
AutoNeRF: Training Implicit Scene Representations with Autonomous Agents |
| Authors: |
Marza, Pierre; Matignon, Laetitia; Simonin, Olivier; Batra, Dhruv; Wolf, Christian; Chaplot, Devendra, Singh |
| Contributors: |
Robots coopératifs et adaptés à la présence humaine en environnements (CHROMA); Centre Inria de l'Université Grenoble Alpes; Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-CITI Centre of Innovation in Telecommunications and Integration of services (CITI); Institut National des Sciences Appliquées de Lyon (INSA Lyon); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National des Sciences Appliquées de Lyon (INSA Lyon); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université de Lyon-Institut National des Sciences Appliquées (INSA)-Centre Inria de Lyon; Institut National de Recherche en Informatique et en Automatique (Inria); Extraction de Caractéristiques et Identification (imagine); Pôle informatique graphique et géométrie (IGG); Laboratoire d'InfoRmatique en Image et Systèmes d'information (LIRIS); Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL); Université de Lyon-Université de Lyon-Université Claude Bernard Lyon 1 (UCBL); Université de Lyon-Institut National des Sciences Appliquées de Lyon (INSA Lyon); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS)-Université Lumière - Lyon 2 (UL2)-École Centrale de Lyon (ECL); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire d'InfoRmatique en Image et Systèmes d'information (LIRIS); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Centre National de la Recherche Scientifique (CNRS); CITI Centre of Innovation in Telecommunications and Integration of services (CITI); Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National de Recherche en Informatique et en Automatique (Inria); Université de Lyon-Institut National des Sciences Appliquées (INSA); Université Claude Bernard Lyon 1 (UCBL); Université de Lyon; Systèmes Cognitifs et Systèmes Multi-Agents (SyCoSMA); Georgia Institute of Technology Atlanta; Meta AI; Naver Labs Europe Meylan; Mistral AI |
| Source: |
IROS 2024 - International Conference on Intelligent Robots and Systems ; https://inria.hal.science/hal-04688608 ; IROS 2024 - International Conference on Intelligent Robots and Systems, Oct 2024, Abu Dhabi, United Arab Emirates. pp.1-8 |
| Publisher Information: |
CCSD |
| Publication Year: |
2024 |
| Collection: |
HAL Lyon 1 (University Claude Bernard Lyon 1) |
| Subject Terms: |
Computer Vision and Pattern Recognition csCV; Machine Learning csLG; Robotics csRO; [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-RB]Computer Science [cs]/Robotics [cs.RO] |
| Subject Geographic: |
Abu Dhabi; United Arab Emirates |
| Description: |
International audience ; Implicit representations such as Neural Radiance Fields (NeRF) allow to map color, density and semantics in a 3D scene through a continuous neural function. However, these models typically require manual and careful human data collection for training. This paper addresses the problem of active exploration for autonomous NeRF construction. We study how an agent can learn to efficiently explore an unknown 3D environment so that the data collected during autonomous exploration enables the learning of a high-quality neural implicit map representation. The quality of the learned representation is evaluated on four robotics-related downstream tasks: classical viewpoint rendering, map reconstruction, planning, and pose refinement. We compare the impact of different exploration strategies including frontier-based and learning-based approaches (end-to-end and modular) with different reward functions tailored to this problem. Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment and that AutoNeRF, a modular exploration policy trained with reinforcement learning, enables obtaining a higher-quality NeRF for the considered downstream robotic tasks. Finally, we show that with AutoNeRF an agent can be deployed to a previously unknown scene and then automatically improve its navigation performance by adapting to the scene through a cycle of exploration, reconstruction, and policy finetuning. |
| Document Type: |
conference object |
| Language: |
English |
| Availability: |
https://inria.hal.science/hal-04688608; https://inria.hal.science/hal-04688608v1/document; https://inria.hal.science/hal-04688608v1/file/IROS2024_AutoNeRF_Camera_Ready.pdf |
| Rights: |
https://creativecommons.org/licenses/by/4.0/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.B6A6A806 |
| Database: |
BASE |