| Title: |
VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities |
| Authors: |
Mathur, Shray; der Vleuten, Noah van; Yager, Kevin G; Tsai, Esther H R |
| Contributors: |
Basic Energy Sciences |
| Source: |
Machine Learning: Science and Technology ; volume 6, issue 2, page 025051 ; ISSN 2632-2153 |
| Publisher Information: |
IOP Publishing |
| Publication Year: |
2025 |
| Description: |
Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Vi rtual S cientific Compan ion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery. |
| Document Type: |
article in journal/newspaper |
| Language: |
unknown |
| DOI: |
10.1088/2632-2153/add9e4 |
| DOI: |
10.1088/2632-2153/add9e4/pdf |
| Availability: |
https://doi.org/10.1088/2632-2153/add9e4; https://iopscience.iop.org/article/10.1088/2632-2153/add9e4; https://iopscience.iop.org/article/10.1088/2632-2153/add9e4/pdf |
| Rights: |
https://creativecommons.org/licenses/by/4.0/ ; https://iopscience.iop.org/info/page/text-and-data-mining |
| Accession Number: |
edsbas.F26D6791 |
| Database: |
BASE |