| Title: |
P-1967. Using Secure Artificial Intelligence Agents Integrated within the Electronic Medical Record for the Evaluation of Blood Culture Appropriateness — Northern California, 2025 |
| Authors: |
Rodriguez-Nava, Guillermo; Keyes, Timothy; Ambers, Nerissa; Miranti, Eugenia; Tariq, Wajeeha; Viana-Cardenas, Erika P; Sampson, Mindy M; Salinas, Jorge |
| Source: |
Open Forum Infectious Diseases ; volume 13, issue Supplement_1 ; ISSN 2328-8957 |
| Publisher Information: |
Oxford University Press (OUP) |
| Publication Year: |
2026 |
| Description: |
Background Large language models (LLMs) have gained attention for their ability to exhibit human-like clinical reasoning with mock clinical cases. However, because of privacy concerns, few studies have evaluated their use in real-world healthcare settings. We aimed to assess the accuracy of LLMs in auditing blood culture appropriateness using real charts.Prompt Provided to Initial Reviewer AI Agent for Blood Culture Appropriateness ClassificationAI agents were guided by structured inclusion and exclusion criteria to assess blood culture appropriateness. Prompts included clinical definitions, required supporting evidence, and explicit instructions to avoid assumptions or external reasoning beyond the documentation in the clinical note. Agents were also asked to provide quoted justification for their classifications. The criteria were adapted from the Johns Hopkins Prevention Epicenter Blood Culture Stewardship Collaborative algorithm and based on: Fabre V, Sharara SL, Salinas AB, Carroll KC, Desai S, Cosgrove SE. Does This Patient Need Blood Cultures? A Scoping Review of Indications for Blood Cultures in Adult Nonneutropenic Inpatients. Clin Infect Dis. 2020; PMID: 31942949.Prompt Provided to Double Checker AI Agent for Verification of Blood Culture Appropriateness AssessmentThe second AI agent was tasked with verifying the initial reviewer’s justification against explicit inclusion criteria for blood culture ordering. The prompt instructed the agent to identify whether the justification explicitly met at least one inclusion criterion and to flag assumptions or unsupported reasoning. Final classifications required quoting or rejecting specific evidence from the patient’s chart. Methods Stanford University deployed secure LLMs with direct access to electronic medical records. Using these, we developed two artificial intelligence (AI) agents—task-specific models designed to audit blood culture order appropriateness based on previously published criteria. We applied the agents to a random sample of 105 ... |
| Document Type: |
article in journal/newspaper |
| Language: |
English |
| DOI: |
10.1093/ofid/ofaf695.2134 |
| Availability: |
https://doi.org/10.1093/ofid/ofaf695.2134; https://academic.oup.com/ofid/article-pdf/13/Supplement_1/ofaf695.2134/66356412/ofaf695.2134.pdf |
| Rights: |
https://creativecommons.org/licenses/by/4.0/ |
| Accession Number: |
edsbas.FAA5BF84 |
| Database: |
BASE |