Development and Temporal Validation of a Machine Learning Model to Predict Clinical Deterioration.
| Title: | Development and Temporal Validation of a Machine Learning Model to Predict Clinical Deterioration. |
|---|---|
| Authors: | Foote HP; Divisions of Pediatric Cardiology.; Shaikh Z; Duke Institute for Health Innovation.; Department of Medicine, Weill Cornell Medical Center, New York, New York.; Witt D; Duke Institute for Health Innovation.; Mayo Clinic Alix School of Medicine, Rochester, Minnesota.; Shen T; Department of Biomedical Engineering.; Ratliff W; Duke Institute for Health Innovation.; Shi H; Duke Institute for Health Innovation.; Gao M; Duke Institute for Health Innovation.; Nichols M; Duke Institute for Health Innovation.; Sendak M; Duke Institute for Health Innovation.; Balu S; Duke Institute for Health Innovation.; Osborne K; Duke University Health System, Duke University, Durham, North Carolina.; Kumar KR; Pediatric Critical Care Medicine.; Jackson K; Pediatric Critical Care Medicine.; McCrary AW; Divisions of Pediatric Cardiology.; Li JS; Divisions of Pediatric Cardiology. |
| Source: | Hospital pediatrics [Hosp Pediatr] 2024 Jan 01; Vol. 14 (1), pp. 11-20. |
| Publication Type: | Journal Article; Research Support, Non-U.S. Gov't |
| Language: | English |
| Journal Info: | Publisher: American Academy of Pediatrics Country of Publication: United States NLM ID: 101585349 Publication Model: Print Cited Medium: Internet ISSN: 2154-1671 (Electronic) Linking ISSN: 21541671 NLM ISO Abbreviation: Hosp Pediatr Subsets: MEDLINE |
| Imprint Name(s): | Publication: July 2011- : Elk Grove Village : American Academy of Pediatrics; Original Publication: Elk Grove Village, IL : American Academy of Pediatrics/Section on Hospital Medicine |
| MeSH Terms: | Clinical Deterioration* ; Early Warning Score*; Child ; Humans ; Retrospective Studies ; Machine Learning ; Child, Hospitalized ; ROC Curve |
| Abstract: | Objectives: Early warning scores detecting clinical deterioration in pediatric inpatients have wide-ranging performance and use a limited number of clinical features. This study developed a machine learning model leveraging multiple static and dynamic clinical features from the electronic health record to predict the composite outcome of unplanned transfer to the ICU within 24 hours and inpatient mortality within 48 hours in hospitalized children.; Methods: Using a retrospective development cohort of 17 630 encounters across 10 388 patients, 2 machine learning models (light gradient boosting machine [LGBM] and random forest) were trained on 542 features and compared with our institutional Pediatric Early Warning Score (I-PEWS).; Results: The LGBM model significantly outperformed I-PEWS based on receiver operating characteristic curve (AUROC) for the composite outcome of ICU transfer or mortality for both internal validation and temporal validation cohorts (AUROC 0.785 95% confidence interval [0.780-0.791] vs 0.708 [0.701-0.715] for temporal validation) as well as lead-time before deterioration events (median 11 hours vs 3 hours; P = .004). However, LGBM performance as evaluated by precision recall curve was lesser in the temporal validation cohort with associated decreased positive predictive value (6% vs 29%) and increased number needed to evaluate (17 vs 3) compared with I-PEWS.; Conclusions: Our electronic health record based machine learning model demonstrated improved AUROC and lead-time in predicting clinical deterioration in pediatric inpatients 24 to 48 hours in advance compared with I-PEWS. Further work is needed to optimize model positive predictive value to allow for integration into clinical practice.; (Copyright © 2024 by the American Academy of Pediatrics.) |
| References: | Crit Care. 2011 Aug 03;15(4):R184. (PMID: 21812993); Paediatr Nurs. 2005 Feb;17(1):32-5. (PMID: 15751446); JMIR Med Inform. 2017 Nov 22;5(4):e45. (PMID: 29167089); JAMA. 2018 Mar 13;319(10):1002-1012. (PMID: 29486493); Resuscitation. 2016 May;102:1-5. (PMID: 26898412); J Thorac Cardiovasc Surg. 2022 Jul;164(1):211-222.e3. (PMID: 34949457); J Gen Intern Med. 2003 Feb;18(2):77-83. (PMID: 12542581); JAMIA Open. 2018 May 23;1(1):26-31. (PMID: 31984317); Pediatr Crit Care Med. 2020 Sep;21(9):820-826. (PMID: 32511200); Hosp Pediatr. 2022 May 1;12(5):447-460. (PMID: 35470399); Crit Care. 2009;13(4):R135. (PMID: 19678924); J Spec Pediatr Nurs. 2009 Apr;14(2):79-85. (PMID: 19356201); Hosp Pediatr. 2023 Apr 1;13(4):275-282. (PMID: 36911912); Congenit Heart Dis. 2014 May-Jun;9(3):194-202. (PMID: 23957443); J Am Heart Assoc. 2019 Jul 16;8(14):e012637. (PMID: 31288613); Int J Med Inform. 2018 Apr;112:15-20. (PMID: 29500014); J Thorac Cardiovasc Surg. 2016 Jul;152(1):171-7. (PMID: 27174513); Resuscitation. 2014 Aug;85(8):1065-71. (PMID: 24813568); Pediatrics. 2010 Apr;125(4):e763-9. (PMID: 20308222); Pediatr Crit Care Med. 2022 May 1;23(5):405-408. (PMID: 35583620); Pediatr Crit Care Med. 2009 May;10(3):306-12. (PMID: 19307806); Transl Pediatr. 2018 Apr;7(2):120-128. (PMID: 29770293); Chest. 2022 Jun;161(6):1621-1627. (PMID: 35143823); JAMA Pediatr. 2014 Jan;168(1):25-33. (PMID: 24217295); Ann Intern Med. 2015 Jan 6;162(1):W1-73. (PMID: 25560730); Crit Care Med. 2019 Jan;47(1):49-55. (PMID: 30247239); JAMIA Open. 2020 Apr 11;3(2):252-260. (PMID: 32734166); Stat Med. 2000 May 15;19(9):1141-64. (PMID: 10797513); J Hosp Med. 2012 Apr;7(4):345-9. (PMID: 22489072); J Pediatr Nurs. 2013 Nov-Dec;28(6):e33-41. (PMID: 23276507); J Pediatr Nurs. 2017 Jan - Feb;32:52-58. (PMID: 27823915); Pediatr Crit Care Med. 2022 Jul 1;23(7):514-523. (PMID: 35446816); J Crit Care. 2006 Sep;21(3):271-8. (PMID: 16990097); J Hosp Med. 2013 May;8(5):254-60. (PMID: 23589468) |
| Grant Information: | T32 HD094671 United States HD NICHD NIH HHS |
| Entry Date(s): | Date Created: 20231206 Date Completed: 20240102 Latest Revision: 20250104 |
| Update Code: | 20260130 |
| PubMed Central ID: | PMC11293885 |
| DOI: | 10.1542/hpeds.2023-007308 |
| PMID: | 38053467 |
| Database: | MEDLINE |
Journal Article; Research Support, Non-U.S. Gov't