Evaluating ChatGPT's moral competence in health care-related ethical problems.
| Title: | Evaluating ChatGPT's moral competence in health care-related ethical problems. |
|---|---|
| Authors: | Rashid AA; Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL 32608, United States.; Skelly RA; Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL 32608, United States.; Valdes CA; Department of Surgery, University of Florida College of Medicine, Gainesville, FL 32608, United States.; Patel PP; Department of Research, Alabama College of Osteopathic Medicine, Dothan, AL 36303, United States.; Solberg LB; Department of Community Health and Family Medicine, University of Florida College of Medicine, Gainesville, FL 32608, United States.; Giordano CR; Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL 32608, United States.; Modave F; Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL 32608, United States. |
| Source: | JAMIA open [JAMIA Open] 2024 Jul 09; Vol. 7 (3), pp. ooae065. Date of Electronic Publication: 2024 Jul 09 (Print Publication: 2024). |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Oxford University Press on behalf of the American Medical Informatics Association Country of Publication: United States NLM ID: 101730643 Publication Model: eCollection Cited Medium: Internet ISSN: 2574-2531 (Electronic) Linking ISSN: 25742531 NLM ISO Abbreviation: JAMIA Open Subsets: PubMed not MEDLINE |
| Imprint Name(s): | Original Publication: [Cary, NC] : Oxford University Press on behalf of the American Medical Informatics Association, [2018]- |
| Abstract: | Objectives: Artificial intelligence tools such as Chat Generative Pre-trained Transformer (ChatGPT) have been used for many health care-related applications; however, there is a lack of research on their capabilities for evaluating morally and/or ethically complex medical decisions. The objective of this study was to assess the moral competence of ChatGPT.; Materials and Methods: This cross-sectional study was performed between May 2023 and July 2023 using scenarios from the Moral Competence Test (MCT). Numerical responses were collected from ChatGPT 3.5 and 4.0 to assess individual and overall stage scores, including C-index and overall moral stage preference. Descriptive analysis and 2-sided Student's t-test were used for all continuous data.; Results: A total of 100 iterations of the MCT were performed and moral preference was found to be higher in the latter Kohlberg-derived arguments. ChatGPT 4.0 was found to have a higher overall moral stage preference (2.325 versus 1.755) when compared to ChatGPT 3.5. ChatGPT 4.0 was also found to have a statistically higher C-index score in comparison to ChatGPT 3.5 (29.03 ± 11.10 versus 19.32 ± 10.95, P =.0000275).; Discussion: ChatGPT 3.5 and 4.0 trended towards higher moral preference for the latter stages of Kohlberg's theory for both dilemmas with C-indices suggesting medium moral competence. However, both models showed moderate variation in C-index scores indicating inconsistency and further training is recommended.; Conclusion: ChatGPT demonstrates medium moral competence and can evaluate arguments based on Kohlberg's theory of moral development. These findings suggest that future revisions of ChatGPT and other large language models could assist physicians in the decision-making process when encountering complex ethical scenarios.; (© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.) |
| Competing Interests: | The authors declare no conflicts of interest. |
| References: | JMIR Form Res. 2024 May 8;8:e51346. (PMID: 38717811); Psychother Psychosom Med Psychol. 2023 May;73(5):173-179. (PMID: 36195103); Health Serv Res Manag Epidemiol. 2024 Mar 5;11:23333928241234863. (PMID: 38449840); JAMA Netw Open. 2023 Aug 1;6(8):e2330320. (PMID: 37606922); Wien Klin Wochenschr. 2024 Mar;136(5-6):129-136. (PMID: 36547763); Health Care Anal. 2021 Jun;29(2):113-126. (PMID: 32944887); JAMA Intern Med. 2023 Jun 1;183(6):589-596. (PMID: 37115527); R Soc Open Sci. 2024 Feb 7;11(2):231393. (PMID: 38328569); Diagn Pathol. 2021 Mar 17;16(1):24. (PMID: 33731170); Int J Environ Res Public Health. 2021 May 23;18(11):. (PMID: 34070958); JAMA Intern Med. 2023 Sep 1;183(9):1028-1030. (PMID: 37459090); Int J Environ Res Public Health. 2020 Jul 24;17(15):. (PMID: 32722060); J Am Coll Dent. 2011 Fall;78(3):41-6. (PMID: 22263371); Sci Rep. 2023 Apr 6;13(1):4569. (PMID: 37024502); Lancet Digit Health. 2019 Oct;1(6):e271-e297. (PMID: 33323251); J Emerg Med. 2022 Apr;62(4):492-499. (PMID: 35164977); J Gen Intern Med. 2000 Jul;15(7):503-8. (PMID: 10940138); PLoS One. 2023 Mar 2;18(3):e0282123. (PMID: 36862644); Front Psychol. 2021 Dec 13;12:764360. (PMID: 34966326); Nurs Ethics. 2007 May;14(3):320-8. (PMID: 17459816); Gac Med Mex. 2022 Dec 15;158(Suplement 1):17-21. (PMID: 36921221); JMIR Med Educ. 2023 Aug 7;9:e48978. (PMID: 37548997); Adv Health Sci Educ Theory Pract. 2013 Dec;18(5):1067-85. (PMID: 23463178) |
| Contributed Indexing: | Keywords: artificial intelligence; clinical decision-making; ethics; judgement; morals |
| Entry Date(s): | Date Created: 20240710 Latest Revision: 20240711 |
| Update Code: | 20260130 |
| PubMed Central ID: | PMC11233145 |
| DOI: | 10.1093/jamiaopen/ooae065 |
| PMID: | 38983845 |
| Database: | MEDLINE |
Journal Article