| Title: |
ChatGPT‐4 in Clinical Neurology: An Alzheimer’s Disease Information Quality Analysis |
| Authors: |
Yasin, Ihsaan; Irfan, Bilal; Reader, Jonathan M; Fox‐Fuller, Joshua T |
| Source: |
Alzheimer's & Dementia ; volume 20, issue S4 ; ISSN 1552-5260 1552-5279 |
| Publisher Information: |
Wiley |
| Publication Year: |
2024 |
| Collection: |
Wiley Online Library (Open Access Articles via Crossref) |
| Description: |
Background The integration of Large Language Models (LLMs) like ChatGPT‐4 in clinical settings offers potential enhancements in medical practice, particularly in neurology and dementia care. There is rising public usage of ChatGPT‐4 for preliminary information gathering. This study aims to evaluate the effectiveness of ChatGPT‐4 in responding to neurology‐focused queries, with an emphasis on Alzheimer’s Disease (AD). It addresses the challenges of accuracy and reliability in artificial intelligence (AI)‐generated medical information, which are crucial for practical clinical applications. Method This investigation utilized ChatGPT‐4 to respond to six diverse neurology‐related questions covering symptomatology and caregiver guidance for AD. The responses were assessed using a context‐adapted DISCERN and AGREE II scoring framework, which are rating systems for evaluating the clarity and appropriateness of healthcare information and advice. Two blinded neurologists independently reviewed and scored the AI’s responses. Statistical analyses, including correlation, variance, and linear regression, were conducted to quantify the relationship between the AI’s adherence to clinical guidelines (AGREE II scores) and the quality of information provided (DISCERN scores). Result ChatGPT‐4’s responses achieved a moderate level of alignment with clinical guidelines, indicated by a total AGREE average score of 2.27/7. The general quality rating average was 5.25/7, reflecting moderate accuracy and relevance. The combined AGREE and rating average score was 2.51/7, with a total DISCERN average of 2.14/5. Statistical analysis revealed a moderate positive correlation (Pearson coefficient: 0.58) between AGREE and DISCERN scores. Variance analysis showed low variability in AGREE scores (0.0499) and higher variability in DISCERN scores (0.2200). Regression analysis indicated that AGREE scores moderately predicted DISCERN scores (R² = 0.334), but the relationship was not statistically significant (p > 0.05). Conclusion ... |
| Document Type: |
article in journal/newspaper |
| Language: |
English |
| DOI: |
10.1002/alz.093401 |
| Availability: |
https://doi.org/10.1002/alz.093401; https://alz-journals.onlinelibrary.wiley.com/doi/pdf/10.1002/alz.093401 |
| Rights: |
http://creativecommons.org/licenses/by/4.0/ |
| Accession Number: |
edsbas.8EFCFC78 |
| Database: |
BASE |