| Title: |
An LLM-Driven Chatbot in Higher Education for Databases and Information Systems |
| Language: |
English |
| Authors: |
Alexander Tobias Neumann (ORCID 0000-0002-9210-5226); Yue Yin (ORCID 0009-0006-8369-8396); Sulayman Sowe (ORCID 0000-0002-8605-2009); Stefan Decker (ORCID 0000-0001-6324-7164); Matthias Jarke (ORCID 0000-0001-6169-2942) |
| Source: |
IEEE Transactions on Education. 2025 68(1):103-116. |
| Availability: |
Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=13 |
| Peer Reviewed: |
Y |
| Page Count: |
14 |
| Publication Date: |
2025 |
| Document Type: |
Journal Articles; Reports - Research |
| Education Level: |
Higher Education; Postsecondary Education |
| Descriptors: |
Computer Science Education; Databases; Information Systems; Classroom Environment; Learning Management Systems; Self Management; Independent Study; Technology Uses in Education; Artificial Intelligence; Student Attitudes; Usability; User Satisfaction (Information); Influence of Technology; College Students; Higher Education; Natural Language Processing; Computer Software; Intelligent Tutoring Systems |
| DOI: |
10.1109/TE.2024.3467912 |
| ISSN: |
0018-9359; 1557-9638 |
| Abstract: |
Contribution: This research explores the benefits and challenges of developing, deploying, and evaluating a large language model (LLM) chatbot, MoodleBot, in computer science classroom settings. It highlights the potential of integrating LLMs into LMSs like Moodle to support self-regulated learning (SRL) and help-seeking behavior. Background: Computer science educators face immense challenges incorporating novel tools into LMSs to create a supportive and engaging learning environment. MoodleBot addresses this challenge by offering an interactive platform for both students and teachers. Research Questions: Despite issues like bias, hallucinations, and teachers' and educators' resistance to embracing new (AI) technologies, this research investigates two questions: (RQ1) To what extent do students accept MoodleBot as a valuable tool for learning support? (RQ2) How accurately does MoodleBot churn out responses, and how congruent are these with the established course content? Methodology: This study reviews pedagogical literature on AI-driven chatbots and adopts the retrieval-augmented generation (RAG) approach for MoodleBot's design and data processing. The technology acceptance model (TAM) evaluates user acceptance through constructs like perceived usefulness (PU) and Ease of Use. Forty-six students participated, with 30 completing the TAM questionnaire. Findings: LLM-based chatbots like MoodleBot can significantly improve the teaching and learning process. This study revealed a high accuracy rate (88%) in providing course-related assistance. Positive responses from students attest to the efficacy and applicability of AI-driven educational tools. These findings indicate that educational chatbots are suitable for integration into courses to improve personalized learning and reduce teacher administrative burden, although improvements in automated fact-checking are needed. |
| Abstractor: |
As Provided |
| Entry Date: |
2025 |
| Accession Number: |
EJ1460241 |
| Database: |
ERIC |