Abstract
This study utilizes ChatGPT to evaluate the probability and specific topics of artificial intelligence hallucinations (AI hallucinations) in junior high school history subject using contemporary large language model technology. Furthermore, it explores whether large language models can become a reliable learning tool that students can trust at this stage. Because large language models possess multiple abilities that learners must achieve in the cognitive field, they have been widely used in teaching to improve students’ learning efficiency. However, when large language models generate text, they may produce answers that are inconsistent with the facts due to variations in training datasets across languages. This situation is called “AI hallucination”. History is a subject that emphasizes local knowledge and culture. The traditional culture or proper nouns involved may never have appeared in the language model’s training datasets, thereby increasing the probability of AI hallucinations and potentially affecting students’ learning. Therefore, this study uses the item bank of the National Academy for Educational Research to test and quantify the probability of AI hallucinations occurring when answering questions in ChatGPT. The results show that the probability of AI hallucinations in each unit of Taiwanese history ranges from 1.54% to 10.77%.Therefore, in the humanities and social science subjects, as the amount of digital text in artificial intelligence training increases, the probability of hallucinations can be significantly reduced, thereby improving the quality of generated content for educational use.
Presenters
Yun Jung HsuStudent, Digtal Content and Technology, National Taichung University of Education, Taiwan
Details
Presentation Type
Paper Presentation in a Themed Session
Theme
KEYWORDS
History Subject, AI Hallucinations, Large Language Model, Junior High School
