Fifteenth International Conference on The Constructed Environment

Abstract

This paper examines the effects of incorporating ChatGPT-based English conversations into a Project-Based English Learning course, focusing on TOEIC Listening and Reading scores and Speech Ace assessment metrics. A total of 110 first- and second-year university students participated in the study, divided into two groups: a Homework group, which engaged in weekly ChatGPT conversations, and a No-Homework group, which did not. The findings indicate mixed results. While the Homework group demonstrated significant improvements in pronunciation, vocabulary, and fluency, especially among second-year students, there was no clear correlation between the number of completed assignments and score improvements. Moreover, TOEIC scores showed greater progress in Listening and Reading for the Homework group compared to the No-Homework group. However, the reliance on text-based interaction with ChatGPT seems to have disproportionately benefited reading and vocabulary development, while its impact on spoken communication skills was less consistent. One notable drawback was an increase in unnatural pauses, suggesting that AI-generated conversations may not always foster natural speech patterns. These findings highlight both the potential and the limitations of using AI tools in English education. While ChatGPT can be an effective supplement for developing language skills, task design must be carefully refined to encourage more natural and dynamic conversational engagement. This study suggests that while AI-driven learning has promise, further pedagogical adjustments are necessary to fully integrate it into language education effectively.