Evaluating advanced EFL learners’ writing performance and perceptions of large language model assistance

Abstract

Since the release of Large Language Models (LLMs), they have been widely used in language teaching and acquisition. Previous studies have explored LLMs’ assistance in improving language performance, but their impact on advanced EFL learners’ writing remains unclear (Barrot, 2023; Liu, et al., 2023; Song & Song, 2023). This study conducts an experiment based on their IELTS or TOEFL scores. After a pre-test to exclude participants who could not follow instructions, they were randomly divided into two groups. Participants in the experiment group were instructed to complete the essay with guidance but not to directly generate it. Following the task, participants completed a motivation and perception questionnaire. Surprisingly, results show that LLMs did not significantly improve writing performance in advanced EFL learners. Additionally, it appears that the current level of LLMs matches the writing capabilities of advanced learners. This study explores how LLMs affect EFL performance and offers insights into their potential assistance.