Using Large Language Models (LLMs) to facilitate L2 proficiency development through personalized feedback and scaffolding: An empirical study

Abstract

This present research is part of a longitudinal study that focuses on understanding the capabilities and affordances of generative artificial intelligence (AI), especially Large Language Models (LLMs), and how they can be harnessed to support second language (L2) learning, teaching, and research. Leverating LLMs’ significant new capabilities that resemble those of human languages: zero-shot language generation, in context learning, and chain of thought prompting, a LLM-based application is developed to foster English language learners’ proficiency growth, especially in productive skills. Specifically, utilizing the capabilities of the GPT 4o model, this application guides learners through structured text-based conversations on topics selected based on learners’ current proficiency level (CEFR, 2020). Further, two individualized feedback mechanisms programmed on top of the same GPT model are designed and embedded in the conversation practices to provide two layers of scaffolding: 1) graduated corrective feedback informed by the sociocultural theory of learning to guide learners self-identifying and self-correcting errors, and 2) metacognitive reviews to help learners become aware of areas for improvement based on the detailed proficiency descriptions outlined in CEFR.  The research presented here centers on the current phase in this project that focuses on evaluating the quality and effectiveness of the feedback mechanisms, particularly learners’ responsiveness to AI-generated graduated feedback, and assessing the system’s potentials for supporting independent language learning, classroom integration, and L2 research. The main data sources include the conversation logs between English learners at different proficiency levels and the AI system generated over 4 weeks, and learners’ responses to post-session evaluation surveys. The researchers will explain the design principles and processes underpinning this AI-powered application, provide an overview of the learner experience using the application, discuss the results from data analysis in this phase and their implications for L2 pedagogy and future research.