The Matthew effect in CALL: Examining the equity of a novel intelligent writing assistant as English language support

Abstract

As practitioners introduce new educational technologies into their classrooms, the potential for unintended outcomes from their use might arise. One such potential negative artifact is an increase in the achievement gaps between learners, where high performers tend to benefit more from newly introduced educational technologies than their peers. This phenomenon is commonly referred to as the Matthew effect. In this study, we leverage natural language processing (NLP) based transformers to introduce English language support to English as a Foreign Language (EFL) learners while they are in the writing process. A web-based application was created that uses next-word prediction and automatic reverse translation to help EFL participants in their writing. Adult English language learners from professional development language schools participated in a counterbalanced repeated measures study. To understand the presence of the Matthew effect, learners were grouped based on their self-reported EIKEN scores. Their performance according to two writing factors as well as their perceived cognitive load while using the tool were measured to establish which groups benefit the most from using the tool. This research sets the stage for understanding how emerging tools can support learning without exacerbating Matthew effects. These effects should be considered in both the development and application of educational technology.