Developing and validating a scale for engagement in writing with machine translation

Abstract

Machine translation (MT) is increasingly used by second language writers worldwide, but its use by low-proficiency English as a foreign language (EFL) students has been criticised for causing cognitive disengagement in writing. Moreover, research on MT’s impact in this context is limited. As students’ engagement in the writing process is essential for using MT for language learning, this study developed and validated a scale with multidimensional engagement constructs to understand how low-proficiency EFL students engage in writing using MT. A 24-item instrument covering behavioural, cognitive, affective, social, and agentic subscales was administered to 773 Japanese university students, mostly at the Common European Framework of Reference A2 level. Exploratory factor analysis with half the participants identified five engagement constructs, with cognitive engagement subdivided into pre-editing and post-editing, with agentic engagement being excluded because of low factor loadings. This hypothesised five-factor model was tested with the remaining participants using confirmatory factor analysis, which yielded satisfactory reliability coefficients, as well as construct, convergent, and discriminant validity. Concurrent validity was weakly supported, with a near moderate correlation between engagement and self-efficacy. The scale offers insight into Japanese students’ engagement with MT and provides educators with fresh insights into their use of MT.