Assessing L2 Japanese speaking in role-play tasks on an online meeting platform

Abstract

This study examines the assessment of L2 Japanese speaking in online role-play tasks from a sociolinguistic-interactional perspective. Twenty L2 learners from China engaged in online role-plays with a professor while performing various speech acts. Four raters evaluated the participants’ video-recorded performances using five criteria adapted from Youn (2015; 2020). Multifaceted Rasch Measurement (MFRM) confirms the raters were able to assess the participants’ oral performances effectively using the rubric. In addition to the quantitative analysis, qualitative transcript analysis highlights differences between higher- and lower-rated learners’ oral performances. Compared with the high-scoring performances, the low-scoring performances showed difficulties such as inaccurate language use and long pauses both within and between turns. When communication breaks down, managing intersubjectivity becomes essential. Speakers with high interactional proficiency handle it subtly, while for low-proficiency learners, the negotiation of meaning, use of repairs, and visible struggles to maintain intersubjectivity are much more apparent. Findings provide insights into L2 learners’ interactive performance in online settings compared to face-to-face contexts, contributing to a deeper understanding of assessing L2 speakers’ sociolinguistic-interactional skills in online role-play.