Volume 5, Issue 1 December 2022, pp. 34–51 Download PDF

Regular Articles

The processes of rating L2 speaking performance using an analytic rating scale – A qualitative exploration

Thuy Thai1, & Susan Sheehan2

1 University of Huddersfield, UK lamthuy.ulis@gmail.com
2 University of Huddersfield, UK s.sheehan@hud.ac.uk

DOI: https://doi.org/10.29140/lea.v5n1.777


Abstract

In language performance tests, raters are important as their scoring decisions determine which aspects of  performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater variability impacts on rating decisions, much still remains unclear about what raters actually do when they assess speaking performances. This paper aimed to extend our understanding of the rating process by analysing think-aloud protocols provided by 13 VSTEP speaking raters while they used an analytic rating scale to assess 15 recorded performances with varying proficiency levels. The data suggested that although all the raters seemed to experience similar stages in the rating process, differences in rating behaviour between novice raters and experienced raters exist. To finalise the scores, the raters used five different ways of deciding the scores.



Copyright

© Thuy Thai, Susan Sheehan

CC  4.0
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Suggested citation

Thai, T., & Sheehan, S. (2022). The processes of rating L2 speaking performance using an analytic rating scale – A qualitative exploration. Language Education and Assessment, 5(1), 34–51. https://doi.org/10.29140/lea.v5n1.777


Author biodata

Thuy is a PhD holder in language testing and assessment at School of Education and Professional Development, University of Huddersfield, the UK. Her main interests are speaking assessment, rater cognition, and classroom-based assessment.


Related Articles:

Promoting acts of kindness on campus: Views of Chinese international students in the UK
Ming Cheng, Olalekan Adekola
Intercultural Communication Education Published: 30 April, 2022, Volume 5(1), 17–32. Download PDF


Tasks, self-efficacy, and L2 motivational self system in an online emergency EFL speaking class: A mixed-methods study
Nguyễn Nhật Quang, Phạm Nhật Linh, Nguyễn Thị Thu Hiền
The JALT CALL Journal Published: 2 April, 2022, Volume 18(1), 1–33. Download PDF


Implementing backward design to foster intercultural communicative competence in textbook-based curricula: A proposed framework for English language practitioners
Hiba B. Ibrahim
Intercultural Communication Education Published: 30 April, 2022, Volume 5(1), 1–16. Download PDF


Challenges and responses: A Complex Dynamic Systems approach to exploring language teacher agency in a blended classroom
Grace Yue Qi, Yuping Wang
The JALT CALL Journal Published: 9 April, 2022, Volume 18(1), 54–82. Download PDF


The effects of corpus use on learning L2 collocations
Yoshiho Satake
The JALT CALL Journal Published: 9 April, 2022, Volume 18(1), 34–53. Download PDF


Captions and learnability factors in learning grammar from audio-visual input
Anastasia Pattemore, Carmen Muñoz
The JALT CALL Journal Published: 9 April, 2022, Volume 18(1), 83–109. Download PDF


Chinese language learners evaluating machine translation accuracy
Li-Ching Chang
The JALT CALL Journal Published: 9 April, 2022, Volume 18(1), 110–136. Download PDF


Statistical analysis of the impact of the e-learning platform Furago on French learners’ listening skills
Albéric Derible, Éric Wiel
The JALT CALL Journal Published: 9 April, 2022, Volume 18(1), 137–161. Download PDF


Work integrated language learning: Boundary crossing, connectivity, and L2 affordances
Linda Febring, Alastair Henry
Migration and Language Education Published: 29 April, 2022, Volume 3(1), 1–22. Download PDF


The role of literature in intercultural language education
Melina Porto, Michalinos Zembylas
Intercultural Communication Education Published: 10 December, 2022, Volume 5(3), 86–104. Download PDF