Volume 1, Issue 2  November 2018, pp. 45-58          Download PDF

Articles
The impact of spelling errors on trained raters' scoring decisions

Ikkyu Choi https://orcid.org/0000-0002-8281-72111, Yeonsuk Cho https://orcid.org/0000-0002-4997-61062

1 Educational Testing Service, USA ichoi001@ets.org
2 Educational Testing Service, USA ycho@ets.org

DOI: https://doi.org/10.29140/lea.v1n2.61


Abstract

Second language (L2) writing assessments seldom allow a spellchecker and often have a time limit. Naturally, test takers often submit responses with spelling errors. However, little is known about whether and how spelling errors in test taker responses affect trained raters' scoring decisions. In this study, we investigated the impact of spelling errors on trained raters' holistic evaluation of response quality. We selected 148 responses to four L2 writing tasks, and created error-free versions of the responses by correcting spelling errors in the original responses. Both the original and corrected responses were randomly assigned to trained raters, who scored the responses according to the same holistic scoring rubrics. We compared the resulting scores of original and corrected responses to gauge the impact of spelling errors. We also examined whether the impact of spelling errors varied across different task types and spelling error characteristics. The results showed that the spelling error correction led to an average of more than a half score point increase. The score gains from the error correction varied across task types and the quantity of corrected errors. These findings have multiple implications, including suggestions for rater training and assessment development.



Copyright

© Ikkyu Choi, Yeonsuk Cho

CC BY-NC 4.0
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Suggested citation

Choi, I., & Cho, Y. (2018). The impact of spelling errors on trained raters' scoring decisions. Language Education & Assessment, 1(2), 45-58. https://doi.org/10.29140/lea.v1n2.61


Related Articles:

Incorporating hypnotic suggestion into teacher education programs
Farshad Ghasemi
Australian Journal of Applied Linguistics Published: 31 December, 2019, Volume 2(3), pp. 83-103. Download PDF

What makes students speak Japanese in immersion programs?
Kumiko Katayama, Kayoko Hashimoto
Australian Journal of Applied Linguistics Published: 31 December, 2019, Volume 2(3), pp. 104-120. Download PDF

Voicing the academy
Davina Allison
Australian Journal of Applied Linguistics Published: 31 December, 2018, Volume 1(3), pp. 102-117. Download PDF

Theory of Mind development and narrative writing: A longitudinal study
Birgitta E. Svensson
Australian Journal of Applied Linguistics Published: 31 December, 2018, Volume 1(3), pp. 118-134. Download PDF

Incidental vocabulary learning through watching movies
Robert John Ashcroft, Joseph Garner, Oliver Hadingham
Australian Journal of Applied Linguistics Published: 31 December, 2018, Volume 1(3), pp. 135-147. Download PDF

Note from the editor
Troy McConachy
Intercultural Communication Education Published: 20 December, 2019, Volume 2(3), pp. 101-101. Download PDF

The Council of Europe Reference Framework of Competences for Democratic Culture: Ideological refractions, othering and obedient politics
Ashley Simpson, Fred Dervin
Intercultural Communication Education Published: 20 December, 2019, Volume 2(3), pp. 102-119. Download PDF

"I don't want to be stereotypical, but..." Norwegian EFL learners' awareness of and willingness to challenge visual stereotypes
Cecilie Waallann Brown
Intercultural Communication Education Published: 20 December, 2019, Volume 2(3), pp. 120-141. Download PDF

(Re)imagining a course in language and intercultural communication for the 21st century
Adriana Raquel Diaz, Paul J. Moore
Intercultural Communication Education Published: 29 December, 2018, Volume 1(3), pp. 83-99. Download PDF

Critical intercultural communication education: cultural analysis and pedagogical applications
Sofia A. Koutlaki, Zohreh R. Eslami
Intercultural Communication Education Published: 29 December, 2018, Volume 1(3), pp. 100-109. Download PDF