Volume 1, Issue 2 November 2018, pp. 45–58 Download PDF

Articles

The impact of spelling errors on trained raters' scoring decisions

Ikkyu Choi https://orcid.org/0000-0002-8281-72111, & Yeonsuk Cho https://orcid.org/0000-0002-4997-61062

1 Educational Testing Service, USA ichoi001@ets.org
2 Educational Testing Service, USA ycho@ets.org

DOI: https://doi.org/10.29140/lea.v1n2.61


Abstract

Second language (L2) writing assessments seldom allow a spellchecker and often have a time limit. Naturally, test takers often submit responses with spelling errors. However, little is known about whether and how spelling errors in test taker responses affect trained raters' scoring decisions. In this study, we investigated the impact of spelling errors on trained raters' holistic evaluation of response quality. We selected 148 responses to four L2 writing tasks, and created error-free versions of the responses by correcting spelling errors in the original responses. Both the original and corrected responses were randomly assigned to trained raters, who scored the responses according to the same holistic scoring rubrics. We compared the resulting scores of original and corrected responses to gauge the impact of spelling errors. We also examined whether the impact of spelling errors varied across different task types and spelling error characteristics. The results showed that the spelling error correction led to an average of more than a half score point increase. The score gains from the error correction varied across task types and the quantity of corrected errors. These findings have multiple implications, including suggestions for rater training and assessment development.



Copyright

© Ikkyu Choi, Yeonsuk Cho

CC  4.0
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Suggested citation

Choi, I., & Cho, Y. (2018). The impact of spelling errors on trained raters' scoring decisions. Language Education and Assessment, 1(2), 45–58. https://doi.org/10.29140/lea.v1n2.61


Related Articles:

Beginning again
Glenn Stockwell
Australian Journal of Applied Linguistics Published: 1 April, 2018, Volume 1(1), 1–2. Download PDF


Insights from replication on the factors affecting task engagement in mobile-based learning activities
Glenn Stockwell
Technology in Language Teaching and Learning Published: 28 June, 2019, Volume 1(1), 33–51. Download PDF


Syntactical and lexical development in NNS-NNS Asynchronous CMC
Glenn Stockwell
The JALT CALL Journal Published: 31 December, 2005, Volume 1(3), 33–49. Download PDF


Investigating an intelligent system for vocabulary learning through reading
Glenn Stockwell
The JALT CALL Journal Published: 31 December, 2013, Volume 9(3), 259–274. Download PDF


Review of Developing writing skills for IELTS: A research-based approach
Hassan Mohebbi
Australian Journal of Applied Linguistics Published: 28 April, 2021, Volume 4(1), 34–36. Download PDF


Making global knowledge accessible to EFL speakers of an undergraduate leadership program through a flipped and ubiquitous learning environment
Nobue Tanaka-Ellis, Sachiyo Sekiguchi
Technology in Language Teaching and Learning Published: 28 June, 2019, Volume 1(1), 3–20. Download PDF


Encouraging autonomy through the use of a social networking system
Adrian Leis
The JALT CALL Journal Published: 30 April, 2014, Volume 10(1), 69–80. Download PDF


Exploring EFL teachers’ professional identity development in a CALL teacher preparation program
Hussein Meihami, Rajab Esfandiari
The JALT CALL Journal Published: 25 August, 2021, Volume 17(2), 135–157. Download PDF


(Re)imagining a course in language and intercultural communication for the 21st century
Adriana Raquel Díaz, Paul J. Moore
Intercultural Communication Education Published: 29 December, 2018, Volume 1(3), 83–99. Download PDF


Reframing and hospicing mobility in higher education: Challenges and possibilities
Adriana Raquel Díaz, Marisa Cordella, Samantha Dispray, Barbara E. Hanna, Anna Mikhaylova
Intercultural Communication Education Published: 19 April, 2021, Volume 4(1), 106–121. Download PDF