ChatGPT use patterns and higher-order thinking performance in EFL argumentative writing

Abstract

The growing use of ChatGPT in higher education raises questions about its relationship with students' higher-order thinking (HOT) and how students engage with AI during learning. This study examined whether structured ChatGPT-supported instruction was associated with EFL students' HOT performance in argumentative writing and their patterns of ChatGPT use during the intervention. A quasi-experimental mixed-methods design involved 100 fourth-semester undergraduates drawn from two intact classes, comprising an experimental group (n = 50) and a control group (n = 50). Over 12 weeks, the experimental group completed structured ChatGPT-supported argumentative tasks, while the control group completed comparable tasks through conventional instruction. HOT performance was assessed through supervised pretest and posttest argumentative essays measuring analysis, evaluation, and creation. ANCOVA showed that the experimental group achieved significantly higher adjusted posttest HOT scores than the control group, with creation showing the largest dimension-level difference. Interaction logs showed that reflective-evaluative use was more common than product-oriented use during the supervised ChatGPT-supported tasks. Students frequently evaluated AI suggestions, compared alternatives, and revised selectively. These patterns reflected students' use of the prompting, evaluation, and revision practices introduced during the intervention and were interpreted in relation to cognitive amplification and cognitive offloading. The findings highlight the value of structured AI-supported tasks that maintain students' responsibility for evaluation and revision.