Teacher vs. AI: a comparison of the effectiveness of corrective written feedback provided by either a teacher or AI, and the influence of students' attitudes on the outcome

Thesis

In contemporary times, Artificial Intelligence (AI) has achieved a level of indispensability, yet there persists a lack of academic research that specifically addresses its practical implementation within the context of daily pedagogic practice. One potential benefit of the integration of AI into the field of education is its capacity to enhance the effectiveness of written corrective feedback (WCF), thereby reducing teachers' workload. However, in order to effectively accomplish this objective, it is imperative to consider not only whether AI is performing the task with the same level of proficiency as a human teacher, but also to examine how students respond to AI feedback and whether this response has a significant impact on their learning outcomes. In order to provide a more comprehensive understanding of the issue, a mixed-method study was conducted in a Swiss vocational school. The students (n=21) participated in two interventions, one teacher-led and one AI-led, to determine if their writing performance improved more, after receiving teacher or AI feedback. Further, they answered a questionnaire to see if they reacted differently to teacher and AI feedback and to determine whether they believed one of the interventions to be more useful than the other and if this perceived usefulness had any influence on their later performance. In addition, the learners kept a learning journal to provide deeper insight into the reactions to the feedback. In three out of four pre- and post-test, there was a strong interrater reliability between the marks given by AI and the ones given by teachers, confirming earlier studies. However, contrary to former studies, there was no evidence found that performance improved following either teacher or AI feedback when comparing the gain scores from pre- to post-test, nor was there evidence that one intervention was more successful than the other. Also, there was no evidence that performance was influenced by the perceived usefulness of the intervention. Interestingly, the learners’ subjective impressions and the objective measures of performance sometimes diverged widely, suggesting that teachers should exercise a certain sensitivity when dealing with AI for classroom intervention. Nevertheless, if AI is implemented carefully, so that students feel confident in using it, it has the potential to alleviate teachers’ workload and provide substantial assistance to students in enhancing their writing abilities.

Authors: Ulrich, I

• DOI: 10.5287/ora-2ayyapg0g
• Type of award: MSc taught course
• Level of award: Masters
• Awarding institution: University of Oxford
• Language: English
• Keywords: AI feedback; qualitative; teacher feedback; quantitative; ALLT2025; mixed method; journal; comparison; effectiveness; WCF; students' attitude; ALLT; questionnaire
• Subjects: Teacher's Report Form; Qualitative research; Quantitative research; Grading and marking (Students); Qualitative research--Methodology; Educational evaluation; Teacher-student relationships; Mixed methods research; Artificial intelligence; Effective teaching; Applied linguistics; Computer-assisted instruction