AI or peer feedback: what works best in improving writing?

Thesis

Feedback is a key driver of student learning, yet teachers often struggle to provide detailed written corrective feedback (WCF) at scale. Alternatives such as peer review and automated writing evaluation (AWE) have therefore gained attention, but their relative effectiveness remains underexplored, particularly in low-resource contexts. This study investigates whether AI- or peer-generated feedback leads to greater improvements in writing, whether the two differ in scoring and the types of feedback they produce, and how students perceive and evaluate them.

The research involved 36 female undergraduates in a B.Ed. program in Karachi, Pakistan, randomly assigned to AI feedback (n=12), peer feedback (n=11), or control (n=13) conditions. All participants wrote an initial essay, reviewed a peer’s draft, and revised their own essay either in light of feedback (AI or peer) or no feedback (control). Both drafts were scored on a 54-point rubric across three domains: content and organisation, language use, and mechanics. Gain in scores across drafts were calculated. Quantitative analysis (ANCOVA) was complemented by a survey and independent coding of feedback quality. Participants language proficiency was accounted for.

Results showed that AI feedback yielded the largest mean gains (M=5.0), followed by peer feedback (M=3.9) and control (M=2.9), though differences were not statistically significant. Importantly, AI feedback produced a significant interaction with proficiency in mechanics: less proficient students showed greater gains when revising with AI. Qualitative data revealed that AI comments were more directive and elaborated, while peer feedback leaned toward praise and surface-level suggestions. Students found AI comments more comprehensive and accurate.

Authors: Mahmood, S

• DOI: 10.5287/ora-vg6bvxvko
• Type of award: MSc
• Level of award: Masters
• Awarding institution: University of Oxford
• Language: English
• Keywords: feedback; peer feedback; AI feedback
• Subjects: Education