Expert: Dr Deborah Talmi, Psychologist, University of Cambridge; lead of the OpRaise project on AI in assessment The gap was not random noise, and it is not explained by the models being set a trick task. The essays were genuine: 761 long-form undergraduate psychology essays from 125 students at the University of Cambridge (133), the University of Nottingham (172) and Manchester Metropolitan University (456), submitted for formal assessment between 2022 and 2025 and marked through each institution's normal moderation process, spanning 50 modules and 87 distinct assignments. Each model was systematically tested across prompt variants covering criteria specificity, calibration and scoring strategy, so the comparison used the best-performing prompt for every system. The failure has a shape. First, band-level agreement was inconsistent across institutions - 63% at Cambridge, 53% at Nottingham, 35% at Manchester Metropolitan - a spread the authors attribute to the range of marks at each institution: narrowest among Cambridge's invigilated exam-hall essays, widest at Manchester Metropolitan, where all analysed essays were coursework. Second, the bias is central tendency. AI and human marks most often coincide in the upper 50s to low 60s, around a low 2:1 - the centre of the grade distribution - while the extremes are misjudged: top essays are routinely undervalued and the weakest essays overvalued. As co-author Dr Alexandru Marcoci of Cambridge's Institute for Technology and Humanity puts it, human assessors judge each essay on its own argumentative and conceptual merits while AI marks are statistical predictions, so the models assign middling marks to everything. The practical consequence is that AI is least accurate precisely at the boundaries that matter most - First versus Upper Second, pass versus fail. Third, all three systems rewarded surface features: essay length, vocabulary range and sentence complexity, attributes the report notes are often unrelated to academic standards. Consistency is not accuracy. Re-marking the same essays produced the same or similar marks each time, and the different models sat far closer to each other than to human examiners - they agree with themselves and with one another, not with the standard. The AI's student feedback was also three to eight times longer than the human assessors' feedback; when it was trimmed to a comparable word count, focus groups of staff and students struggled to tell it apart, though once the author's identity was revealed not everyone valued the insights. The report, 'AI in University Assessment: Evaluating the Opportunities and Risks of Automated Marking' (OpRaise project, supported by ai@cam and the Accelerate Programme for Scientific Discovery), does not rule AI out of assessment entirely. It recommends the models for error detection, consistency checks - a 'second pair of eyes' - and triaging feedback, with large AI-human discrepancies used to flag assignments needing human review. What it rules out is the AI deciding the mark. "Leaning heavily on the best current AI models would see student grading that is homogenised, underestimates brilliance, and favours linguistic style over the substance of sound academic judgement," said Dr Deborah Talmi, the Cambridge psychologist who leads the OpRaise project, adding that assessment is part of how educational meaning is made and that leaning on AI puts that at risk. Dr Yael Benn of Manchester Metropolitan University, a collaborator on the project, reported that many students said they would feel cheated if AI marked their work, and that staff warned reliance on AI risks weakening trust, motivation and professional judgement. A human should always determine the final mark. Source: https://www.cam.ac.uk/stories/ai-university-essay-grading