AI Tutoring Cut US College Grades 0.37 SD While a UK Trial Found Gains, RCTs Show

A randomized controlled trial posted in October 2026 found that giving university students access to an AI course tutor lowered their final grades rather than raising them. The study, led by Jing Liu and Tracy Sweet and posted as EdWorkingPaper No. 26-1598 through the Annenberg Institute at Brown University, followed 2,379 undergraduates across 30 instructors and multiple disciplines at a large US public university. In the same month, a separate trial run in English secondary schools and posted to arXiv on September 13, 2026 found the opposite: students who used a different AI tutoring tool for GCSE science scored higher than those who revised on their own.

Both studies are randomized trials, the design researchers trust most for isolating a tool's effect from the many other things that separate high- and low-performing students. Both were run in 2026. And both targeted secondary outcomes beyond test scores. Yet one found a drop of 0.37 standard deviations in final grades, equivalent to about 4.12 points on a 0-100 scale, and the other found a gain of 0.33 standard deviations in post-test attainment. Reading them side by side says more about how AI tutoring succeeds or fails than either result does alone.

A university course tutor that backfired

In the Maryland-led trial, students in randomly chosen sections of the same course were offered access to a GPT-4o-based tutor integrated into their coursework; students in other sections of the same course were not. Access reduced final grades by 0.37 standard deviations and course letter grades by 0.13 standard deviations, the authors report. Engagement with the learning management system fell even more sharply: page views dropped 0.37 standard deviations, active days declined 0.51 standard deviations, and total participation fell 0.90 standard deviations among students offered the tool.

Only 14.6% of students who were offered the tutor actually used it, according to the paper, and of those who did, 73.8% of their requests asked for direct answers or explanations rather than for feedback or practice. The authors conclude the tutor "displace[d] engagement with existing learning activities without improving academic performance." The losses were largest among first-generation students, who saw roughly double the academic decline of continuing-generation students — a finding the authors flag as an equity concern for any unstructured rollout of AI tools in required coursework.

A secondary-school science tutor that worked

The UK trial, led by Wayne Harrison, Rahil Khowaja, Emma Dobson, Germaine Uwimpuhwe and Steve Higgins of Durham University, took a different shape. It ran as a four-week, individually randomized "micro-randomised" trial across English secondary schools, comparing an AI tutoring platform called Medly against students' usual self-directed revision in GCSE Biology, Chemistry and Physics. Of 929 students who completed a baseline assessment, 644 completed post-testing. Students assigned to Medly showed higher post-test attainment overall (Hedges' g = 0.33, 95% CI 0.18–0.48), with gains in Physics (g = 0.31), Chemistry (g = 0.32) and Biology (g = 0.52), and the paper reports no differential impact by disadvantage status.

Why usage, not the technology, decides the result

A third relevant trial helps explain the gap. A two-year cluster-randomized study of Khan Academy's Khanmigo tutor across 18 Tennessee middle schools, led by Philip Oreopoulos and Nina Low and also posted through the Annenberg Institute, found modest gains of 1.3 national percentile ranks per term, or roughly 0.06 to 0.08 standard deviations over a school year. Ninety-six percent of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced. "The binding constraint appears to be engagement," the authors write: "realizing the promise of AI tutoring will require getting students to use it, not just giving them access."

Taken together, the three trials point less to a verdict on AI tutoring in general than to a pattern in how it is deployed. The UK's Medly trial embedded the tool directly into exam preparation with practitioner oversight and a specific, time-boxed task. The Tennessee trial relied on daily routine use inside an existing remedial program but still saw inconsistent engagement. The Maryland trial offered an open-ended, general-purpose tutor inside a university course with no required use, and most students who touched it used it to get answers rather than to practice — the use pattern most likely to substitute for, rather than add to, learning.

What it means for students, teachers and researchers

For researchers, the comparison is a caution against reading any single randomized trial as the final word on "AI tutoring" as a category: design, subject, age group and how a tool is integrated into existing coursework all shape the result, sometimes reversing its sign. For instructors and institutions, the Maryland findings are a specific warning about unstructured access to general-purpose tutors in place of required engagement with course materials, particularly for first-generation students. For students, the throughline across all three trials is the same: a tool's presence does not predict its effect, and how the tool is actually used — for practice and feedback, or for answers — does.