Retrieval Practice and the Testing Effect: What Actually Replicates

Ask a room of teachers what helps students remember material, and someone will mention "testing yourself." The claim has unusually strong backing in learning science compared with much of the advice that circulates in classrooms — but the evidence has edges, and a live dispute among researchers over where those edges fall has not been settled. Separating what is replicated, what rests on a handful of studies, and what is simply popular is the point of this piece.

The classic demonstration

The modern wave of interest traces to a single study: Henry Roediger III and Jeffrey Karpicke's 2006 paper in Psychological Science, "Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention." Students studied prose passages under different combinations of restudying and free-recall testing — for instance, four rounds of study (SSSS) versus three rounds of study followed by a test (SSST), or one round of study followed by three tests (STTT). Tested five minutes after the last learning session, the repeated-study group recalled the most, around 83%, exactly as more exposure would predict. Tested again one week later, the ranking reversed: the repeated-study group had dropped to roughly 40%, while the group that had practiced retrieval three times held at roughly 61%. Repeated studying produced better performance in the short term and worse performance in the long term than repeated testing.

That reversal is the core of the testing effect: the act of retrieving information from memory, not just being exposed to it again, changes how durably it is stored. The 2006 paper's finding mattered partly because it used meaningful prose rather than the word lists common in earlier memory research, which made the result harder to dismiss as a laboratory curiosity.

What the meta-analysis adds

A single striking study is not the same as a replicated effect. The strongest evidence that the testing effect generalizes comes from Cynthia Rowland's 2014 meta-analysis in Psychological Bulletin, which pooled 159 effect sizes from 61 studies published between 1975 and 2013, every one comparing a test condition against a restudy condition. The overall effect favored testing, with a mean effect size of about 0.50 standard deviations. Of the 159 individual comparisons, 81% favored testing, 18% favored restudy, and roughly 1% showed no difference. The analysis also found that intermediate tests with feedback produced a larger benefit (about 0.73 standard deviations) than tests without feedback (about 0.39), a moderator worth separating from the headline number because it changes how large an effect a given classroom practice should expect.

That combination — a scale of 159 comparisons and a consistent direction in roughly four out of five of them — is what distinguishes the testing effect from a one-off finding. Few recommendations in learning science carry that weight of replication.

Extending beyond rote recall

A separate question is whether retrieval practice helps only with fact recall, or also with the kind of understanding that requires making inferences. Jeffrey Karpicke and Janell Blunt addressed this directly in a 2011 study in Science, comparing retrieval practice against elaborative studying with concept mapping — the technique of building a diagram of linked ideas, often held up as a deep-processing study method in its own right. Across two experiments with 200 students studying science texts, retrieval practice produced more learning than concept mapping, including on test questions that required inference rather than verbatim recall, and even when the final test itself involved building a concept map. The result suggested the benefit of testing is not confined to surface-level memorization, though this was one study, not a body of replications on the scale of Rowland's meta-analysis.

Where the evidence is contested

The most important caveat concerns material complexity, and it is genuinely unresolved rather than settled science dressed up as doubt. In 2015, Tamara van Gog and John Sweller argued in Educational Psychology Review that the testing effect "decreases or even disappears as the complexity of learning materials increases" — pointing to research suggesting that when material has many interacting elements that must be held in mind together, as in complex problem-solving, generating answers through testing can push working memory past a useful threshold rather than deepen learning.

Karpicke, working with William Aue, published a direct rebuttal in the same journal the same year, arguing that Van Gog and Sweller's review omitted studies that had shown retrieval-practice benefits with complex, multi-element material, including some that manipulated complexity directly. The two camps did not converge; the special issue of Educational Psychology Review that carried both papers simply recorded the disagreement. A 2024 paper by Wesley Pyke, Johan Lunau and Amir-Homayoun Javadi, published in the Quarterly Journal of Experimental Psychology, proposed a reconciliation rather than a verdict: that task difficulty should be matched to the material's complexity and the learner's expertise, with retrieval-style difficulty helping more for simple material and novices, and added difficulty becoming counterproductive as element interactivity and expertise demands rise.

Classroom evidence, not just the laboratory

Laboratory robustness is one thing; whether the effect survives contact with a real classroom, a real curriculum and a real teacher's workload is another. Roediger, together with Pooja Agarwal, Mark McDaniel and Kathleen McDermott, reported in a 2011 Journal of Experimental Psychology: Applied study that low-stakes quizzing embedded in an actual middle-school social-studies course over a year and a half produced measurably better performance months later on material that had been quizzed versus material that had not, with the teacher otherwise continuing to teach as normal. That the quizzes were low-stakes and covered only part of the taught material is itself part of the finding: it is a specific, bounded claim about what was tested, not a general claim that any quizzing improves any learning.

What follows for studying and teaching

Pulling the threads together, three tiers of confidence are visible:

• Well replicated: that retrieving information from memory, rather than re-reading or re-studying it, improves long-term retention, with Rowland's 2014 meta-analysis providing the broadest evidential base across decades of individual studies.
• Supported by fewer but substantive studies: that the benefit extends beyond rote recall to inference-based understanding (Karpicke and Blunt, 2011), and that it survives translation into real classrooms with low-stakes quizzing over an extended period (Roediger, Agarwal, McDaniel and McDermott, 2011).
• Actively contested: whether and how the benefit shrinks for genuinely complex, high-element-interactivity material, where Van Gog and Sweller's 2015 argument and Karpicke and Aue's 2015 rebuttal reached opposite readings of the same literature, and where a 2024 proposal to match difficulty to expertise and material complexity has not yet settled the matter.

For a student deciding how to revise, the first tier is the one to act on with confidence: closing the book and trying to recall material, rather than re-reading it, is a change supported by a large and consistent body of evidence. For a student or teacher working with genuinely difficult, multi-part material — a derivation, a multi-step diagnostic process, a proof — the honest answer is that researchers who have looked closely at the same body of work disagree about how much, if any, of the plain testing-effect benefit carries over, and no single study or meta-analysis has yet closed that gap.