Paper Mills, Editors, and Systematic Reviews: What a 2025 Fraud-Network Study Teaches About Reading Evidence

Most advice on reading a paper critically focuses on honest error: underpowered samples, flexible statistics, a discussion section that overreaches its data. That advice assumes the paper was written by someone trying, however imperfectly, to describe something real. A 2025 study in the Proceedings of the National Academy of Sciences is useful precisely because it addresses the case where that assumption fails outright — where the manuscript was manufactured to be published, not to report anything.

The study, by Reese Richardson, Spencer Hong, Jennifer Byrne, Thomas Stoeger and Luís Amaral, is not another commentary on the existence of "paper mills," the commercial operations that write and sell fabricated manuscripts, authorship slots and even entire peer-review reports. It is a piece of empirical detective work: the team mined editorial metadata from PLOS ONE, retraction records from Hindawi and IEEE conference proceedings, PubPeer comments, and image-duplication flags, then used network analysis to trace which editors, authors and brokers kept reappearing around the same clusters of retracted papers, as described in the paper itself.

The headline number is a growth-rate comparison, not a raw count. The authors calculate that the population of suspected paper-mill articles has been doubling roughly every 1.5 years, while the overall scientific literature doubles roughly every 15 years — a tenfold difference in pace, reported in coverage of the study by Retraction Watch. A growth-rate gap of that size means the proportion of the literature affected keeps rising even if the absolute number of legitimate papers is also rising — which is exactly why treating fraud as a fixed, contained problem understates it.

What makes the paper a genuine methods lesson, rather than just an alarming statistic, is how the team turned "this looks suspicious" into something countable. Rather than reading manuscripts one at a time, they treated editorial handling, image reuse and co-authorship as data: who is assigned which submissions, how often a given editor's papers get retracted relative to their peers, and how often the same images turn up attached to different papers under different author names. That is a template any reader can borrow in miniature — not by running the same analysis, but by asking the same question of any strong claim: is this an isolated result, or does it sit inside a pattern that a wider look would expose?

Following the brokers: what the network analysis found

At PLOS ONE, the authors identified 45 editors — a small fraction of the journal's editorial board — who together handled about 1.3 percent of the journal's 2024 output but were responsible for 30.2 percent of its retracted articles; more than half of those editors had also authored papers that were later retracted themselves, according to the study as summarized by Retraction Watch. That is not noise. It is the statistical signature of a small number of people acting as what the authors call brokers — editors who wave fabricated submissions through review in exchange for money or co-authorship, concentrated far more tightly than chance handling would predict.

The image-duplication analysis tells a similar story from a different angle. Among papers flagged for suspected duplicated images, 34.1 percent had already been retracted — a much higher rate than for the literature generally — and subfields the authors identified as compromised by paper-mill activity showed retraction rates around 4 percent, against roughly 0.1 percent in uncompromised subfields, per the same reporting. Both numbers point the same way: fraud is not evenly spread across science, it clusters in identifiable pockets of journals, editors and subfields, which is exactly why aggregate retraction rates for a whole discipline can look reassuringly low while specific corners of it are saturated.

The paper also documents how these operations adapt once caught. One entity described in the study, referred to as ARDA, kept shifting its list of "friendly" journals as individual titles were de-indexed by the databases that libraries and researchers rely on to filter reputable venues — a pattern Chemistry World described as journal-hopping in direct response to enforcement. That matters for readers because it means a journal's current indexing status tells you about today, not about the papers it published two or three years ago under different editorial oversight.

Where fabricated papers go after publication: the systematic-review problem

A separate 2025 study in JAMA Network Open, by Gengyan Tang and Hao Cai, asks a question the PNAS paper doesn't: once a paper-mill article is published and later retracted, does the damage stop there? The authors searched roughly 200,000 life-sciences systematic reviews indexed in Web of Science and published between 2013 and 2024, matching their reference lists against known retracted paper-mill articles, as detailed in the published study.

They found that 299 of those 200,000 reviews — about 0.15 percent — had incorporated a retracted paper-mill article into their evidence synthesis: 256 reviews cited one such article, 43 cited more than one, for 385 contaminating citations in total. Oncology reviews were disproportionately affected, accounting for 48 of the 299 contaminated reviews, or 16.1 percent. A small number of reviews were especially heavily affected: five cited five or more retracted articles each, and one cited thirteen, with several of the worst cases concentrated in journals from a single publisher, Multidisciplinary Digital Publishing Institute (MDPI).

The most uncomfortable finding is about timing. Of the 385 contaminating citations, 32.2 percent — 124 citations — occurred after the cited article had already been retracted, and 13 of those citations happened more than 500 days after the retraction date. Retraction notices exist precisely to stop a discredited finding from propagating further, and in roughly a third of these cases they didn't. A systematic review is supposed to be the more trustworthy summary a reader turns to instead of chasing primary studies individually; this study shows that trust can inherit contamination the review's own authors never checked for.

What a reader can actually check

Neither study is really about paper mills as a curiosity. Both are demonstrations that verification is a set of concrete, repeatable checks, not a feeling of confidence in a journal's name or an author's affiliation. A few of those checks are within reach of any individual reader, not just a research-integrity team with database access:

• Check the retraction status of everything you cite, not just the paper in front of you. Reference managers can automate this: Zotero, for instance, cross-checks a user's library against the Retraction Watch database and flags matches even after an item has been added to a document, as described in Zotero's own documentation of the feature.
• Treat a journal's current indexing status as a snapshot, not a history. The ARDA pattern documented in the PNAS study shows that de-indexing follows discovery, sometimes years after suspect articles were published — so a currently reputable-looking journal may still carry unretracted paper-mill output from an earlier period.
• Be more skeptical of citations inside systematic reviews and meta-analyses than the review's own authority suggests you should be. The Tang and Cai study shows that reviewers doing careful evidence synthesis missed retracted paper-mill sources at a nontrivial rate, and did so more in some fields (oncology) and some publishers than others.
• Notice clustering, not just individual red flags. A single retracted paper by an editor or author may be an honest error; the PNAS study's method was to look for editors and authors who appear repeatedly across many retracted or suspect papers, which is a much stronger signal than any one case in isolation.

The limit of all this: detection lags production

The PNAS authors are explicit that current detection is incomplete in a specific, measurable way. Only about 28.7 percent of the suspected paper-mill products they identified had been retracted at the time of the study, and extrapolating from current trends, they estimate that only around 25 percent of suspected paper-mill products will ever be retracted, with only around 10 percent ever ending up in a journal that gets de-indexed, according to the findings reported by Retraction Watch.

That asymmetry is the practical takeaway for anyone trying to evaluate evidence rather than just study the phenomenon of fraud from a distance. The absence of a retraction notice on a paper is not evidence that the paper is legitimate — it may simply mean detection hasn't caught up yet, particularly in a fast-doubling population of fabricated articles chasing a much slower-growing base of legitimate ones. Co-author Luís Amaral put the asymmetry bluntly to Retraction Watch: fighting coordinated publication fraud with today's tools is "like emptying an overflowing bathtub with a spoon."

None of this means every unfamiliar paper is suspect, or that retraction databases and image checks substitute for actually reading the methods. It means the checklist for reading a paper carefully now has to include questions that didn't used to be necessary: has this article, or the ones it cites, been through a retraction check; does the editor or journal show up repeatedly in reporting on fraud networks; is a systematic review's reference list actually clean, or just assumed to be. The two studies together are a reminder that reproducibility and honesty are not the same failure mode, and that the tools for catching the second — network analysis, image forensics, citation-timing checks — are different from the tools for catching the first, and increasingly necessary to know about.