A Reviewer Checked a Paper Against Her Own Dataset. The Numbers Didn't Match

In August 2026, Dorothy Bishop, an emeritus professor of developmental neuropsychology at the University of Oxford, agreed to review a manuscript for a neuroscience journal. The abstract described an analysis of an OpenNeuro dataset she recognized immediately: it was her own, deposited four years earlier as part of the CANDICE project comparing language lateralisation measured by fMRI and functional transcranial Doppler sonography, and containing raw scans from 50 participants doing language tasks, as reported by The Scientist.

The dataset itself, registered on OpenNeuro as ds004073, lists 51 participants across two testing sites, Bangor and Oxford, who completed six language tasks — word and sentence generation, phonological and syntactic decision, and word and sentence comprehension — during fMRI sessions, according to the dataset's own OpenNeuro listing. That listing, with its task names, trial counts and participant codes, is exactly the kind of documentation a reviewer, or any reader, can hold a manuscript up against.

Bishop did exactly that, and the manuscript didn't hold up. The description of what had supposedly been done to the data didn't match what she and her collaborators had actually recorded, and several of the paper's citations didn't support the claims attached to them, she told The Scientist. "There was clearly something seriously wrong," she said. The editor rejected the submission, and Bishop suspected, without being able to confirm it, that the authors would try their luck elsewhere.

That suspicion sent her looking, and the search turned up a pattern. Two papers already published in 2026 in the Journal of Neurolinguistics, an Elsevier title, claimed to analyze OpenNeuro datasets in ways that, on inspection, didn't line up with what those datasets actually contain, Bishop and the Royal Holloway, University of London psychologist Kathy Rastle told Retraction Watch.

The first, published in May 2026 by Afzal Khan of King Saud University's English Language Skills Department and Dawood Khan of Iqra University, used functional MRI scans of language tasks that Rastle and colleagues had deposited, running a brain-connectivity analysis on them. It reported that behavioral scores were available for only some participants, when the dataset actually has them for everyone, and cited reading-comprehension scores the dataset does not contain at all, per The Scientist's account. A figure in the paper mapped reading activity onto brain regions that, Rastle said, "bore no resemblance to what we know about reading in the brain."

The second paper, published in August 2026 and credited to Afzal Khan alone, claimed to draw on four OpenNeuro datasets and reported participant counts that didn't match any of them. One figure showed a participant who had reportedly used a second language for more than 50 years, in a dataset whose own maximum value for that variable is 37, as The Scientist reported. Khan disputed that the figure exceeded 50 at all, a response that read, to Bishop and Rastle, as further evidence that nobody on the author list had actually checked the graph against the data it was supposed to represent. Rastle put it bluntly to Retraction Watch: "I don't believe that they actually used our data. I believe that they said they did in order to create a veneer of respectability."

What makes this case useful beyond its own details is what Bishop wrote about why she almost didn't catch it. In a post on her blog describing the August manuscript, she noted that the paper wasn't the kind of obviously formulaic fraud that trips up a careless reviewer: it asked "an original and interesting question," and its results were "rather messy, raising more questions than answers," the kind of untidy outcome real research produces and fabricated research usually avoids, because messiness reads as credible. She wrote that detection methods built around spotting implausibly clean results "now seem inadequate" against this. The point matters for any reader, not just peer reviewers, because it says plausibility and tidiness are not the test. Matching the source is.

Bishop's post sets out a short list of checks a referee can run even without re-executing an analysis, and most of them translate directly to a reader working from a PDF next to an open repository's documentation page:

• Compare the paper's stated sample size, after any exclusions, against the participant count the dataset's own listing gives.
• Check whether a variable the paper reports, a score, a demographic field, a scan parameter, actually appears in the dataset's documentation at all.
• Check whether any reported value falls outside the range the source data can produce, the way a 50-plus-year figure cannot come from a field whose recorded maximum is 37.
• If code is provided, try running it against the stated data rather than assuming a working repository link means a working analysis.
• Spot-check a handful of citations against the claims they're attached to; misaligned citations were one of the signals in the manuscript Bishop rejected.
• Ask whether the described workload is realistic for the listed authors and the time between data availability and submission.

None of these require specialist statistical training. They require opening the dataset's own page next to the paper and reading both at once, which is also the main reason this kind of mismatch is catchable at all: OpenNeuro, like other public repositories, publishes its documentation where anyone can check it, rather than keeping sample sizes and variable lists behind a data-access agreement only the original authors can see.

As of the Retraction Watch report, neither paper had been retracted. Bishop had asked the Journal of Neurolinguistics, whose editor Nai Ding was named in the coverage, to retract both. Elsevier told Retraction Watch it was investigating but that the process "may take considerable time." Khan told the outlet he intended to redo the analyses and submit corrections rather than accept that the published results were unsupported.

The slow pace of that response is itself part of the pattern this case illustrates. A paper can be flagged within days of a careful read against its own cited data, and still sit in the published record, cited and indexed, for months or years while a publisher's investigation proceeds. That gap is a reason to run the checks above as a reader rather than waiting on an editor: nothing stops anyone, a student using the paper as a citation, a researcher building on its claimed findings, from running the same comparison against the dataset's listing before relying on the number.

The broader point does not depend on whether either Journal of Neurolinguistics paper used generative AI to produce its text, something Bishop and Rastle suspect but that neither the publisher nor the authors have confirmed. A paper's claim to have reused a named open dataset is a specific, checkable claim, no different in kind from a claim about a drug dose or a survey's sample frame. The qualifiers that travel with it, how many participants after exclusions, what variables the data actually contains, what range of values the source permits, are not incidental details to skim past. They are the part of the paper a reader with no lab access and no statistical software can verify directly, by opening the same public page the authors say they worked from.