Industries / Health / Pharmaceuticals / Clinical research
Analysis and reporting
Analysing a trial's results as planned, reporting them in registries and journals, and filing them with regulators in set formats so that others can check and reuse them.
7
Milestones achieved, from 1996 to 2018.
2
To watch for: announced, not yet confirmed outside.
3
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1996
A checklist for trial reports
CONSORT gave journals a minimum list of what the report of a randomised trial must state, with a diagram of how participants moved through it. It was updated in 2001, 2010 and April 2025, and now has 30 items, including a new section on open science.
Credit
C. Begg, D. Moher, K. F. Schulz, D. Rennie and colleagues (JAMA 1996); the 2025 update led by S. Hopewell of the Oxford Clinical Trials Research Unit
1998
Shared rules for trial statistics
ICH's guideline E9, final on 5 February 1998, set common principles for the statistics of trials submitted for approval. The UK put it into effect in March 1998 and Japan in November 1998, and China's regulator adopted it in 2020.
Credit
the ICH E9 Expert Working Group
2005
Register first, or no paper
The International Committee of Medical Journal Editors (ICMJE) announced in September 2004 that its journals would publish a trial starting on or after 1 July 2005 only if it had been entered in a public registry before the first patient enrolled. Since 1 July 2018 its journals have also required a statement on whether each trial's data will be shared.
Credit
ICMJE (C. De Angelis, J. M. Drazen and colleagues)
2014
Regulators' reports change a review
A Cochrane review of the flu drugs oseltamivir (Tamiflu) and zanamivir (Relenza) was built on 107 clinical study reports, the full reports companies file with regulators, obtained from EMA, GlaxoSmithKline and Roche, not on journal articles. In adults, oseltamivir shortened the time to first relief of symptoms by 16.8 hours, from 7 to 6.3 days, and had no significant effect on hospital admissions.
LessonJournal articles leave out much of what regulators see, and the makers released the reports only "after a protracted campaign".
Credit
T. Jefferson, M. A. Jones, P. Doshi, C. B. Del Mar, C. J. Heneghan and colleagues (Cochrane)
2015
Reanalysis reverses Study 329
Outside researchers reanalysed SmithKline Beecham's Study 329, a trial of antidepressants in teenagers, from its full data under the RIAT initiative ("restoring invisible and abandoned trials"). Neither paroxetine nor imipramine beat placebo on any pre-planned outcome, and harms rose, including suicidal thinking and behaviour on paroxetine.
LessonA published conclusion can be reversed once the full data and protocol are open to others.
Credit
J. Le Noury, J. M. Nardo, D. Healy, J. Jureidini and colleagues
2017
Electronic dossiers required
From 5 May 2017, applications to the FDA to sell new drugs, generics and biologics had to be filed in the electronic Common Technical Document (eCTD) format, a fixed structure of folders and files; commercial applications to start trials followed on 5 May 2018.
Credit
the FDA's drug and biologics centres (CDER and CBER)
2018
Missing results counted in public
EU and US rules require many trials to post results within a year of finishing. Oxford researchers built live public trackers that name each sponsor, and found that only 49.5% of 7,274 due EU trials had reported (2018) and only 40.9% of 4,209 due US trials had reported on time (2020). The FDA sent its first Notice of Noncompliance for missing results on 27 April 2021 and had sent 11 by October 2026.
LessonThe laws alone did not get results posted, and enforcement has been small: 11 notices against 32,441 US trials the tracker counted as due on 17 November 2026.
pubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.govfdaaa.trialstracker.netfda.gov
Credit
Ben Goldacre, Nicholas J. DeVito, S. Bacon and colleagues (EBM DataLab, University of Oxford); FDA
Watch for
Bayesian analysis for pivotal trials
The FDA's draft guidance on Bayesian statistics, which combine what was known before a trial with its new data, covers using them for the main analysis of pivotal trials, including how to set success criteria, justify prior assumptions and borrow data from outside the trial.
Achieved whenthe guidance is final and a public approval document shows a Bayesian main analysis as the basis for approval.
Claimed by
FDA, January 2026
AI tracks outcome changes
GPT-4o, used without extra training, compared each ClinicalTrials.gov record at the start of a trial with its final version, and found that 11,078 of 27,227 trials of FDA-regulated drugs (40.7%) had changed their main outcome, 47.7% of industry-funded trials against 32.9% of others. Against 100 records checked by hand it scored 0.88 for spotting added outcomes, 0.81 for removed ones and 0.67 for modified ones, on a scale where 1 is perfect.
Achieved whena peer-reviewed paper appears and another group reproduces the accuracy, or the method runs as a public monitor kept up to date.
Claimed by
Ian Bulovic, Timothy Miller, Florence Bourgeois and colleagues (Boston Children's Hospital), at the International Congress on Peer Review and Scientific Publication, September 2025
Open
01
Promised outcome versus published
For every finished trial whose results paper is linked to its registry record, compare the main outcome registered before the trial started with the one reported in the paper, flag outcomes added, dropped or changed in measure or timing, and publish the list.
The briefCredit · unclaimed02
Check what EU results say
For every results document posted in CTIS, the EU trial system, record whether it says how many people took part and whether it gives a result for every registered main outcome, and publish the findings by sponsor.
The briefCredit · unclaimed03
Data-sharing promises checked
Sort every data-sharing statement on ClinicalTrials.gov from trials that say they will share individual participant data (vague, on request, named repository, complex conditions), and where a repository such as Vivli, YODA or CSDR is named, check whether the trial is listed there.
The briefCredit · unclaimed
Open 01 · the brief
Promised outcome versus published
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
For every finished trial whose results paper is linked to its registry record, compare the main outcome registered before the trial started with the one reported in the paper, flag outcomes added, dropped or changed in measure or timing, and publish the list.
Why it is still open
The COMPare team did this by hand for 67 trials in five journals and found 58 (87%) needed a correction. A 2026 automated screen of 46,365 trials by word-matching caught only 1 of 6 real mismatches in a hand-checked set of 123, and the 2025 language-model study compared registry versions only, naming papers as future work. A checker with a measured, high catch rate is still missing.
pubmed.ncbi.nlm.nih.govapi.crossref.orgpeerreviewcongress.org
Built with what exists
the ClinicalTrials.gov API, including the history of each record; PubMed or Europe PMC for links from trial numbers to papers and for open-access texts; COMPare's public coding sheets as a starting answer key; Claude or Gemini to read the papers.
First step
Rebuild COMPare's 67 trials as the first test set and score word-matching and a language model on it.
Done looks like
On at least 300 hand-labelled pairs, including COMPare's 67, the checker catches at least 80% of real mismatches and at least 90% of its flags are right. It then runs on all linked trials, with a public table updated monthly.
Open 02 · the brief
Check what EU results say
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
For every results document posted in CTIS, the EU trial system, record whether it says how many people took part and whether it gives a result for every registered main outcome, and publish the findings by sponsor.
Why it is still open
Bruckner and colleagues checked 234 trials by hand in an April 2026 preprint: fewer than half (49.6%) fully reported results on time, and 47 posted documents lacked main-outcome data, participant numbers or both. The CTIS.eu scoreboard tracks timing (61.6% of 1,728 overdue trials posted on time, on 18 November 2026) but not content, and EMA has not said when it will require results in a structured format.
Built with what exists
the CTIS public portal; Bruckner and colleagues' published data and code; pdftotext; Claude or Gemini, which can read scanned and table-heavy PDFs.
First step
Download the preprint's data and run a first model pass on its 234 trials to see whether it finds the same 47.
Done looks like
Every posted CTIS results document classified on both checks, with at least 95% agreement against two people on 200 documents, in a public table by sponsor that also finds the 47 documents the preprint found.
Open 03 · the brief
Data-sharing promises checked
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
Sort every data-sharing statement on ClinicalTrials.gov from trials that say they will share individual participant data (vague, on request, named repository, complex conditions), and where a repository such as Vivli, YODA or CSDR is named, check whether the trial is listed there.
Why it is still open
A 2024 study sorted only 200 statements, all from Covid-19 trials, by hand; two experts agreed only moderately (kappa 0.62), and the authors wrote that machine reading of such statements was not yet possible. I found no check across the whole registry.
Built with what exists
the ClinicalTrials.gov API, which returned 47,381 trials of a treatment marked as sharing participant data on 18 November 2026, 309 of them naming Vivli; the public study lists of Vivli, YODA and CSDR; Gemini Decision 1 or Claude to sort the statements. The note "A thousand questions, one pass" covers Gemini Decision 1.
First step
Code 100 statements by hand with the 2024 study's scheme, then match the 309 trials that name Vivli against Vivli's public list.
Done looks like
All 47,381 statements sorted, with agreement against two human coders on 300 at least as good as the experts' agreement with each other (kappa 0.62 or better), and a public rate at which trials that name a repository are actually listed there.