BRENNEN

Industries / Health / Pharmaceuticals / Clinical research

Analysis and reporting

Analysing a trial's results as planned, reporting them in registries and journals, and filing them with regulators in set formats so that others can check and reuse them.

7

Milestones achieved, from 1996 to 2018.

2

To watch for: announced, not yet confirmed outside.

3

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1996

    A checklist for trial reports

    CONSORT gave journals a minimum list of what the report of a randomised trial must state, with a diagram of how participants moved through it. It was updated in 2001, 2010 and April 2025, and now has 30 items, including a new section on open science.

    pubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.gov

    Credit

    C. Begg, D. Moher, K. F. Schulz, D. Rennie and colleagues (JAMA 1996); the 2025 update led by S. Hopewell of the Oxford Clinical Trials Research Unit

  2. 1998

    Shared rules for trial statistics

    ICH's guideline E9, final on 5 February 1998, set common principles for the statistics of trials submitted for approval. The UK put it into effect in March 1998 and Japan in November 1998, and China's regulator adopted it in 2020.

    ich.org

    Credit

    the ICH E9 Expert Working Group

  3. 2005

    Register first, or no paper

    The International Committee of Medical Journal Editors (ICMJE) announced in September 2004 that its journals would publish a trial starting on or after 1 July 2005 only if it had been entered in a public registry before the first patient enrolled. Since 1 July 2018 its journals have also required a statement on whether each trial's data will be shared.

    pubmed.ncbi.nlm.nih.govicmje.org

    Credit

    ICMJE (C. De Angelis, J. M. Drazen and colleagues)

  4. 2014

    Regulators' reports change a review

    A Cochrane review of the flu drugs oseltamivir (Tamiflu) and zanamivir (Relenza) was built on 107 clinical study reports, the full reports companies file with regulators, obtained from EMA, GlaxoSmithKline and Roche, not on journal articles. In adults, oseltamivir shortened the time to first relief of symptoms by 16.8 hours, from 7 to 6.3 days, and had no significant effect on hospital admissions.

    LessonJournal articles leave out much of what regulators see, and the makers released the reports only "after a protracted campaign".

    pubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.gov

    Credit

    T. Jefferson, M. A. Jones, P. Doshi, C. B. Del Mar, C. J. Heneghan and colleagues (Cochrane)

  5. 2015

    Reanalysis reverses Study 329

    Outside researchers reanalysed SmithKline Beecham's Study 329, a trial of antidepressants in teenagers, from its full data under the RIAT initiative ("restoring invisible and abandoned trials"). Neither paroxetine nor imipramine beat placebo on any pre-planned outcome, and harms rose, including suicidal thinking and behaviour on paroxetine.

    LessonA published conclusion can be reversed once the full data and protocol are open to others.

    pubmed.ncbi.nlm.nih.gov

    Credit

    J. Le Noury, J. M. Nardo, D. Healy, J. Jureidini and colleagues

  6. 2017

    Electronic dossiers required

    From 5 May 2017, applications to the FDA to sell new drugs, generics and biologics had to be filed in the electronic Common Technical Document (eCTD) format, a fixed structure of folders and files; commercial applications to start trials followed on 5 May 2018.

    fda.govfda.gov

    Credit

    the FDA's drug and biologics centres (CDER and CBER)

  7. 2018

    Missing results counted in public

    EU and US rules require many trials to post results within a year of finishing. Oxford researchers built live public trackers that name each sponsor, and found that only 49.5% of 7,274 due EU trials had reported (2018) and only 40.9% of 4,209 due US trials had reported on time (2020). The FDA sent its first Notice of Noncompliance for missing results on 27 April 2021 and had sent 11 by October 2026.

    LessonThe laws alone did not get results posted, and enforcement has been small: 11 notices against 32,441 US trials the tracker counted as due on 17 November 2026.

    pubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.govfdaaa.trialstracker.netfda.gov

    Credit

    Ben Goldacre, Nicholas J. DeVito, S. Bacon and colleagues (EBM DataLab, University of Oxford); FDA

Watch for

  1. Bayesian analysis for pivotal trials

    The FDA's draft guidance on Bayesian statistics, which combine what was known before a trial with its new data, covers using them for the main analysis of pivotal trials, including how to set success criteria, justify prior assumptions and borrow data from outside the trial.

    Achieved whenthe guidance is final and a public approval document shows a Bayesian main analysis as the basis for approval.

    govinfo.gov

    Claimed by

    FDA, January 2026

  2. AI tracks outcome changes

    GPT-4o, used without extra training, compared each ClinicalTrials.gov record at the start of a trial with its final version, and found that 11,078 of 27,227 trials of FDA-regulated drugs (40.7%) had changed their main outcome, 47.7% of industry-funded trials against 32.9% of others. Against 100 records checked by hand it scored 0.88 for spotting added outcomes, 0.81 for removed ones and 0.67 for modified ones, on a scale where 1 is perfect.

    Achieved whena peer-reviewed paper appears and another group reproduces the accuracy, or the method runs as a public monitor kept up to date.

    peerreviewcongress.org

    Claimed by

    Ian Bulovic, Timothy Miller, Florence Bourgeois and colleagues (Boston Children's Hospital), at the International Congress on Peer Review and Scientific Publication, September 2025

Open

  1. 01

    Promised outcome versus published

    For every finished trial whose results paper is linked to its registry record, compare the main outcome registered before the trial started with the one reported in the paper, flag outcomes added, dropped or changed in measure or timing, and publish the list.

    The brief
    Credit · unclaimed
  2. 02

    Check what EU results say

    For every results document posted in CTIS, the EU trial system, record whether it says how many people took part and whether it gives a result for every registered main outcome, and publish the findings by sponsor.

    The brief
    Credit · unclaimed
  3. 03

    Data-sharing promises checked

    Sort every data-sharing statement on ClinicalTrials.gov from trials that say they will share individual participant data (vague, on request, named repository, complex conditions), and where a repository such as Vivli, YODA or CSDR is named, check whether the trial is listed there.

    The brief
    Credit · unclaimed

Open 01 · the brief

Promised outcome versus published

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

For every finished trial whose results paper is linked to its registry record, compare the main outcome registered before the trial started with the one reported in the paper, flag outcomes added, dropped or changed in measure or timing, and publish the list.

Why it is still open

The COMPare team did this by hand for 67 trials in five journals and found 58 (87%) needed a correction. A 2026 automated screen of 46,365 trials by word-matching caught only 1 of 6 real mismatches in a hand-checked set of 123, and the 2025 language-model study compared registry versions only, naming papers as future work. A checker with a measured, high catch rate is still missing.

pubmed.ncbi.nlm.nih.govapi.crossref.orgpeerreviewcongress.org

Built with what exists

the ClinicalTrials.gov API, including the history of each record; PubMed or Europe PMC for links from trial numbers to papers and for open-access texts; COMPare's public coding sheets as a starting answer key; Claude or Gemini to read the papers.

  1. First step

    Rebuild COMPare's 67 trials as the first test set and score word-matching and a language model on it.

  2. Done looks like

    On at least 300 hand-labelled pairs, including COMPare's 67, the checker catches at least 80% of real mismatches and at least 90% of its flags are right. It then runs on all linked trials, with a public table updated monthly.

Open 02 · the brief

Check what EU results say

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

For every results document posted in CTIS, the EU trial system, record whether it says how many people took part and whether it gives a result for every registered main outcome, and publish the findings by sponsor.

Why it is still open

Bruckner and colleagues checked 234 trials by hand in an April 2026 preprint: fewer than half (49.6%) fully reported results on time, and 47 posted documents lacked main-outcome data, participant numbers or both. The CTIS.eu scoreboard tracks timing (61.6% of 1,728 overdue trials posted on time, on 18 November 2026) but not content, and EMA has not said when it will require results in a structured format.

medrxiv.orgtransparimed.orgctis.eu

Built with what exists

the CTIS public portal; Bruckner and colleagues' published data and code; pdftotext; Claude or Gemini, which can read scanned and table-heavy PDFs.

  1. First step

    Download the preprint's data and run a first model pass on its 234 trials to see whether it finds the same 47.

  2. Done looks like

    Every posted CTIS results document classified on both checks, with at least 95% agreement against two people on 200 documents, in a public table by sponsor that also finds the 47 documents the preprint found.

Open 03 · the brief

Data-sharing promises checked

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

Sort every data-sharing statement on ClinicalTrials.gov from trials that say they will share individual participant data (vague, on request, named repository, complex conditions), and where a repository such as Vivli, YODA or CSDR is named, check whether the trial is listed there.

Why it is still open

A 2024 study sorted only 200 statements, all from Covid-19 trials, by hand; two experts agreed only moderately (kappa 0.62), and the authors wrote that machine reading of such statements was not yet possible. I found no check across the whole registry.

pmc.ncbi.nlm.nih.gov

Built with what exists

the ClinicalTrials.gov API, which returned 47,381 trials of a treatment marked as sharing participant data on 18 November 2026, 309 of them naming Vivli; the public study lists of Vivli, YODA and CSDR; Gemini Decision 1 or Claude to sort the statements. The note "A thousand questions, one pass" covers Gemini Decision 1.

  1. First step

    Code 100 statements by hand with the 2024 study's scheme, then match the 309 trials that name Vivli against Vivli's public list.

  2. Done looks like

    All 47,381 statements sorted, with agreement against two human coders on 300 at least as good as the experts' agreement with each other (kappa 0.62 or better), and a public rate at which trials that name a repository are actually listed there.