Industries / Health / Pharmaceuticals / Clinical research
Trial design
Deciding before a trial starts what the treatment is compared with, who takes part, how they are assigned to groups, what is measured and how many people are needed.
7
Milestones achieved, from 1954 to 2020.
3
To watch for: announced, not yet confirmed outside.
3
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1954
Polio vaccine tested blind
The US field trial of Jonas Salk's polio vaccine gave 623,972 schoolchildren either the vaccine or a dummy injection (a placebo), and in 84 of its test areas the children were assigned at random with no one knowing who got which. The results, announced in 1955, showed the vaccine was 80 to 90% effective against paralytic polio, by the account of a 1998 history in the BMJ.
Credit
Thomas Francis Jr. of the University of Michigan, who directed the independent evaluation; Jonas Salk; the National Foundation for Infantile Paralysis, which paid for the trial
1970
Rules for a fair test
US rules reissued on 8 May 1970 set out what an "adequate and well-controlled" study needs, among them a valid comparison with a control group, a way of assigning patients that limits bias, and a stated method of analysis, and said uncontrolled studies alone could not prove that a drug works. The US Supreme Court upheld the standard in 1973, calling doctors' impressions of a drug, "no matter how fervently held", treacherous.
Credit
US Food and Drug Administration (FDA); US Supreme Court, in Weinberger v. Hynson, Westcott & Dunning
1989
Fixing heartbeats raised deaths
Irregular heartbeats after a heart attack were known to raise the risk of sudden death, and the Cardiac Arrhythmia Suppression Trial (CAST) tested whether drugs that suppress them reduce that risk. Patients on encainide or flecainide died more often than those on placebo, 56 of 730 (7.7%) against 22 of 725 (3.0%), and those parts of the trial were stopped.
LessonA drug can improve a stand-in measure, here the heart rhythm, and still harm patients, so the main measure of a trial should be one that matters to them.
Credit
the CAST Investigators
2007
Start with the smallest dose
In March 2006 six healthy young men in London became critically ill within 16 hours of the first human doses of TGN1412, an antibody that switches on immune cells; all six survived. In 2007 the European Medicines Agency (EMA) told sponsors of higher-risk first-in-human trials to start from the lowest dose expected to have any biological effect in people, and to use the lowest of the starting doses that different methods give.
LessonSafety tests in animals can mislead when a drug's target behaves differently in people.
Credit
EMA's Committee for Medicinal Products for Human Use (CHMP); the clinical report by Ganesh Suntharalingam and colleagues at Northwick Park Hospital
2016
Many drugs, one adaptive trial
I-SPY 2, which began in March 2010, tests several breast-cancer drugs at once and sends more new patients to the drugs doing best in each tumour type, a method called adaptive randomisation. In 2016 it reported its first two successes: neratinib left no cancer at surgery in 56% of one group of patients against 33% of controls, and veliparib with carboplatin did so in 51% of another group against 26%.
LessonIn 2018 the larger BrighTNess trial found that veliparib added nothing to carboplatin (53% against 58%); I-SPY 2's control group had not received carboplatin, so it could not tell which of the two drugs was working.
pubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.govpubmed.ncbi.nlm.nih.gov
Credit
Laura Esserman (University of California, San Francisco), D. A. Berry and the I-SPY 2 investigators; sponsor QuantumLeap Healthcare Collaborative
2019
Naming the exact question
ICH E9(R1), adopted on 20 November 2019, asks every trial to state its "estimand", the exact treatment effect it sets out to measure, including how it will count events such as a patient stopping the drug or dying. ICH is the International Council for Harmonisation, where regulators and drug makers agree shared rules.
LessonAdoption did not change practice at once: of 50 trial protocols published in October 2020, none stated an estimand for its main outcome, by the count of B. C. Kahan and colleagues at the MRC Clinical Trials Unit at UCL.
Credit
the ICH E9(R1) Expert Working Group
2020
Answers within months
The UK's RECOVERY trial assigned hospital patients with Covid-19 at random to one of several treatments or to usual care, under one protocol to which arms could be added and from which they could be dropped, a design called a platform trial. Its hydroxychloroquine arm closed on 5 June 2020 for lack of benefit (27.0% died within 28 days, against 25.0% on usual care), while dexamethasone cut deaths from 25.7% to 22.9%, and from 41.4% to 29.3% among patients on ventilators.
LessonA large, simple randomised trial run inside routine hospital care can settle a question within months.
Credit
P. Horby and M. J. Landray (University of Oxford) and the RECOVERY Collaborative Group
Watch for
One rulebook for adaptive trials
ICH's draft guideline E20, "Adaptive Designs for Clinical Trials", would set shared rules for trials that change course on interim results, for example by dropping an arm or changing the number of people enrolled. It was released for public comment on 25 June 2025, and ICH's work plan of July 2026 expects final sign-off in February 2027.
Achieved whenE20 reaches ICH's final step and at least one member regulator, such as the FDA or the European Commission, lists it as in effect.
Claimed by
the ICH E20 Expert Working Group (rapporteur Gregory Levin of the FDA), June 2025
One pivotal trial by default
Two FDA leaders announced that one adequate and well-controlled trial plus "confirmatory evidence" is the agency's new default standard for approval. By their own figures, a second trial of a drug that does nothing cuts the chance of a false positive from 250 in 10,000 to 6 in 10,000. A revised draft guidance followed on 24 June 2026.
Achieved whenthe guidance is final and an independent count shows how many new drugs were approved on a single pivotal trial before and after.
Claimed by
Vinay Prasad and Martin Makary of the FDA, in the New England Journal of Medicine, February 2026
AI that sizes trials
PowerGPT pairs a language model with a statistics engine to choose a test and work out how many participants a trial needs. In its developers' own randomised comparison, users got the sample size right 94.1% of the time with it against 55.4% with their usual methods, in 4.0 minutes against 9.3.
Achieved whenthe work is peer reviewed and a group not involved in building it reproduces the gain.
Claimed by
Yiwen Lu, Yong Chen and colleagues, in a preprint, September 2025
Open
01
Audit estimands at scale
A public table, with its code, covering every phase 3 trial on ClinicalTrials.gov that started in 2020 or later and has its statistical analysis plan attached: 2,130 trials on 18 November 2026. For each trial's main outcome, record the five parts of the estimand (population, treatment, outcome, how events such as stopping the drug are handled, and the summary measure), and flag the parts that are missing or unclear.
The briefCredit · unclaimed02
Recheck sample-size sums
For at least 500 two-arm randomised trials on ClinicalTrials.gov that have both a protocol attached and results posted, pull out the inputs of the sample-size calculation (the error rates allowed, the expected effect and the expected result in the control group), redo the calculation, and compare the assumed control-group result with the one the trial observed.
The briefCredit · unclaimed03
Track phase 1 dose rules
A monthly count of which rule each newly published phase 1 cancer trial used to raise the dose: the "3+3" rule, which raises the dose for each new group of three patients until severe side effects appear, or a model-based design such as BOIN or CRM, which uses a statistical model of all patients so far.
The briefCredit · unclaimed
Open 01 · the brief
Audit estimands at scale
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public table, with its code, covering every phase 3 trial on ClinicalTrials.gov that started in 2020 or later and has its statistical analysis plan attached: 2,130 trials on 18 November 2026. For each trial's main outcome, record the five parts of the estimand (population, treatment, outcome, how events such as stopping the drug are handled, and the summary measure), and flag the parts that are missing or unclear.
Why it is still open
The checks so far are small and done by hand. Kahan and colleagues read 50 protocols; a 2025 study had GPT-4o review the analysis plans of 15 ClinicalTrials.gov trials against ICH E9 and did not look at estimands. I found no study that extracts estimands across a registry.
Built with what exists
the ClinicalTrials.gov API, filtered with AREA[LargeDocHasSAP]true AND AREA[Phase]PHASE3 AND AREA[StartDate]RANGE[2020-01-01,MAX]; pdftotext or PyMuPDF to pull text from the attached PDFs; Claude or Gemini, which can read a protocol and analysis plan of 200 pages in one pass; Kahan and colleagues' coding scheme.
First step
Pull the 2,130 trial records and their document links, and code 30 by hand to settle the rules.
Done looks like
All 2,130 trials coded. On a random 200 coded separately by two people, the model agrees with them on at least 90% of parts. A public chart shows the share of trials with a fully stated estimand, by start year.
Open 02 · the brief
Recheck sample-size sums
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
For at least 500 two-arm randomised trials on ClinicalTrials.gov that have both a protocol attached and results posted, pull out the inputs of the sample-size calculation (the error rates allowed, the expected effect and the expected result in the control group), redo the calculation, and compare the assumed control-group result with the one the trial observed.
Why it is still open
The best-known check was done by hand. Charles and colleagues read 215 trial reports in 2009; of the 157 they could recalculate, 47 (30%) were more than 10% away from the size the authors reported. New AI tools such as PowerGPT size new trials; I found none that audits the calculations in posted ones.
Built with what exists
the ClinicalTrials.gov API, which listed 30,624 randomised trials with a protocol document on 18 November 2026; the R packages pwr and rpact, or Python's statsmodels; Claude or Gemini to read the inputs out of the PDFs.
First step
Take 50 trials with a yes-or-no main outcome and repeat the 2009 method on them.
Done looks like
500 trials processed, with the extracted inputs checked by hand on 100 and at least 95% matching. A public list of trials whose recalculated size differs by more than 10%, and a chart of assumed against observed control-group results.
Open 03 · the brief
Track phase 1 dose rules
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A monthly count of which rule each newly published phase 1 cancer trial used to raise the dose: the "3+3" rule, which raises the dose for each new group of three patients until severe side effects appear, or a model-based design such as BOIN or CRM, which uses a statistical model of all patients so far.
Why it is still open
The count is redone by hand every few years. A 2024 review of 256 trials from 2020 to 2022 found 3+3 in 73.85% and model-based designs in about 10%. I found no automated or live count.
Built with what exists
the Europe PMC REST API for open-access full texts; the ClinicalTrials.gov API; Gemini Decision 1, which returns a probability for each option of a fixed-choice question, or Claude, to classify each methods section. The note "A thousand questions, one pass" covers Gemini Decision 1.
First step
Rebuild the 2024 review's literature search and label 50 papers by hand.
Done looks like
The classifier agrees with hand labels on at least 90% of 150 papers. A public chart of the share of each design from 2015 to 2026, updated monthly.