Industries / Health / Healthcare / Medical labs
Quality control
Checking that a lab's results are right: running control samples of known value alongside patients' samples, comparing results with other labs, and being inspected against the rules.
6
Milestones achieved, from 1947 to 2025.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1947
The same sample everywhere
Proficiency testing means sending the same unknown sample to many labs and comparing their answers. A CDC conference paper credits William P. Belk and F. William Sunderman with introducing it in 1947. When the College of American Pathologists (CAP) ran its first national survey in 1949, about 500 labs sent back results that varied so much that CAP's board delayed publishing them; CAP began offering standard solutions in 1951.
LessonLabs could not see how far apart their results were until an outside body sent them all the same sample.
Credit
William P. Belk and F. William Sunderman; College of American Pathologists
1950
Control charts reach the lab
Stanley Levey and E. R. Jennings brought control charts into the clinical lab. Each day's result on a control sample is plotted, and limits are set at the mean plus or minus 2 standard deviations (SD), a measure of normal scatter. The Levey-Jennings chart is still the basic tool of daily lab quality control.
Credit
Stanley Levey and E. R. Jennings (American Journal of Clinical Pathology, 1950)
1981
Rules that cut false alarms
A single 2 SD limit rejects many good runs: by Westgard's figures, about 9% when two controls are run. James Westgard and colleagues proposed reading several rules together. A result beyond 2 SD only prompts a closer look, and the run is rejected if, for example, one result falls beyond 3 SD or two in a row fall beyond 2 SD on the same side. Known as the Westgard rules, they keep false rejections low while still catching errors.
Credit
James O. Westgard, P. L. Barry, M. R. Hunt and T. Groth (Clinical Chemistry, 1981)
1988
Faulty tests prompt a law
Walt Bogdanich of The Wall Street Journal won the 1988 Pulitzer Prize for Specialized Reporting "for his chilling series of reports on faulty testing by American medical laboratories". The US Government Accountability Office (GAO) later wrote that renewed concern about quality, including errors in Pap smear tests, led Congress to pass the Clinical Laboratory Improvement Amendments of 1988 (CLIA). CLIA sets quality standards for almost every US site that tests human samples for health care; the GAO counted about 193,000 such labs in 2005.
LessonRules take years to reach the bench; by the GAO's account, proficiency testing for people who read Pap smears began only in 2005, and failures were not penalized in its first two years.
Credit
Walt Bogdanich, The Wall Street Journal; US Congress
2016
A lab loses its license
After inspecting Theranos's lab in Newark, California, in late 2015, the Centers for Medicare & Medicaid Services (CMS), which enforces CLIA, wrote in January 2016 that its practices posed "immediate jeopardy to patient health and safety", citing hematology (blood counts), the lab director and other problems. In July 2016 Theranos announced the sanctions: its CLIA certificate revoked, founder Elizabeth Holmes barred from owning or running a lab for at least two years, a fine, and loss of Medicare and Medicaid payments. Walgreens had already ended its partnership, citing voided test results.
LessonAn inspection of routine lab work, not a company's account of its own device, decided whether the lab could keep testing patients.
Credit
Centers for Medicare & Medicaid Services
2025
Court strikes the test rule
A laboratory-developed test (LDT) is one a lab designs, makes and uses itself. In May 2024 the FDA issued a rule to end its long-standing hands-off approach and treat LDTs like commercial test kits. On 31 March 2025 the US District Court for the Eastern District of Texas vacated the rule in a case brought by the American Clinical Laboratory Association, and in September 2025 the FDA removed it from its regulations.
Credit
US District Court for the Eastern District of Texas; US FDA
Watch for
Neural networks catch drift sooner
Patient-based quality control watches running averages of patients' own results for a shift that signals an instrument error. Researchers at Peking University First Hospital report that their neural-network version, tested on six tumor markers with errors added by simulation, cut the number of patient results affected before an error was caught by an average of 62% against the standard method, at the same false-alarm rate of 0.1%.
Achieved whenanother lab reproduces the gain on its own or shared public data, or a lab reports real errors caught sooner in routine use.
Claimed by
B. Su, Y. Zhang, X. Shi and colleagues, Peking University First Hospital, May 2026
AI checks scanned slides
Leica Biosystems announced FDA clearance of Aperio iQC DX, software that looks for six common faults in slide scans while the slides are still in the scanner: air bubbles, pen marks, clipped tissue, missing tissue, blur and striping. Leica reports that in a study with Heidelberg University's Institute of Pathology its models detected up to 24% more artifacts than histotechnicians and cut hands-on review time by 69%, and calls it the industry's first standalone AI quality control software. An open-source tool for the same job, GrandQC, was published in December 2024.
Achieved whena lab other than Leica's study partner publishes catch rates and rescan counts from routine use.
Claimed by
Leica Biosystems, August 2026
Open
01
QC chart test for AI
A public set of at least 1,000 made-up control runs, each given both as a Levey-Jennings chart image and as a table of numbers, with an answer key from a rules engine: accept or reject, which Westgard rule was broken, and whether the pattern points to random or systematic error. Then score AI models on reading them.
The briefCredit · unclaimed02
A shared patient-QC test
A public benchmark for patient-based quality control: result streams for at least 10 common tests built from a published hospital dataset, a fixed library of injected errors (constant and proportional shifts of set sizes, starting at set points), and a scoring script that reports the false-alarm rate and the number of patient results affected before detection. Run the standard methods (moving average, moving median, exponentially weighted moving average) and at least one neural-network method on it.
The briefCredit · unclaimed
Open 01 · the brief
QC chart test for AI
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public set of at least 1,000 made-up control runs, each given both as a Levey-Jennings chart image and as a table of numbers, with an answer key from a rules engine: accept or reject, which Westgard rule was broken, and whether the pattern points to random or systematic error. Then score AI models on reading them.
Why it is still open
Sten Westgard wrote in February 2026 that the AI-generated Levey-Jennings charts he sees are "often wildly wrong", and asked whether AI "can actually interpret a real LJ chart with QC data". A search found no published test of how well AI models read QC charts.
Built with what exists
Python with NumPy and Matplotlib to make the charts; the published Westgard rule definitions; multimodal models such as Claude, Gemini and GPT for the images; Mistral Large 4, whose weights can be downloaded, for the tables (see the Note "A trillion open parameters"). The advance is AI models that read charts and tables directly. No samples and no lab equipment are involved.
First step
Generate 50 charts with 10 known violations and test one model on both the images and the tables.
Done looks like
1,000 runs, 30% with a violation, spread across the 1-3s, 2-2s, R-4s, 4-1s and 10-x rules. Two medical laboratory scientists confirm the key on a random 100. At least five models scored on accept-or-reject accuracy, false rejections and correct rule named, for images and tables separately, with data and prompts published.
Open 02 · the brief
A shared patient-QC test
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public benchmark for patient-based quality control: result streams for at least 10 common tests built from a published hospital dataset, a fixed library of injected errors (constant and proportional shifts of set sizes, starting at set points), and a scoring script that reports the false-alarm rate and the number of patient results affected before detection. Run the standard methods (moving average, moving median, exponentially weighted moving average) and at least one neural-network method on it.
Why it is still open
Each new method is tested on one hospital's private results. A 2026 comparison from Peking Union Medical College Hospital used only its own data and states that "standardized evaluation and optimization strategies remain limited"; a 2025 neural-network method was built on one group's data from 611,031 patients, and its authors say multi-center studies are still to come. Open simulation tools exist, but no shared data set and error library that lets methods be ranked on the same ground.
Built with what exists
MIMIC-IV, a de-identified record of hospital stays at Beth Israel Deaconess Medical Center in Boston that includes lab results, available to credentialed researchers through PhysioNet. It shifts each patient's dates separately, so streams must be rebuilt in a plausible order rather than the analyzer's true order; the open PBRTQC simulation app and R pipeline on GitHub; Python. Published data only; no new samples.
First step
Get PhysioNet credentials, extract sodium and potassium results, and run a moving-average method with 10 injected shifts.
Done looks like
Streams and error library published with a DOI; at least five methods scored on every test and error; one outside lab reruns the scoring script on its own data and reports whether the ranking of methods holds.