BRENNEN

Industries / Health / Healthcare / Imaging centers

AI reading aids

AI reading aids are programs that look at scans alongside the radiologist to flag urgent cases, mark suspected findings, set aside normal studies or draft the text of a report.

7

Milestones achieved, from 2016 to 2023.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 2016

    Told to stop training radiologists

    In 2016 the computer scientist Geoffrey Hinton said that "people should stop training radiologists now", predicting that deep learning, a kind of AI that learns from examples, would take over reading scans. The work grew instead. Works in Progress counts a record 1,208 US radiology residency places offered in 2025 and vacancy rates at all-time highs, and cites a 2012 study in which staff radiologists at three hospitals spent 36% of their time interpreting images.

    LessonReading images is only part of a radiologist's job, so a program that does one reading task well did not remove the job.

    joshuagans.substack.comworksinprogress.co

    Credit

    Geoffrey Hinton, for the prediction; Joshua Gans, an economist who heard him say it, for the account

  2. 2018

    Models that learned the hospital

    Researchers trained programs to spot pneumonia on 158,323 chest X-rays from three US hospital systems. A model trained on two of them scored an area under the curve (AUC, a measure from 0.5 to 1 of how well a test separates sick from well) of 0.931 on new images from those hospitals, but 0.815 at the third. The programs could tell which hospital system an X-ray came from in more than 99.9% of cases, and sorting by hospital alone scored 0.861, because one hospital had far more pneumonia.

    LessonA model can score well by picking up clues about where an image was taken, so it has to be tested at hospitals it never saw.

    europepmc.org

    Credit

    J. R. Zech, E. K. Oermann and colleagues, in PLOS Medicine

  3. 2018

    Software alerts the stroke team

    On 13 February 2018 the US Food and Drug Administration (FDA) granted Viz.ai's ContaCT a De Novo, the route for a new kind of device, and with it created the device type "radiological computer-assisted triage and notification software". ContaCT reads CT angiograms, CT scans of blood vessels filled with dye, for a suspected large vessel occlusion, a blocked main artery in the brain, and alerts a stroke specialist while radiologists still read every scan as usual. In the 300-scan study Viz.ai submitted, it found 87.8% of occlusions and correctly cleared 89.6% of scans without one. In 44 cases with timings, its alert came a median 5.6 minutes after the scan, against 51.5 minutes for the usual route, a radiologist's call after reading. By June 2026 the FDA's list of AI-enabled devices held 86 devices under that triage product code.

    accessdata.fda.govaccessdata.fda.govfda.gov

    Credit

    Viz.ai; FDA

  4. 2021

    Software may replace TB readers

    On 22 March 2021 the World Health Organization (WHO) published 17 new and updated recommendations on screening for tuberculosis (TB). One says that computer-aided detection (CAD) software "may be used in place of human readers for interpreting digital chest X-rays for screening and triage for TB disease" in adolescents and adults. It is a conditional recommendation. A 2025 review says such software gets around a shortage of radiologists and the differences between readers. An outside test that year, on chest X-rays of 23,954 people at three screening centers in Dhaka, Bangladesh, found that all five commercial programs tested beat the three radiologists, and that all five did worse in people over 60 and in people who had had TB before.

    who.inteuropepmc.orgeuropepmc.org

    Credit

    WHO; Z. Z. Qin, J. Creswell and colleagues, in The Lancet Digital Health

  5. 2022

    Normal X-rays reported by software

    In March 2022 Oxipit, a company in Vilnius, Lithuania, received CE certification at class IIb for ChestLink, which writes the report itself for chest X-rays it is highly confident are normal, with no radiologist involved. Class IIb is the European class for devices whose errors could seriously harm a patient's health. The consultancy Hardian Health confirms that Oxipit was the first radiology AI certified at that class for this use, though at least three other chest X-ray AIs held class IIb without claiming autonomy. Outside tests followed. At four Copenhagen-area hospitals, ChestLink caught 99.1% of 1,100 abnormal X-rays and would have reported 28% of the normal ones alone, 7.8% of all X-rays. At University Hospital Birmingham, in 82,761 reports, it reported 9,821 of 37,048 normal X-rays (26.5%); clinicians had found something on 426 X-rays it called normal, and 15 of those misses were judged clinically significant.

    oxipit.aihardianhealth.comeuropepmc.orgauntminnie.com

    Credit

    Oxipit; L. L. Plesner, M. B. Andersen and colleagues, Herlev and Gentofte Hospital (2023); S. Vadera and colleagues, Birmingham (2024)

  6. 2023

    Four products, more false alarms

    The same Danish group tested four commercial chest X-ray programs on 2,040 consecutive patients for three findings: airspace disease (such as pneumonia), pneumothorax (a collapsed lung) and pleural effusion (fluid around the lung). The programs found 72% to 91% of airspace disease, 63% to 90% of pneumothoraxes and 62% to 95% of effusions, and raised more false alarms than the radiologists' reports. For airspace disease, the share of X-rays without it correctly cleared fell from 85%–96% on X-rays with no other finding or one to 27%–69% on X-rays with four or more.

    LessonThe programs did worst on X-rays showing several problems at once and on small findings, which hospitals see every day.

    europepmc.org

    Credit

    L. Lind Plesner, M. Brun Andersen and colleagues, in Radiology

  7. 2023

    Readers follow wrong advice

    At the University of Cologne, 27 radiologists rated 50 mammograms on BI-RADS, the 0-to-6 scale for how suspicious a breast image looks, with suggestions from a program presented to them as AI. On 12 of the 40 test images the suggestion was deliberately wrong. Inexperienced readers rated 79.7% of images correctly when the suggestion was right and 19.8% when it was wrong; very experienced readers, 82.3% and 45.5%.

    LessonA reading aid's mistakes pass to the person using it, even an expert, so a test has to measure the reader with the tool and not the tool alone.

    europepmc.org

    Credit

    T. Dratsch, D. Pinto dos Santos and colleagues, in Radiology

Watch for

  1. Drafts that save reading time

    Northwestern Medicine in Chicago built a generative model, AI that writes text, that drafts the report for each plain X-ray before the radiologist opens it. In a prospective study from November 2023 to April 2024, reports written with a draft took 159.8 seconds on average against 189.2 without, 15.5% faster, across 23,960 X-rays. Peer review of 800 found no difference in accuracy, and the model flagged unexpected pneumothoraxes needing treatment with 72.7% sensitivity among 97,651 studies. The health system built the model and ran the study itself.

    Achieved whenanother health system, or a group with no stake in the model, repeats the timing and accuracy results.

    europepmc.org

    Claimed by

    J. Huang, M. Etemadi and colleagues, Northwestern University, June 2025

  2. Autonomous normals at scale

    Oxipit says ChestLink identifies normal chest X-rays with 99.9% sensitivity, and on 5 March 2026 it agreed to be bought by Sectra, a Swedish maker of imaging software, which its chief executive said gives "a powerful platform to responsibly scale autonomous AI". The outside tests published so far looked back at past X-rays; the Danish group's 2024 study of a commercial tool used off-label said its results should be confirmed in a prospective study.

    Achieved whena group not tied to the company publishes a prospective study of ChestLink reporting on its own in daily use, with its miss rate.

    oxipit.aieuropepmc.org

    Claimed by

    Oxipit and Sectra, March 2026

Open

  1. 01

    A kit for testing over-reliance

    An open reader-study kit that measures automation bias, the habit of following a machine's advice when it is wrong. It holds a fixed set of public, de-identified images with confirmed answers, AI suggestions that are deliberately wrong on a set share of cases, a web reading form that records each answer and the time taken, and an analysis script. A department, training scheme or software maker could run its readers through it, compare them with published figures, and test ways to cut over-reliance, such as boxing the area of concern or showing the AI only after the reader's first answer. The images are public and no answer reaches a patient.

    The brief
    Credit · unclaimed
  2. 02

    A public normal X-ray test

    A repeatable public test of how many normal chest X-rays a program can set aside without missing disease. Run each available program on public test sets labeled by radiologists, find the setting at which it catches 99% and then 99.9% of abnormal X-rays, and report what share of normal X-rays it would then clear. Two radiologists grade every abnormal X-ray it would have cleared as critical, clinically significant or insignificant, the scale the Danish group used. It is a test of software on past, de-identified images, not a reading of any patient.

    The brief
    Credit · unclaimed

Open 01 · the brief

A kit for testing over-reliance

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

An open reader-study kit that measures automation bias, the habit of following a machine's advice when it is wrong. It holds a fixed set of public, de-identified images with confirmed answers, AI suggestions that are deliberately wrong on a set share of cases, a web reading form that records each answer and the time taken, and an analysis script. A department, training scheme or software maker could run its readers through it, compare them with published figures, and test ways to cut over-reliance, such as boxing the area of concern or showing the AI only after the reader's first answer. The images are public and no answer reaches a patient.

Why it is still open

At least four studies have measured the effect, each with its own sham AI and its own cases: 27 radiologists on 50 mammograms in Cologne (2023); 6 radiologists on 90 chest X-rays, where a box around the suspect area reduced the harm (Bernstein and colleagues, 2023); 220 physicians reading 8 chest X-rays each with a simulated tool, ChestAId, whose accuracy fell to 23.6% when its advice was wrong (Prinster, Mahmood, Yi and colleagues, 2024); and 10 readers given GPT-5 and GPT-4o suggestions on 100 Korean teaching cases (Song and colleagues, 2026). None of the accounts read here describes a shared kit that another department could rerun to compare its own readers. The reading software from one mammography experiment of this kind is on GitHub, but it runs only on Windows and comes without its images, its case list or its schedule of wrong suggestions.

github.comeuropepmc.orgeuropepmc.orgrsna.orgeuropepmc.org

Built with what exists

CBIS-DDSM, 2,620 scanned film mammography studies with verified pathology, free under a CC BY 3.0 license from The Cancer Imaging Archive; the OHIF Viewer, an open-source, MIT-licensed web image viewer, for the reading screen; iMRMC, the FDA's free software for analyzing studies in which many readers read many cases, or Python or R mixed-effects models, as in the published studies. Readers take part as research volunteers, so the study needs an ethics committee's approval or exemption.

cancerimagingarchive.netraw.githubusercontent.comgithub.com

  1. First step

    Pick 60 CBIS-DDSM cases with verified pathology, write the rule for which suggestions are wrong, and try the reading form with three readers.

  2. Done looks like

    The kit released openly with at least 60 cases, about one in five with a wrong suggestion. Results from at least 30 readers at three experience levels from at least three sites, giving accuracy with right and wrong suggestions for each level and for at least one way of cutting over-reliance. A second team reruns it and gets results within the first team's confidence intervals.

Open 02 · the brief

A public normal X-ray test

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A repeatable public test of how many normal chest X-rays a program can set aside without missing disease. Run each available program on public test sets labeled by radiologists, find the setting at which it catches 99% and then 99.9% of abnormal X-rays, and report what share of normal X-rays it would then clear. Two radiologists grade every abnormal X-ray it would have cleared as critical, clinically significant or insignificant, the scale the Danish group used. It is a test of software on past, de-identified images, not a reading of any patient.

Why it is still open

Every published test of autonomous normal reporting read here used one health system's own X-rays, which outsiders cannot get: four Danish hospitals (2023 and 2024), two NHS trusts (Smith and colleagues, 2023) and Birmingham (2024). A 2025 preprint, not peer-reviewed, trained a commercial triage tool on public datasets but tested it on 1,045 X-rays from one Brazilian hospital. So no two products have been compared on the same X-rays at the same sensitivity. A 2024 Stanford review of 98 FDA-cleared detection and triage devices found only 13 described in peer-reviewed papers and two tested prospectively.

europepmc.orgarxiv.orgpreprints.orgmedrxiv.org

Built with what exists

the VinDr-CXR test set, 3,000 chest X-rays from two hospitals in Hanoi, 2,052 of them normal, each labeled by a consensus of five radiologists, free on PhysioNet after credentialing; PadChest, more than 160,000 images from 67,000 patients at Hospital San Juan, Spain, as a second set whose labels come from the reports; open models such as TorchXRayVision's pretrained chest X-ray networks, installed with pip, and MedGemma, whose weights can be downloaded; commercial products whose makers agree to run the set. VinDr-CXR's license is PhysioNet's credentialed health data license, so run the models on your own machines.

physionet.orgbimcv.cipf.esraw.githubusercontent.com

  1. First step

    Apply for PhysioNet credentialed access, download the VinDr-CXR test set, and run TorchXRayVision's "all" model to find its 99% sensitivity setting.

  2. Done looks like

    For at least four programs on at least two public sets, the share of normal X-rays cleared at 99% and 99.9% sensitivity, with 95% confidence intervals, and every miss graded by two radiologists. Code, settings and grades are published so a maker can add its own product and be compared on the same X-rays.