Industries / Health / Healthcare
Diagnosis
Working out what is wrong with a person from what they report, an examination, and tests on blood, tissue and the heart's electrical signals.
7
Milestones achieved, from 1816 to 2021.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1816
Listening through a tube
R. T. H. Laennec, a physician at the Necker Hospital in Paris, invented the stethoscope in 1816 and matched what he heard in patients' chests with what he later found at autopsy. His book on the method, De l'Auscultation Médiate, followed in 1819.
Credit
R. T. H. Laennec
1901
Recording the heart's current
Willem Einthoven built the string galvanometer, a very sensitive meter that drew the heart's small electric currents as a trace, the electrocardiogram (ECG). The first machines were massive and could not be moved. He received the 1924 Nobel Prize "for his discovery of the mechanism of the electrocardiogram".
Credit
Willem Einthoven
1979
A program matches the experts
MYCIN was a computer program that recommended antibiotics by applying a set of rules. In a blinded test on 10 cases of meningitis, eight outside experts rated its choices acceptable 65% of the time, against 42.5% to 62.5% for five faculty specialists. It never became an established clinical system. A 2023 review gives the reason as the limited number of rules that could be fitted into clinical work.
LessonMatching experts on a test does not get a tool used in care.
Credit
V. L. Yu, B. G. Buchanan and colleagues, who ran the test
2017
Slides read on a screen
The US Food and Drug Administration (FDA) allowed Philips to market a system that scans glass slides of tissue so that pathologists can make a diagnosis on screen instead of at a microscope. The FDA said it was the first time it had permitted this, after a study of about 2,000 cases. Digital slides made large open contests possible: in the PANDA challenge, 1,290 developers trained programs on 10,616 prostate biopsies, and several of the programs reached the level of pathologists on independent sets from the US and Europe. Their agreement with expert pathologists scored 0.862 and 0.868, on a scale where 1 is perfect agreement.
Credit
Philips; W. Bulten, M. Eklund and the PANDA consortium
2018
Fraud in a blood test
The US Securities and Exchange Commission (SEC) charged Theranos, its founder Elizabeth Holmes and its former president Ramesh Balwani with a fraud that raised more than $700 million. The company had said its portable analyzer ran comprehensive tests from a few drops of blood from a finger. By the SEC's account it could run only a small number of tests, and most patient tests were run on other makers' machines.
LessonA lab test needs checking by outsiders before patients rely on it.
Credit
US Securities and Exchange Commission
2021
A sepsis alert tested outside
Sepsis is the body's runaway response to infection. Researchers at Michigan Medicine tested the Epic Sepsis Model, a proprietary alert used in hundreds of US hospitals, on 38,455 hospital stays. It scored 0.63 on a scale where 0.5 is chance and 1 is perfect, missed 67% of sepsis cases, and raised alerts on 18% of all stays.
LessonA vendor's model must be tested in each hospital that uses it.
Credit
A. Wong, K. Singh and colleagues
2021
AI-read ECGs in a trial
An AI model that reads an ordinary ECG can flag a weak heart pump, called low ejection fraction. In a randomized trial with 120 primary care teams and 22,641 adults, teams shown the AI result diagnosed low ejection fraction within 90 days in 2.1% of patients, against 1.6% for teams in usual care.
Credit
X. Yao, P. A. Noseworthy and colleagues
Watch for
AI works up hard cases
Microsoft's MAI-DxO runs several AI models as a virtual panel of doctors that asks questions, orders tests and names a diagnosis. On 304 case records from the New England Journal of Medicine (NEJM), it was correct in 83.2% of cases, against 36.1% for generalist physicians allowed reference books and search, and 22.3% for physicians without them. On 325 emergency department cases from MIMIC-IV, a research database of real hospital records, it scored 80.3% against 46.8% for physicians. All 15 authors work for Microsoft, and NEJM cases lean toward hard, rare problems.
Achieved whena group outside Microsoft repeats the result on cases it chose, or a study in working clinics shows it.
Claimed by
Microsoft AI, June 2025 (preprint) and October 2026 (Science)
An AI takes the history
Google's AMIE, a chat-based AI, interviewed 100 adults by text before their urgent primary care visits at Beth Israel Deaconess Medical Center in Boston, between April and November 2025, with a physician watching each chat. No chat had to be stopped for safety. By the study's count, the final diagnosis was among AMIE's top seven possibilities in 90% of cases and was its first choice in 56%. Blinded reviewers found no significant difference between its diagnosis lists and those of the primary care doctors, and rated the doctors' treatment plans more practical and cheaper. Alphabet, Google's parent company, funded the study.
Achieved whena controlled trial run outside Google shows that patients interviewed by AMIE are diagnosed as accurately and safely as patients who are not.
Claimed by
Google and Beth Israel Deaconess Medical Center, March 2026 (preprint) and October 2026 (The Lancet)
Open
01
An outside rerun of MAI-DxO
At least 100 open diagnostic cases made from openly licensed case reports, written up in the format of Microsoft's Sequential Diagnosis Benchmark (SDBench), in which an AI must ask for each fact and test before it names a diagnosis. Every case comes from an article published after the training cutoff of each model tested. Run the benchmark on at least three models, including one with downloadable weights, and publish accuracy and test costs beside Microsoft's.
The briefCredit · unclaimed02
A lab-result answer key
A public set of at least 500 made-up lab reports, such as blood counts and metabolic panels, laid out the way patient portals show them. Each has an answer key: which values fall outside the reference range, which cross a critical limit that calls for same-day contact with a clinician, and which diagnoses the numbers do not support. Then score popular chatbots on what they tell a patient who pastes the report in.
The briefCredit · unclaimed
Open 01 · the brief
An outside rerun of MAI-DxO
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
At least 100 open diagnostic cases made from openly licensed case reports, written up in the format of Microsoft's Sequential Diagnosis Benchmark (SDBench), in which an AI must ask for each fact and test before it names a diagnosis. Every case comes from an article published after the training cutoff of each model tested. Run the benchmark on at least three models, including one with downloadable weights, and publish accuracy and test costs beside Microsoft's.
Why it is still open
Microsoft's results come from its own staff. Its code is public under the MIT license, but its NEJM case set must be licensed from NEJM, its MIMIC set needs credentialed access, and the only open set is five made-up cases that its makers say should not be used to report performance. I found no outside run published since the code appeared in October 2026.
Built with what exists
the SDBench code; case reports from the PubMed Central Open Access Subset, which can be retrieved through PMC's BioC API or OAI-PMH service, keeping only articles whose license allows reuse; hosted models such as Claude, Gemini or GPT; open-weight models such as Google's MedGemma 27B or Mistral Large 4 (see the Note "A trillion open parameters"). The advance that makes this a small-team job is language models that can play the patient record, revealing only the facts asked for, as SDBench's "gatekeeper" does.
First step
Pull ten 2026 case reports under CC BY licenses from PMC, convert them to SDBench's case format, and run the bundled baseline agent on them.
Done looks like
100 or more cases, each with its source article and license. A clinician checks 20 cases and confirms that the case file holds no facts the article does not. Results for at least three models are published with the cases and code, and an outside person reruns 10 cases and gets the same correct-or-wrong verdict on at least 9.
Open 02 · the brief
A lab-result answer key
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public set of at least 500 made-up lab reports, such as blood counts and metabolic panels, laid out the way patient portals show them. Each has an answer key: which values fall outside the reference range, which cross a critical limit that calls for same-day contact with a clinician, and which diagnoses the numbers do not support. Then score popular chatbots on what they tell a patient who pastes the report in.
Why it is still open
Under the US 21st Century Cures Act Final Rule, patients get results as soon as they are ready, often before a clinician has seen them. In a 2023 survey of 8,139 patients, 95.7% preferred that, and 16.5% of those with abnormal results said it made them more worried. The best-known test of chatbots on lab questions used 53 questions taken from Yahoo! Answers, and medical experts reviewed the answers to only 7. A search found no public test that scores whether chatbots flag critical values to patients.
Built with what exists
Synthea, MITRE's open-source generator of made-up patient records, or a short script that draws values from published reference ranges; LOINC codes to name each test; a hospital laboratory's published critical-value list; chatbots through their public apps or APIs; a scoring script.
First step
Write 50 basic metabolic panels from published reference ranges, 10 of them with a critical potassium or sodium value, and test one chatbot.
Done looks like
500 reports, 100 of them with at least one critical value. Two people with clinical laboratory training agree on the key for at least 95% of items. At least five chatbots are scored on two numbers: the share of critical values flagged with advice to contact a clinician that day, and the share of answers that name a diagnosis the numbers do not support. Data, prompts and scores are published and rerun after each major model release.