Industries / Technology / Quantum computing
Benchmarks
Benchmarks are agreed tests that say how good a quantum computer is, from the error rate of a single operation to whether a machine beats the best ordinary computer on a task.
7
Milestones achieved, from 2007 to 2025.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
2007
Randomized benchmarking
Emanuel Knill, David Wineland and colleagues at NIST measured gate errors by running long random sequences of operations and watching how fast results degrade, which separates gate errors from errors in setting up and reading the qubits. On trapped ions they measured an error of 0.00482 per operation. The method became standard: the 2025 Caltech array and IBM's layer-fidelity test both use forms of it.
Credit
Emanuel Knill, D. Leibfried, David Wineland and colleagues
2014
No speedup found
Troels Rønnow, Matthias Troyer, Daniel Lidar, John Martinis and colleagues tested a D-Wave Two quantum annealer, a special-purpose machine for optimisation, with up to 503 qubits against classical methods on random problems and found no evidence of quantum speedup across the full data set.
Lessona speedup has to be defined against the best classical method on the same problems, and the answer can change with the question asked.
Credit
Troels Rønnow, Zhihui Wang, Joshua Job, Sergio Boixo, Sergei Isakov, David Wecker, John Martinis, Daniel Lidar and Matthias Troyer
2018
Quantum volume
Andrew Cross, Jay Gambetta and colleagues at IBM proposed quantum volume, a single number set by the largest random circuit of equal width and depth that a machine runs successfully; the machines they tested then reached 16. Quantinuum has reported records in it and publishes its data.
Lessonin 2023 IBM researchers noted that quantum volume looks only at a machine's best qubits, and proposed layer fidelity, which covers many connected qubits at once.
Credit
Andrew Cross, Lev Bishop, Sarah Sheldon, Paul Nation and Jay Gambetta
2019
Beyond classical, then matched
Google reported that its 53-qubit Sycamore chip had sampled random circuits, a task it said would take the fastest supercomputer thousands of years. In October 2019 IBM's Edwin Pednault and colleagues argued that the Summit supercomputer could do it in a matter of days, and in 2021 Feng Pan, Keyang Chen and Pan Zhang produced one million samples in about 15 hours on 512 graphics processors.
Lessona task chosen because it is hard for classical computers stays hard only until classical methods catch up.
Credit
Google AI Quantum; Edwin Pednault and colleagues, IBM; Feng Pan, Keyang Chen and Pan Zhang
2021
Application benchmarks
Thomas Lubinski and colleagues released an open-source suite, later known as the QED-C benchmarks, that runs small versions of well-known algorithms at growing sizes and reports both result quality and time to solution. Metriq's open runner includes the suite.
Credit
Thomas Lubinski, Sonika Johri, Paul Varosy, Timothy Proctor and colleagues
2024
A government checks the plans
DARPA, the US defence research agency, launched its Quantum Benchmarking Initiative in mid-2024 to test whether any approach can build an industrially useful quantum computer by 2033, one whose computing value exceeds its cost. Its independent team verifies each company's design in three stages, with awards of up to USD 1 million, 15 million and 300 million, and it had evaluated more than 20 organisations by October 2026.
Credit
DARPA
2025
A public tracker for advantage
In November 2025 the Quantum Advantage Tracker opened, with a materials experiment by Algorithmiq and IBM among its first entries; Algorithmiq calls it the first public tracker of quantum advantage. It publishes candidate results with their circuits and sorts them into active candidates, superseded candidates and classical baselines as challenges come in.
Credit
IBM, Algorithmiq and collaborators
Watch for
Four designs reach Stage C
DARPA moved Atom Computing (neutral atoms), Diraq (silicon spins), IBM (superconducting circuits) and IonQ (trapped ions) to the final stage of its benchmarking programme, where its team checks that each proposed machine can be built and run as designed. They join Microsoft and PsiQuantum, which reached the equivalent stage through an earlier DARPA programme. DARPA estimates it is more likely than not that at least one company will build an industrially useful quantum computer by 2033.
Achieved whenDARPA's verification team reports that a design can be built and operated as intended.
Claimed by
Atom Computing, Diraq, IBM and IonQ for their designs; advanced by DARPA, October 2026
Quantum volume of 33 million
Quantinuum reported a quantum volume of 2^25, or 33,554,432, on its H2 trapped-ion computer, measured by its own team.
Achieved whenan outside party recomputes the pass from Quantinuum's released data, or reruns the test on H2 and gets the same result.
Claimed by
Quantinuum, September 2025
Open
01
Beat a tracker candidate
Take one active candidate on the Quantum Advantage Tracker and produce a classical calculation that matches its results within its error bars, or show where across its range a classical method fails. Submit the result to the tracker.
The briefCredit · unclaimed02
Records rechecked from data
Recompute published benchmark records from the raw data their makers release, starting with Quantinuum's quantum volume files, and publish for each record whether it passes its own stated test, with the code. Where the machine can be rented, rerun a smaller version of the test and compare.
The briefCredit · unclaimed
Open 01 · the brief
Beat a tracker candidate
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
Take one active candidate on the Quantum Advantage Tracker and produce a classical calculation that matches its results within its error bars, or show where across its range a classical method fails. Submit the result to the tracker.
Why it is still open
Active candidates are by definition unbeaten. Algorithmiq and IBM say no classical method has reproduced their materials result across its full range, and Algorithmiq has released its own classical method so that others can try. Classical teams answered IBM's 2023 claim within two weeks, so the open question is which candidates can be answered now.
Built with what exists
Algorithmiq's monoprop; open tensor-network libraries such as quimb and ITensor; NVIDIA's cuQuantum on rented graphics processors; the tracker's published circuit instances.
First step
Download the smallest circuit instance from the tracker's observable-estimation track and run monoprop on it.
Done looks like
The tracker moves the candidate to superseded, or accepts the submission as a classical baseline with error bars across the stated range.
Open 02 · the brief
Records rechecked from data
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
Recompute published benchmark records from the raw data their makers release, starting with Quantinuum's quantum volume files, and publish for each record whether it passes its own stated test, with the code. Where the machine can be rented, rerun a smaller version of the test and compare.
Why it is still open
Quantinuum publishes its quantum volume data in a public repository, and its own staff published the statistical method for judging a pass. A November 2026 search found no outside group that had recomputed the records from those files.
Built with what exists
Python with the released data; Quantinuum's published analysis method; Qiskit to regenerate the random circuits and compute ideal outputs; cloud access to Quantinuum or IBM machines for the smaller reruns.
First step
Download the most recent quantum volume dataset in the repository and recompute its share of "heavy" outputs, the most likely results, that the test counts.
Done looks like
Every record in the repository recomputed, with the pass or fail and confidence level for each, from code that runs in one command; any disagreement sent to Quantinuum before publication; one smaller rerun on a rented machine.