Industries / Government, education and community / Nonprofits / Standards bodies
Test methods
This work writes the step-by-step procedures used to measure a material or product, such as its strength, what it is made of or how it burns, and proves that different laboratories following the same procedure get the same answer.
6
Milestones achieved, from 1898 to 2025.
1
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1898
Tests settle buyer-seller fights
The Pennsylvania Railroad opened a test department at Altoona in 1874 and added a chemistry laboratory under Charles B. Dudley in 1875. Dudley's laboratory studied why rails failed so that the railroad could write specifications, written requirements for what its suppliers delivered, which were then a new idea. His work convinced him, in the words of Samuel Etris, a former editor of ASTM's magazine, that the trade needed "a neutral party to arbitrate the divergent views of producer and consumer and furnish a neutral ground for their discussion". In 1898 Dudley and cooperating steel companies formed an American branch of the International Association for Testing Materials. It became a separate body, the American Society for Testing and Materials, in 1902, and published its first standard that year: A1, a specification for carbon-steel rails, still in use as the specification for carbon steel tee rails. The society renamed itself ASTM International in 2001.
Credit
Charles B. Dudley and the Pennsylvania Railroad's test department, with cooperating steel makers
1906
Reference samples for foundries
In 1905 the American Foundrymen's Association asked the US National Bureau of Standards (NBS, now NIST) for help. Cast iron railcar wheels were breaking and causing derailments, and the association had found alloys that would cure the problem, but the chemical laboratories at the foundries supplying the railroads "could not analyze the materials with sufficiently consistent accuracy" to check each batch. NBS, with analyses from member companies and a chemist, John Cain, lent by the association, certified the make-up of four bottles of cast iron chips. They went on sale in 1906 as "Standardizing Iron Sample A" to "D", so that any foundry laboratory could test its own analysis against a known answer. By about 1910 they were part of a series called Standard Samples, which NIST now sells as Standard Reference Materials. NIST says two of the original cast irons have been remade every time stock ran out; Cast Iron C had reached its 14th lot after 95 years.
LessonWritten instructions alone did not make laboratories agree; a shared sample with a certified value did.
Credit
the American Foundrymen's Association; NBS, with John Cain
1959
Round robins drawn as pictures
A round robin, or interlaboratory test, sends portions of the same material to many laboratories and compares their results. W. J. Youden, a statistician at NBS, published a simple way to read one in May 1959. Each laboratory tests two similar samples, A and B, and its pair of results becomes one point on a graph. If only random error were at work the points would fall equally in all four quarters of the graph; in his example, the seven-day tensile strengths reported by 25 laboratories on two cement samples, and in "any existing test procedure" he had seen, they bunched in the upper right and lower left, meaning each laboratory read high on both samples or low on both. He concluded that laboratories "are following rather carefully their own versions of the test procedure". Youden also pointed out the cost: makers build products better than the specification to allow for testing error, and half the tests on a product that just meets the specification will rate it below. He proposed a "ruggedness" test, deliberately varying the conditions that may differ between laboratories, to find a method's weak steps before a round robin. NIST's handbook of statistical methods still teaches the Youden plot.
LessonDisagreement between laboratories is mostly each laboratory's own steady bias, which more repeats inside one laboratory cannot reveal or remove.
Credit
W. J. Youden, NBS
1974
Small fire tests, big claims
Laboratory flame tests on small pieces of foamed plastic let makers describe insulation and other products as "self-extinguishing" or "slow burning". In 1974 the US Federal Trade Commission (FTC) ruled that such descriptions of cellular, or foamed, plastics were misleading, and 25 companies agreed not to use those terms unless they reflected the products' performance under actual fire conditions, according to a 1978 review by the Center for Fire Research at NBS. The review said many flammability test methods had been written for materials that behave quite differently from plastics, that many plastics which pass existing tests had been found "considerably flammable under actual end-use conditions", and that different tests on the same material could give conflicting results.
LessonA pass in a small test describes the test, not a real fire, and claims that blurred the two had to be withdrawn.
Credit
the FTC; James A. Slater of the NBS Center for Fire Research, for the review
2022
Microplastic methods tried together
California's Senate Bill 1422 of 2018 required the State Water Resources Control Board to adopt a standard method for measuring microplastics, plastic particles smaller than 5 mm, in drinking water. To choose one, the board and the Southern California Coastal Water Research Project (SCCWRP) ran a method study with 22 laboratories in six countries. Each received three water samples spiked with known numbers of particles of four plastics in six colours and four size ranges, plus look-alikes such as hair, natural fibres and shell, and a blank. Laboratories counting under a stereo microscope recovered 76% of the particles on average (±10% standard error), and 92% of those larger than 20 µm. Infrared (FTIR) and Raman spectroscopy, which identify a plastic from the way it absorbs or scatters light, named the right plastic for 95% and 91% of the particles checked. The blanks picked up 91 stray particles on average. The board released its two methods in September 2021, revised them in May 2022, and set out the monitoring rules in a policy handbook dated 9 August 2022, which says no government in the world had yet required such monitoring.
Credit
Hannah De Frond, Leah Thornton Hampton, Stephen Weisberg, Charles Wong, Chelsea Rochman and co-authors (University of Toronto and SCCWRP); California's State Water Board
2025
Eighty-four labs, wide spread
VAMAS, the Versailles Project on Advanced Materials and Standards, is a forum that tests measurement methods before they become standards. Under it, a team that included Korinna Altmann of BAM, Germany's federal institute for materials research and testing, sent tablets pressed from known powders of polyethylene (PE) and PET, with particles of roughly 10 to 200 µm, to 84 laboratories around the world. The laboratories used five methods, two that heat the sample and analyse what comes off (thermo-analytical) and three that use light (spectroscopic). Reproducibility, the spread of results between laboratories as a percentage of the average, was 62% to 117% for PE and 45.9% to 62% for PET with the heating methods, and 121% to 129% for PE and 64% to 70% for PET with the light methods. The authors found that dissolving the tablets was "a very challenging step".
LessonEven with techniques already approved by ISO, the spread between laboratories for one plastic was larger than the average result, and the weak step was sample handling, not the instrument.
Credit
Daniel Ciornii, Korinna Altmann and co-authors from the participating laboratories (Analytical Chemistry, April 2025)
Watch for
Plastic counted in human brains
A team that included Matthew Campen of the University of New Mexico reported in Nature Medicine in February 2025 that brain tissue held micro- and nanoplastics at a median of 4,917 µg per gram in 2024 samples, up from 3,345 µg/g in 2016, about 75% of it polyethylene. The main measurement was pyrolysis gas chromatography–mass spectrometry (Py-GC-MS), which heats a sample until it breaks apart and identifies plastics from the fragments. In a matters-arising reply, Fazel Monikh and co-authors, among them BAM's Korinna Altmann, argue that the brain is about 60% fatty acids, that residual fat left by incomplete digestion gives false polyethylene signals, and that this likely inflated the brain results. National Geographic reported in July 2026 that Campen's group still defends the work, while Kevin Thomas of the University of Queensland thinks the study measured residual lipids.
Achieved whentwo independent laboratories using different methods, for example Py-GC-MS with a check for fat interference and infrared or Raman microscopy, analyse split portions of the same tissue, with spiked recoveries and blanks reported, and get amounts that agree.
Claimed by
Nihart, Campen and co-authors, February 2025; the group defended its figures to National Geographic in July 2026
Open
01
Find the weak step first
A tool that reads a written test method and lists the steps where two careful laboratories could do different things and both say they followed it, such as an unstated temperature, filter type, waiting time or way of mixing, ranked by how much each is likely to move the result. These are the factors Youden's ruggedness test was meant to find by experiment. Score the tool against published interlaboratory studies whose authors named the step that made laboratories disagree.
The briefCredit · unclaimed02
Filter photos with known counts
An open set of microscope photos of filters from water samples spiked with a known number of plastic particles, each photo paired with the true count and with counts by several trained people, and a scoring script, so that automatic counting tools and human counters can be compared on the same images, size range by size range.
The briefCredit · unclaimed
Open 01 · the brief
Find the weak step first
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A tool that reads a written test method and lists the steps where two careful laboratories could do different things and both say they followed it, such as an unstated temperature, filter type, waiting time or way of mixing, ranked by how much each is likely to move the result. These are the factors Youden's ruggedness test was meant to find by experiment. Score the tool against published interlaboratory studies whose authors named the step that made laboratories disagree.
Why it is still open
Interlaboratory studies find the weak step only after dozens of laboratories have done the work; in the 84-lab VAMAS comparison it was dissolving the sample tablets. Work on language models and laboratory procedures so far tests whether models understand or write protocols, as in BioProBench (first posted May 2025, revised July 2026), not whether they can predict where laboratories will disagree, and a search found no published test of that.
Built with what exists
Openly published methods and study reports, such as California's microplastics methods (attachments to the State Water Board's handbook), the VAMAS and SCCWRP papers and NBS interlaboratory reports; an AI model that reads long documents, such as Claude (see the Note "Seven more flagged", on a Claude-based reviewer of drug-making records), and an open-weight model such as Mistral Large 4 (see the Note "A trillion open parameters"); Python for scoring.
First step
Collect ten open interlaboratory reports that name a cause of disagreement, and save the method text each study started from.
Done looks like
At least 20 published interlaboratory studies, each with the method text as it stood before the study and the cause of disagreement its authors named, fixed in a public file before the tool is run; the tool's five top-ranked steps include the named cause in at least 60% of studies, set beside the score of two experienced analysts doing the same task without seeing the answers.
Open 02 · the brief
Filter photos with known counts
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
An open set of microscope photos of filters from water samples spiked with a known number of plastic particles, each photo paired with the true count and with counts by several trained people, and a scoring script, so that automatic counting tools and human counters can be compared on the same images, size range by size range.
Why it is still open
Counting is where laboratories lose particles: in California's 22-lab study, counts under a stereo microscope recovered 76% of spiked particles on average. Open microplastic image sets exist, but none found pairs light-microscope photos with a known spiked count. The nearest is RefIMP, from the Technical University of Munich: a reference infrared (FTIR) image of a filter holding more than 1,200 plastic and non-plastic particles, which "serve as a ground truth" for software that analyses such images, but it tests infrared identification, not counting under the stereo microscopes most laboratories use. The MiNa set (2024) labels electron microscope images by plastic type, and a 2024 deep-learning counter for beach debris matched manual counts with 96% accuracy for pieces of 0.5 to 5 mm, far larger than drinking-water particles, and lost accuracy as particles got smaller.
Built with what exists
Polyethylene and polystyrene microspheres of known size, sold by scientific suppliers with a stated concentration, diluted into clean water; filters and a vacuum filtration set; a stereo microscope with a camera; Label Studio for marking particles; an open detection model such as Ultralytics YOLO and a general segmentation model such as Meta's Segment Anything; Zenodo to publish the images.
First step
Spike five filters with a known number of 100 µm polystyrene spheres, photograph them, and have three people count each photo to measure how far they disagree.
Done looks like
At least 300 photos from at least 30 spiked filters across four size ranges from 20 µm to 1 mm, each with its spiked count and counts by at least three trained people, taken on at least two different microscope set-ups; a published table of each person's and each tool's recovery by size range.