BRENNEN

Industries / Industrial equipment and services / Industrial services / Safety training

Measuring whether it works

Checking whether training changed what workers know, what they do at work and how often they are hurt.

7

Milestones achieved, from 1959 to 2026.

1

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1959

    Four levels of evidence

    Donald Kirkpatrick set out, in a series of articles in 1959 and 1960, four levels on which to judge training: reaction (did trainees like it), learning (did they gain knowledge or skill), behavior (do they work differently) and results (did outcomes such as injuries change). A 1989 review called its popularity "widespread and enduring" and questioned its assumption that the levels are causally linked. A 1997 pooled analysis of 34 studies found that trainees' judgment of how useful a course was went with later performance on the job more closely than whether they enjoyed it, and more closely than their test scores.

    LessonTrainees who liked a course are not proof that work became safer.

    api.openalex.orgapi.openalex.org

    Credit

    Donald Kirkpatrick; George Alliger and colleagues for the tests

  2. 2006

    Practice beats lectures

    Michael Burke and colleagues pooled 95 studies of safety training, run from 1971 to 2003 in 15 countries with 20,991 participants. The more a method made trainees act, from lectures and videos to feedback and then to hands-on practice and behavior modeling (watching a model and copying it), the more they learned and the more accidents and injuries fell; the most engaging methods were about three times as effective for knowledge as the least. A 2011 analysis by the same group found the gap widest for severe hazards and small for minor ones.

    pmc.ncbi.nlm.nih.govebi.ac.uk

    Credit

    Michael J. Burke, Sue Ann Sarpy and colleagues (Tulane University and others)

  3. 2007

    Lifting lessons fail

    Training in how to lift is considered a major way to prevent back pain among workers who handle loads, in the words of the review that tested it. A Cochrane review (a systematic review under the Cochrane Collaboration's rules) found in 2007 that such advice and training did no better than no intervention; its 2011 update, with nine randomized trials covering 20,101 employees, found no trial that showed a preventive effect.

    LessonA course that looks sensible can do nothing for the outcome it targets; the review concludes that training, with or without lifting aids, does not prevent back pain compared with no intervention.

    pmc.ncbi.nlm.nih.govebi.ac.uk

    Credit

    Kari-Pekka Martimo, Jos Verbeek and colleagues (Finnish Institute of Occupational Health and others)

  4. 2012

    Behavior yes, injuries unclear

    A team at the Institute for Work & Health in Toronto, with NIOSH researchers, reviewed 22 randomized trials of safety training published from 1996 to 2007. It found strong evidence that training changes what workers do and insufficient evidence that it reduces symptoms, injuries or illnesses. Its advice was to keep training workers but not to expect large health effects from training alone.

    sjweh.fi

    Credit

    Lynda Robson and colleagues, Institute for Work & Health

  5. 2017

    Training records meet claims

    Researchers linked the training records of 17,106 union carpenters in Washington State with their workers' compensation claims from 2000 to 2008. OSHA's Outreach Training, the 10- and 30-hour courses widely taken in US construction, went with 13% fewer claims, a difference too small to rule out chance; the effect was larger for apprentices, for drywall installers and as time since training grew.

    LessonEven nine years of records on 17,000 workers did not show a clear drop, and the authors conclude that one course alone should not be expected to change injury rates much.

    ebi.ac.ukosha.gov

    Credit

    Ashley Schoenfisch, Hester Lipscomb, Clayton Sinyai and Darrin Adams

  6. 2025

    Tests rise, injuries barely

    Tiffany Bisbey, Eduardo Salas and colleagues pooled 666 effects from 157 studies of safety training. In standardized effect sizes, the gains were large for what trainees learned (1.18), moderate for using it at work (0.61) and small for safety indicators such as accidents (0.26) and for health (0.15). A review of reviews written by RAND Europe for the Lloyd's Register Foundation the same year noted that in one meta-analysis only 3 of 90 evaluations of safety training measured accident or injury rates.

    LessonMost studies measure what is easiest to measure, a test after class, and few count injuries.

    api.openalex.orglrfoundation.org.uk

    Credit

    Tiffany M. Bisbey (George Washington University), Eduardo Salas (Rice University) and colleagues; RAND Europe

  7. 2026

    Knowledge fades after class

    Ruggiero Lovreglio and colleagues at Massey University in New Zealand pooled 83 studies, published up to December 2024, that tested safety knowledge days to months after training. Knowledge declined (Hedges' g of −0.673, a standardized effect size); over half the studies used classroom training, most tested with closed-ended questions, and fewer participants stayed in the longer studies. How long after training the test came did not explain the differences between studies.

    api.openalex.org

    Credit

    Ruggiero Lovreglio and colleagues, Massey University

Watch for

  1. Retraining picked by AI

    Vector Solutions says its AI Safety Training Recommendation Engine, launched on 7 January 2026, reads an incident's type, root cause, injury details and description, proposes the most relevant courses from its library for retraining, and records course completion in the incident file, which it says helps prevent repeat events.

    Achieved whenan outside study compares repeat incidents after AI-picked retraining with those after investigators' own choices or after no retraining.

    vectorsolutions.com

    Claimed by

    Vector Solutions, January 2026

Open

  1. 01

    Check AI retraining picks

    A test set of real incident descriptions, each labeled by safety professionals with the fix most likely to prevent a repeat: a change to equipment or the task, a written procedure, or training, and if training, on what. AI tools that recommend training after incidents are then scored on how often they propose retraining where the experts chose a physical fix.

    The brief
    Credit · unclaimed
  2. 02

    Rate courses by how active

    A public audit of safety training courses offered online and in catalogs, coding each by the three levels of engagement used in Burke's 2006 analysis: least (lecture, video, reading), moderate (programmed instruction, feedback) and most (practice, behavior modeling, dialogue), and by whether the course only tests knowledge or also has the trainee do something.

    The brief
    Credit · unclaimed

Open 01 · the brief

Check AI retraining picks

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A test set of real incident descriptions, each labeled by safety professionals with the fix most likely to prevent a repeat: a change to equipment or the task, a written procedure, or training, and if training, on what. AI tools that recommend training after incidents are then scored on how often they propose retraining where the experts chose a physical fix.

Why it is still open

Vector Solutions' engine (2026) proposes courses from incident reports, and no outside test of it was found. In 302 hospital root cause analyses reviewed by Kathryn Kellogg and colleagues (2017), training was the most common proposed fix (20% of 731), the authors class it among weaker actions, and the same kinds of events kept recurring. Since 1 January 2015 US employers under federal OSHA have had to report every amputation, in-patient hospitalization and loss of an eye, and OSHA publishes the reports with their narratives. Sangameshwar Patil and colleagues at TCS Research had three open AI models write preventive recommendations for 275 OSHA incidents (2024); raters with industrial safety experience judged only 57.6% of Mistral 7B's and 49.4% of Phi-3's recommendations specific to the incident, but the study did not ask whether training was proposed where a physical change was needed. A search found no labeled test set for training recommendations.

aclanthology.orgeutils.ncbi.nlm.nih.govosha.govosha.gov

Built with what exists

OSHA's severe injury report data; a model for long documents such as Meta's Muse Ember (see the Note "The jobs agents repeat"); Gemini Decision 1, which returns probabilities for fixed-answer questions, to score each label choice (see the Note "A thousand questions, one pass"); a labeling spreadsheet; two or more experienced safety professionals as raters.

  1. First step

    Pull 100 narratives for one industry from OSHA's data and have two raters label them, to settle the list of fixes.

  2. Done looks like

    500 incident descriptions, each labeled by two raters who agree on the main fix in at least 80% of cases; a published table, for each tool tested, of how often it proposes retraining where the raters chose an equipment or task change.

Open 02 · the brief

Rate courses by how active

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A public audit of safety training courses offered online and in catalogs, coding each by the three levels of engagement used in Burke's 2006 analysis: least (lecture, video, reading), moderate (programmed instruction, feedback) and most (practice, behavior modeling, dialogue), and by whether the course only tests knowledge or also has the trainee do something.

Why it is still open

The two largest pooled analyses agree that active methods teach more (Burke and colleagues, 2006; Bisbey and colleagues, 2025), and Burke's team wrote in 2006 that its findings "challenge the current emphasis on more passive computer-based and distance training methods". Lovreglio's 2026 review found that over half of the studies it pooled used classroom training. Lynda Robson and colleagues at the Institute for Work & Health compared 899 learners who took one standardized Ontario course in person, online with a live instructor or as self-paced e-learning (2025): knowledge scores differed by 2.5 percentage points at most, and engagement was highest in person; the study compared ways of delivering one course, not what the courses on offer ask of trainees. A search found no count of what the courses on offer actually ask trainees to do.

iwh.on.capmc.ncbi.nlm.nih.govapi.openalex.org

Built with what exists

Public course listings and free courses from regulators and training providers in several countries; an AI model to sort course descriptions and screenshots into Burke's levels; two human coders checking a sample; a spreadsheet.

  1. First step

    Code 30 free online courses on fall protection and lockout by hand, using the definitions in Burke's 2006 paper.

  2. Done looks like

    300 courses from at least five countries coded; two human coders agree on at least 80% of a 60-course sample, and the model's codes are scored against theirs; a published share of courses at each level, by hazard.