Industries / Technology / AR and VR
Work and training
Headsets and glasses are used at work to train staff, to guide assembly and repair step by step, and to check that a job was done right.
6
Milestones achieved, from 1992 to 2026.
1
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1992
A name for overlays
Thomas Caudell and David Mizell of Boeing Computer Services described a headset that would show wiring instructions over "the wiring formboard, where wire bundles are manufactured by laying individual wires across pegs", and wrote: "we refer to the technology as 'augmented reality' (AR)". Their test rig used a Private Eye head-mounted display. A Boeing technical fellow wrote in 2018 that the two "coined the term 'augmented reality'".
Credit
Thomas Caudell and David Mizell, Boeing
2016
Glasses on the harness line
In a Boeing pilot, technicians wiring harnesses followed instructions on Google Glass running Upskill's Skylight software. A Boeing engineer told CIO that it would "decrease the assembly time by 25 percent and it significantly reduced error rates", on "an isolated internal test network", not the live system. Upskill's account says errors fell to zero.
Lessonearly results came from pilots, and the user's and the vendor's figures differed.
Credit
Boeing; Upskill
2018
Headsets in every store
Walmart said it would send "four headsets to every Walmart supercenter and two units to every Neighborhood Market", "more than 17,000 Oculus Go headsets in stores by the end of the year", to train staff with software from STRIVR. Walmart said the training improved test scores by 10 to 15 percent; no outside measurement was found.
Credit
Walmart; STRIVR
2020
Fasteners placed in minutes
Lockheed Martin technicians building NASA's Orion spacecraft for Artemis II used HoloLens 2 headsets to place fasteners called click bonds. Lockheed reported "90% less time", with work that took 8 hours done in 45 minutes, and NASA's Carlos Garcia said the method saved up to three weeks.
Credit
Lockheed Martin, with Scope AR's WorkLink software; NASA
2020
Equal, not better
Kaplan and colleagues pooled studies of training in virtual, augmented and mixed reality (Human Factors, online February 2020, in print 2021). They found that such training "does not express a different outcome than training in a nonsimulated, control environment. It is equally effective", and placed its value where "danger or cost may make traditional training impossible".
Lessonon average a headset course matches ordinary training but does not beat it.
Credit
Kaplan and colleagues
2026
Instructions lose their app
Microsoft confirmed in October 2024 that HoloLens 2 production had ended. Dynamics 365 Guides, its app for writing step-by-step instructions shown in the headset, and its Remote Assist app could be bought or renewed only until 1 November 2025, and they reach end of support on 31 December 2026, after which "The apps will no longer be functional".
Lessoninstructions written in one vendor's app end when that app does.
Credit
Microsoft
Watch for
Instructions written by AI
Augmentir says its generative AI can "transform existing content—such as training videos, existing SOPs" (standard operating procedures) "into native step-by-step, interactive procedures", now with AR features, and reports an "82% reduction in new hire onboarding time" across its customers. The release does not say that steps are pinned to real parts, and the figure is the company's own.
Achieved whenan outside test compares AI-converted instructions with the originals on a real task, counting errors and time.
Claimed by
Augmentir, September 2025
Open
01
Catch the wrong step live
A system that watches an assembly through a headset's cameras and warns the worker the moment a step is done wrong or skipped, tested live on a real product with a published tally of mistakes caught, mistakes missed and false alarms.
The briefCredit · unclaimed02
Instructions from the manual
A tool that reads an existing assembly manual, a PDF with drawings, and turns it into steps shown in a headset, each pinned to the real part it concerns, followed by a test with first-time users against the paper manual.
The briefCredit · unclaimed
Open 01 · the brief
Catch the wrong step live
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A system that watches an assembly through a headset's cameras and warns the worker the moment a step is done wrong or skipped, tested live on a real product with a published tally of mistakes caught, mistakes missed and false alarms.
Why it is still open
Research tests run on recorded video. In HoloAssist (2023), a set of recordings made on HoloLens 2 by Microsoft and others, mistakes were "around 6%" of actions and the best mistake detector scored an F-score, a measure that combines hits and false alarms, of 40.19 out of 100. The IndustReal method (2024) runs at "178 fps", fast enough for live use, but was tested offline and "fails to generalize to (out-of-distribution) execution errors", the kinds it was not trained on. The authors of PREGO (2024) wrote that "no technique can currently detect open-set procedural mistakes online".
Built with what exists
a headset whose cameras apps can read, such as HoloLens 2 with the open hl2ss streaming library; IndustReal's released data and 3D part models; a model that answers fixed questions about each frame, such as whether the red wire is on peg 4, for example Gemini Decision 1 (see the Note "A thousand questions, one pass"); a worktable with a fixed kit, such as IndustReal's toy car.
First step
Run IndustReal's released model on live video of ten assemblies of its toy car and log what it catches.
Done looks like
At least 200 live assemblies by 20 people, with errors planted on a schedule the system does not know, scored against a human observer, and the catch rate, miss rate and false alarms per hour published with video and labels.
Open 02 · the brief
Instructions from the manual
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A tool that reads an existing assembly manual, a PDF with drawings, and turns it into steps shown in a headset, each pinned to the real part it concerns, followed by a test with first-time users against the paper manual.
Why it is still open
Research systems stop short of this. PaperToPlace (UIST 2023) turned a photo of a paper instruction sheet into steps placed in mixed reality on a Meta Quest Pro, but could pin them only to fixed objects, citing a "lack of support for real-time arbitrary object tracking with Quest Pro", and was tested on kitchen tasks. Satori (CHI 2025) guided 16 people through everyday tasks with GPT-4V on HoloLens. Microsoft previewed Copilot for Dynamics 365 Guides on HoloLens 2 in November 2023, and Guides itself ends in December 2026.
Built with what exists
Vuforia Model Targets, which track a real object by its shape from a CAD model or 3D scan; object tracking in Apple's visionOS, which learns reference objects in Create ML; a model that reads long documents with drawings, such as Claude Fable 5.2 (see the Note "Three working devices"); Satori's open code.
First step
Take one public PDF manual for a flat-pack product, have a model write the step list with part names and drawing regions, and check it by hand.
Done looks like
Manuals for three real products converted with every hand correction logged, and 30 first-time users per group, headset or paper, timed and scored for errors on each product and retested a week later, with results and code published.