BRENNEN

Industries / Technology / Software and SaaS / AI and machine learning

Training

Training turns data and computing power into a working model: choosing the model's design, running it through the data until its errors shrink, and then teaching it to follow instructions.

7

Milestones achieved, from 1986 to 2025.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1986

    Learning from errors

    David Rumelhart, Geoffrey Hinton and Ronald Williams described back-propagation in Nature: a procedure that repeatedly adjusts the strengths of the connections in a network so that its output moves closer to the desired output. Hidden units, which are neither input nor output, came to represent useful features of the task on their own, which the paper says set the method apart from earlier, simpler ones such as the perceptron's.

    nature.com

    Credit

    David Rumelhart, Geoffrey Hinton and Ronald Williams

  2. 2017

    Attention replaces recurrence

    The Transformer, a model design from Google researchers, dropped recurrence (reading a sentence one word at a time) and relied only on attention, a way for each word to weigh every other word at once. Because the work could be spread across many chips in parallel, it trained quickly: the English-to-French model set a new single-model record after 3.5 days on eight GPUs, by the authors' count.

    arxiv.org

    Credit

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser and Illia Polosukhin, Google Brain, Google Research and the University of Toronto

  3. 2021

    Small add-ons for fine-tuning

    Fine-tuning is further training of a finished model for one job. Edward Hu and colleagues at Microsoft proposed LoRA (low-rank adaptation): keep the model frozen and train two small added tables of numbers instead. Against full fine-tuning of the 175-billion-parameter GPT-3, they report 10,000 times fewer trained parameters, a third of the GPU memory, and no added delay when the model answers. It is now a standard method in Hugging Face's open PEFT library.

    arxiv.orghuggingface.co

    Credit

    Edward Hu, Yelong Shen and colleagues, Microsoft

  4. 2022

    More data, smaller model

    In 2020 Jared Kaplan and colleagues at OpenAI found that a model's error falls as a power law with its size, its data and its compute, and advised spending a fixed budget mainly on very large models trained on relatively little data. In March 2022 Jordan Hoffmann and colleagues at DeepMind trained more than 400 models and found that size and training data should grow in equal proportion. Their 70-billion-parameter Chinchilla used the same compute as their 280-billion-parameter Gopher, with four times the data, and beat it and larger models such as GPT-3. In 2024 Tamay Besiroglu and colleagues at Epoch AI re-fitted one of Chinchilla's three methods, found it inconsistent, and got a result that agrees with the other two.

    LessonThe first scaling rule left the largest models of the time "significantly undertrained", in DeepMind's words, so a rule fitted to experiments needs checking by more than one method.

    arxiv.orgarxiv.orgarxiv.org

    Credit

    Jordan Hoffmann and colleagues, DeepMind; Jared Kaplan and colleagues, OpenAI; Epoch AI for the replication

  5. 2022

    Taught by human rankings

    Long Ouyang and colleagues at OpenAI fine-tuned GPT-3 first on example answers written by labelers, then on labelers' rankings of its answers, using reinforcement learning from human feedback (RLHF), in which the model is rewarded for answers people rank higher. In their tests, people preferred the answers of the resulting 1.3-billion-parameter InstructGPT to those of the 175-billion-parameter GPT-3, a model 100 times larger.

    arxiv.org

    Credit

    Long Ouyang, Jeff Wu, Xu Jiang and colleagues, OpenAI

  6. 2022

    A logbook of failures

    Meta AI trained OPT-175B, a 175-billion-parameter model, on 992 A100 GPUs and published the logbook of the run. Over two months, hardware failures caused at least 35 manual restarts and the swapping out of more than 100 machines, plus an estimated 70 or more automatic restarts. When the loss (the model's training error) suddenly diverged, the team lowered the learning rate and restarted from an earlier saved copy of the model.

    LessonThe team wrote that cost estimates assuming "no hardware failures or training instabilities" understate what a model costs to make.

    arxiv.org

    Credit

    Susan Zhang, Stephen Roller, Naman Goyal and colleagues, Meta AI

  7. 2025

    Reasoning learned by trial

    DeepSeek trained R1-Zero by reinforcement learning on problems a program can mark right or wrong, such as math and coding, without human-written worked solutions, and the model learned habits such as checking its own work. Nature published the work after peer review on 17 September 2025. Its supplement puts the reasoning stage at 147,000 H800 GPU-hours, or $294,000 at an assumed rental price of $2 an hour, on top of the base model it started from.

    LessonA cost figure for one stage is not the cost of a model, yet such figures move markets: when R1 came out on 27 January 2025, Nvidia lost close to $600 billion in market value in one day, CNBC reported, the largest one-day drop for a US company.

    nature.comstatic-content.springer.comcnbc.com

    Credit

    DeepSeek-AI

Watch for

  1. Training for repeatable answers

    Microsoft applied scientists say 50 steps of reinforcement learning on 24 H100 GPUs took the open Qwen3.8-27B model from passing 38 of 507 agent tasks on all 20 tries to passing 107. Tasks passed at least once fell from 453 to 446. No benchmark task was used in training, but the benchmark is Microsoft's own ThinkingBox, and the authors have not measured whether the gain carries over to other agent settings (see the Note "All twenty times").

    Achieved whena group outside Microsoft repeats the gain on held-out tasks of a public benchmark.

    commandline.microsoft.com

    Claimed by

    Tuhin Kundu and Mirco Milletari, Microsoft, November 2026

  2. Training without loss spikes

    A loss spike is a sudden jump in training error that can force a restart. Moonshot AI says its MuonClip optimizer (the rule that decides how to change a model's numbers at each step), which adds a technique it calls QK-clip, kept the 15.5-trillion-token pre-training of its 1-trillion-parameter Kimi K2 free of any loss spike. A preprint from October 2026 by Masato Fujitake reports that Muon-family optimizers can still produce large spikes in sparse mixture-of-experts training; it does not test QK-clip.

    Achieved whenanother lab publishes the full loss curve of a run of similar size that used the method and shows no spike.

    arxiv.orgarxiv.org

    Claimed by

    Moonshot AI (Kimi Team), July 2025, revised February 2026

Open

  1. 01

    Spike warnings on real runs

    A labeled public set of loss spikes taken from large training runs whose logs are published, and a warning tool scored on it: how many steps before each spike it raises an alarm, and how often it raises a false one.

    The brief
    Credit · unclaimed
  2. 02

    Repeatable answers, checked outside

    An open repeat of Microsoft's ThinkingBox result: train a small open model by reinforcement learning to pass the same task every time, then measure it on a public benchmark it never saw in training.

    The brief
    Credit · unclaimed

Open 01 · the brief

Spike warnings on real runs

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A labeled public set of loss spikes taken from large training runs whose logs are published, and a warning tool scored on it: how many steps before each spike it raises an alarm, and how often it raises a false one.

Why it is still open

Teams still handle spikes by hand. The OPT-175B team made at least 35 manual restarts and rolled back to earlier copies of the model when the loss diverged. Open tools such as trainspotter (2026) locate spikes in a log once they have happened, and its authors call its checks heuristics. Searches of arXiv and GitHub found papers on why spikes happen and a 2023 MosaicML prediction repository with no code or description, but no public test of advance warning on real large runs.

arxiv.orggithub.com

Built with what exists

the Allen Institute for AI's published Weights & Biases training logs for its open OLMo 2 models (1, 7 and 13 billion parameters); the OPT-175B logbook; Python with pandas; simple statistics or a small sequence model over the loss and the other measures each log records.

  1. First step

    Download the OLMo 2 7B training log and mark every step where the loss rises more than a set amount above its recent average.

  2. Done looks like

    on runs held out from tuning, the tool warns at least 100 steps before at least half of the labeled spikes, with no more than one false alarm per 10,000 steps, and the labels, code and scores are public.

Open 02 · the brief

Repeatable answers, checked outside

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

An open repeat of Microsoft's ThinkingBox result: train a small open model by reinforcement learning to pass the same task every time, then measure it on a public benchmark it never saw in training.

Why it is still open

Microsoft's November 2026 result was measured on its own benchmark, its authors have not measured transfer to other agent settings, and no outside repeat was found. τ-bench's authors defined pass^k, the share of tasks passed on all k tries, in 2024, and found GPT-4o passed all eight tries on fewer than 25% of retail tasks.

commandline.microsoft.comarxiv.org

Built with what exists

τ²-bench, Sierra's open follow-up to τ-bench, with customer-service tasks an agent completes through tools; an open-weight model of about 8 billion parameters; the verl or Hugging Face TRL reinforcement-learning libraries; one rented machine with eight H100 GPUs.

  1. First step

    Run the untrained model eight times on each τ²-bench retail task and record its pass^1 and pass^8.

  2. Done looks like

    pass^8 before and after training on τ²-bench tasks held out from training, from three training runs with different random seeds, with the share of tasks passed at least once also reported, and code and logs public.