BRENNEN

Industries / Government, education and community / Education / Tutoring

Measuring progress

Tutors, families and the programs that pay for tutoring check whether a student is learning, from one session to the next and over months.

5

Milestones achieved, from 1985 to 2024.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1985

    One-minute progress checks

    In 1985 Stanley Deno set out curriculum-based measurement, a way to track a student's growth in basic skills that began in special education. A 1986 meta-analysis by Lynn and Douglas Fuchs of 21 controlled studies found an average effect of 0.70 standard deviations on achievement from systematic formative evaluation, in which progress is checked often while teaching is under way. Tutoring programs such as Minnesota Reading Corps give one-minute reading checks every week and use them to decide when a student can leave tutoring.

    eric.ed.goveric.ed.goveric.ed.govlrl.mn.gov

    Credit

    Stanley L. Deno; Lynn S. Fuchs and Douglas Fuchs

  2. 2012

    Six districts, no gain

    Supplemental Educational Services, the free after-school tutoring that the US No Child Left Behind Act required for low-income pupils in struggling schools, was largely delivered by private tutoring organizations. A Mathematica evaluation for the Institute of Education Sciences, in six districts with more applicants than places, found no statistically significant effect on reading or math scores; students received an average of 21.2 hours of tutoring in the year.

    LessonMatthew Kraft and colleagues later linked the small results of programs like this to poor attendance at off-site after-school sessions, large groups and rotating tutors.

    eric.ed.govedworkingpapers.com

    Credit

    John Deke, Lisa Dragoset, Karen Bogen and Brian Gill, Mathematica Policy Research

  3. 2023

    Too little data to judge

    England's National Tutoring Programme paid for tutoring through three routes: Tuition Partners (outside tutoring organizations), Academic Mentors placed in schools, and school-led tutoring. The evaluation of its second year, 2021–22, published by the Department for Education in October 2023, found school-led tutoring linked to gains of one month's progress or less in maths and English, and no evidence of gains from the other two routes. Schools chose which pupils to tutor, no subject was recorded for school-led tutoring, and two official sources of who took part agreed poorly.

    LessonWithout random assignment or complete session records, a national program could not show clearly what its tutoring achieved.

    assets.publishing.service.gov.uk

    Credit

    Megan Lucas, Emma Moore, Chris Morton, Ruth Staunton and Stephen Welbourne, NFER (National Foundation for Educational Research)

  4. 2024

    Gains shrink as programs grow

    Matthew Kraft, Beth Schueler and Grace Falken pooled 282 randomized trials of tutoring. For large US programs judged on standardized tests, pooled effects were a third to a half the size of those in the full sample, and they fell steeply as programs served more students. Programs with a set of features (in person, at school during school hours, no more than three students per tutor, at least three sessions a week, at least 15 hours in all and a set curriculum) lost less: effects for programs of 100 to 399 students were 18% below those for programs of 99 or fewer, against 42% across all studies.

    LessonEffects from small trials overstate what a large program should expect.

    edworkingpapers.comedworkingpapers.com

    Credit

    Matthew A. Kraft (Brown University), Beth E. Schueler (University of Virginia) and Grace Falken (Brown University)

  5. 2024

    A district measures its tutoring

    Metro Nashville Public Schools ran its own tutoring program after the pandemic, switched from volunteer tutors to teachers, and delivered more than 125,000 hours to more than 6,800 students in two years. Researchers found reading gains of 0.04 to 0.09 standard deviations and no effect on math scores or course grades.

    LessonAmong their explanations the authors list a small difference between tutored and untutored students, a short program and "miscalibrated expectations" of what tutoring at scale can do.

    edworkingpapers.com

    Credit

    Metro Nashville Public Schools; Matthew Kraft, Danielle Sanderson Edwards and Marisa Cannata

Watch for

  1. An open library of sessions

    The National Tutoring Observatory says its Million Tutoring Moves project will build a large, open, multimodal record of real tutoring. Its first release is 4,654 de-identified text transcripts of math sessions from a US nonprofit online tutoring service, with audio and video planned, and access "subject to appropriate governance and access controls".

    Achieved whenOutside researchers can obtain the data under published terms and report studies that use it.

    arxiv.org

    Claimed by

    René Kizilcec, Kenneth Koedinger, Justin Reich and colleagues, National Tutoring Observatory, April 2026

  2. AI scores tutor moves

    Researchers gave 50 transcripts of college students tutoring middle schoolers in math to AI models, including GPT-4o, Gemini 1.5 Pro and LearnLM. The models spotted praise in 94–98% of cases and student math errors in 82–88%, and their ratings of whether tutors followed good practice matched human judges 83–89% of the time for praise and 73–77% for responding to errors.

    Achieved whenAn outside team gets similar agreement on a larger set of sessions from a different program.

    arxiv.org

    Claimed by

    Danielle Thomas, Conrad Borchers, Kenneth Koedinger and colleagues, June 2025 (accepted at the EC-TEL 2025 conference)

Open

  1. 01

    Scores from speech, not text

    A test of how much speech-recognition errors change automatic scores of tutoring sessions. Experienced tutors write 200 session scripts in which every tutor turn carries a known label, such as praise, a hint or a response to a student's error; consenting adult volunteers act them out over a video call; the recordings are transcribed, and the same AI prompts score both the script and the transcript.

    The brief
    Credit · unclaimed
  2. 02

    Session recaps, checked

    An outside count of the errors in AI-written session recaps that tutoring services now send to families. Build 50 scripted sessions, acted by consenting adults, each with eight planted facts (a skill the "student" masters, one still wrong, homework set, a confusion left unresolved), run the recordings or transcripts through recap tools that accept them and through general AI models, and score every recap against the script.

    The brief
    Credit · unclaimed

Open 01 · the brief

Scores from speech, not text

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A test of how much speech-recognition errors change automatic scores of tutoring sessions. Experienced tutors write 200 session scripts in which every tutor turn carries a known label, such as praise, a hint or a response to a student's error; consenting adult volunteers act them out over a video call; the recordings are transcribed, and the same AI prompts score both the script and the transcript.

Why it is still open

The open tutoring datasets are text: the Million Tutoring Moves paper calls its first release "an initial textual release", with audio and video still to come. Thomas and colleagues call identifying tutor actions at scale from audio transcriptions "an open research problem". The closest study, from the University of Colorado Boulder, measured speech-recognition errors in small-group classroom talk, not one-to-one tutoring.

arxiv.orgarxiv.orgpar.nsf.gov

Built with what exists

Speech-to-text such as ElevenLabs Scribe v3 (see the Note "Fewer word errors") and OpenAI's Whisper; Thomas and colleagues' published prompts for praise and error responses; AI models such as Claude and Gemini; the National Tutoring Observatory's open Sandpiper tool for annotating transcripts.

  1. First step

    Write five scripts, have two volunteers act them, and score script and transcript with one prompt.

  2. Done looks like

    For 200 acted sessions, a published table, per speech-to-text system and scoring model, of word error rate and of the share of tutor-move labels that change between script and transcript, with each model scored 20 times per session to separate transcription error from run-to-run variation (see the Note "All twenty times").

Open 02 · the brief

Session recaps, checked

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

An outside count of the errors in AI-written session recaps that tutoring services now send to families. Build 50 scripted sessions, acted by consenting adults, each with eight planted facts (a skill the "student" masters, one still wrong, homework set, a confusion left unresolved), run the recordings or transcripts through recap tools that accept them and through general AI models, and score every recap against the script.

Why it is still open

ClassDojo's tutoring service sends AI-generated recaps to families four hours after a session, with tutor editing optional, and Brighterly launched AI lesson summaries in beta in April 2026. Neither page gives an error rate, and this research found no outside count.

tutor-help.classdojo.comfortune.com

Built with what exists

Scripts written by experienced tutors; video-call recording; speech-to-text such as Scribe v3 or Whisper; AI models such as Claude, Gemini and GPT; a scoring sheet used by two scorers working independently.

  1. First step

    Write five scripts with planted facts, record them, and compare two general AI models' recaps against the scripts.

  2. Done looks like

    A public table, per tool, of planted facts captured, false statements per recap and recaps that claim progress the script shows did not happen, across all 50 sessions, with the two scorers agreeing on at least 90% of judgments.