BRENNEN

Industries / Government, education and community / Education / Homeschooling

Testing and evaluation

How the learning of children taught at home is checked, by standardized tests, by a qualified person reviewing a portfolio of their work, or by sitting public exams, and what studies of those results have found.

6

Milestones achieved, from 1988 to 2023.

0

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1988

    Evaluators sign off yearly

    Pennsylvania's Act 169 of 1988 added a home education section to the state School Code. Families keep a portfolio of the child's work, which in grades 3, 5 and 8 must include state test results or an approved national test, and each year an evaluator, such as a certified teacher or a licensed psychologist, writes a report on it, due by 30 June.

    pa.govpa.gov

    Credit

    Pennsylvania General Assembly

  2. 1999

    Twenty thousand test scores

    In spring 1998, 20,760 home-educated US students from 11,930 families sat the Iowa Tests of Basic Skills or the Tests of Achievement and Proficiency through Bob Jones University Press's testing service, and their median scores were mostly between the 70th and 80th percentile. The author wrote that the study "does not demonstrate that home schooling is superior" and that he could not tell whether the families who chose to test were representative.

    LessonA large sample of families who chose to test cannot show what homeschooling does for children in general.

    epaa.asu.edu

    Credit

    Lawrence M. Rudner, University of Maryland, in a study commissioned by the Home School Legal Defense Association

  3. 2011

    Matched comparison groups

    Researchers tested home-educated and school-educated children matched by family background, recruited and tested the same way. Children taught at home in a structured way outperformed their matched peers, and those taught in an unstructured way, often called unschooling, underperformed; a 2020 review credits the study with pioneering the method.

    icher.org

    Credit

    Sandra Martin-Chang, Odette Gould and Reanne Meuse, in the Canadian Journal of Behavioural Science

  4. 2020

    The evidence weighed

    A survey of more than 2,000 English-language texts on homeschooling found that many of the largest achievement studies had "serious design flaws" and concluded that homeschooling "does not have that much of an effect" on achievement once family background is controlled. It also reported that in the Cardus Education Surveys, which used representative national samples, formerly homeschooled young adults reported lower SAT scores than privately schooled peers.

    LessonA study says something about homeschooling only when its comparison group comes from similar families.

    icher.org

    Credit

    Robert Kunzman of Indiana University and Milton Gaither of Messiah College, in the journal Other Education

  5. 2021

    Private candidates get a route

    With summer exams cancelled and grades set by teachers, England's Department for Education promised home-educated pupils, who sit GCSEs and A levels as private candidates with no school that taught them, "a clear and accessible route" to a grade, and paid exam centers £200 per entry to assess them. The Joint Council for Qualifications (JCQ), the body of the main exam boards, listed the centers taking them, and it still updates that list each winter; the current list names 190 centers.

    LessonA grading plan built on teachers' judgment has to say who judges the pupils who have no teacher.

    dera.ioe.ac.ukjcq.org.uk

    Credit

    Department for Education; Ofqual; JCQ

  6. 2023

    A third admissions test

    On 8 September 2023 Florida's state university system voted to accept the Classic Learning Test (CLT) for admission alongside the SAT and ACT, after the Legislature had made it a route to the Bright Futures scholarship. The CLT, created in 2015 by Jeremy Tate and built around classical texts, caught on with homeschoolers and private Christian schools; the College Board has criticized its method and rigor.

    flbog.edunbcnews.com

    Credit

    Jeremy Tate; Florida Board of Governors; Florida Legislature

Open

  1. 01

    Measure evaluator agreement

    100 synthetic portfolios, each a year of made-up writing, math and reading logs that an AI model writes at known grade levels with strengths and gaps planted on purpose, plus a scoring sheet that follows Pennsylvania's and Florida's published rules. At least five working homeschool evaluators judge every portfolio, the project measures how often they agree, and AI raters are then run on the same set.

    The brief
    Credit · unclaimed
  2. 02

    Map distance to exam centers

    A map and table of how far every neighborhood in England is from the nearest exam center that accepts private candidates, by car and by public transport, for GCSE English, maths and the sciences, rebuilt each January when JCQ updates its list. An AI agent reads each center's website to add subjects and fees, with the date each was checked.

    The brief
    Credit · unclaimed

Open 01 · the brief

Measure evaluator agreement

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

100 synthetic portfolios, each a year of made-up writing, math and reading logs that an AI model writes at known grade levels with strengths and gaps planted on purpose, plus a scoring sheet that follows Pennsylvania's and Florida's published rules. At least five working homeschool evaluators judge every portfolio, the project measures how often they agree, and AI raters are then run on the same set.

Why it is still open

Florida lets "a teacher selected by the parent" evaluate progress, and Pennsylvania relies on one evaluator's report. When RAND studied Vermont's school portfolio program in the early 1990s, rater agreement in writing ran only 0.28 to 0.57, and its author called the scores "essentially meaningless". This research found no published study of how often homeschool evaluators agree.

leg.state.fl.usedweek.org

Built with what exists

A current AI model to write the work samples, rendered as scanned pages in handwriting fonts; published state standards to fix grade levels; Google Forms for the evaluators; Krippendorff's alpha, a standard measure of how far raters agree beyond chance (the Python krippendorff package); AI raters that return a probability for each yes-or-no call (the Note "A thousand questions, one pass" covers one).

  1. First step

    Write 10 grade-4 portfolios with three planted gaps each, have two evaluators rate them, and time how long each takes.

  2. Done looks like

    A public dataset and report on all 100 portfolios, giving Krippendorff's alpha among the human evaluators for the "adequate progress" call and for grade-level estimates, and each AI rater's agreement with the evaluators' majority view.

Open 02 · the brief

Map distance to exam centers

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A map and table of how far every neighborhood in England is from the nearest exam center that accepts private candidates, by car and by public transport, for GCSE English, maths and the sciences, rebuilt each January when JCQ updates its list. An AI agent reads each center's website to add subjects and fees, with the date each was checked.

Why it is still open

The JCQ list gives center names, qualifications and phone numbers but not subjects or fees, and warns that some listed centers may no longer accept private candidates. A volunteer's November 2024 count found 168 centers on JCQ's list and 27 more on the volunteer-run HE Exams wiki, with only 49 on all three yearly lists the author had and few in rural areas; no travel times were published.

jcq.org.uklearnwhatyoulive.substack.com

Built with what exists

The JCQ list and the HE Exams wiki; postcodes.io for locations; OpenStreetMap with the OSRM router for drive times; OpenTripPlanner with England's Bus Open Data Service timetables for public transport; Office for National Statistics small areas (LSOAs, about 1,500 people each); an AI agent to read center websites (the Note "The jobs agents repeat" covers a cheaper model for such repeated agent tasks).

  1. First step

    Download the current list of 190 centers, place each by postcode, and compute drive times for one county as a check.

  2. Done looks like

    For every LSOA in England, the travel time to the nearest center for each subject group and the share of areas more than one hour away by public transport, published with the date of the JCQ list used, plus fees and subjects for at least 80% of listed centers.