BRENNEN

Industries / Finance and business services / Legal and professional services

Litigation and discovery

Preparing and fighting lawsuits, including discovery: each side's duty to hand over relevant documents and data to the other.

5

Milestones achieved, from 1938 to 2026.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1938

    Discovery written into rules

    The US Supreme Court approved the Federal Rules of Civil Procedure in 1938. They govern pleading, motions, discovery and trials in federal civil cases.

    law.cornell.edu

    Credit

    US Supreme Court and its Advisory Committee, appointed in 1935

  2. 2011

    Software matched manual review

    Maura Grossman and Gordon Cormack used data from the TREC 2009 Legal Track, a shared research test, to compare technology-assisted review (TAR, software that learns from lawyers' decisions on sample documents) with reviewers reading every document. The best TAR processes did better on recall (the share of relevant documents found) and precision (the share of documents found that were relevant), while people read only a small part of the collection.

    grossman.uwaterloo.ca

    Credit

    Maura R. Grossman, Gordon V. Cormack

  3. 2012

    A court approves TAR

    In Da Silva Moore v. Publicis Groupe, US Magistrate Judge Andrew Peck wrote that "computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases", ESI being electronically stored information. The opinion says it "appears to be the first in which a Court has approved of the use of computer-assisted review."

    storage.courtlistener.com

    Credit

    Magistrate Judge Andrew J. Peck, Southern District of New York

  4. 2021

    Convictions on computer evidence quashed

    England's Court of Appeal quashed the convictions of 39 former sub-postmasters whom Post Office Limited had prosecuted between 2003 and 2013, relying on its Horizon accounting system. The court found the Post Office treated "a shortfall shown by an unreliable accounting system as an incontrovertible loss" and denied defendants disclosure of material that could undermine its case.

    LessonRecords produced by a computer are evidence to be tested, not proof.

    judiciary.uk

    Credit

    Court of Appeal (Holroyde LJ, Picken J, Farbey J), Hamilton v Post Office

  5. 2026

    A court accepts generative review

    In Schulte v. LinkedIn, Magistrate Judge Laurel Beeler allowed LinkedIn to use Relativity aiR, a generative AI tool, to decide which documents were responsive after search terms had narrowed the set. She declined to order the extra accuracy measures the plaintiffs asked for. The law firm DLA Piper calls it the first federal decision accepting generative AI for final responsiveness decisions.

    arnoldporter.comdlapiper.com

    Credit

    Magistrate Judge Laurel Beeler, Northern District of California, 30 June 2026

Watch for

  1. Generative review beats TAR

    A June 2026 working paper from the law firm Redgrave reports that Relativity aiR found 88% of the responsive documents in a 45,004-document collection from an opioid maker, against 64% for a 24-person review team using Relativity's active learning. In October 2026 Gordon Cormack, Maura Grossman and Jeremy Pickens replied that the two figures measured different tasks, and on a common basis estimated 59.5% for aiR and 64% for the team.

    Achieved whena blind test on a shared collection, scored by assessors with no stake in the result, shows generative review finding more.

    complexdiscovery.com

    Claimed by

    Redgrave LLP, June 2026

  2. A rule for machine evidence

    Proposed Federal Rule of Evidence 707 would make machine output offered without an expert witness meet the reliability tests that apply to expert testimony. Public comment ran from August 2025 to February 2026, and in June 2026 the Judicial Conference's Standing Committee decided "not to recommend action on Rule 707 at this time" and sent it back for more study.

    Achieved whenthe rule, or a revised version, is adopted and takes effect.

    proskauer.com

    Claimed by

    the federal Advisory Committee on Evidence Rules, August 2025

Open

  1. 01

    Blind test for AI review

    A shared test in which any document-review tool, generative or not, is run on the same public collection and the same document requests, and scored by assessors who do not know which tool marked which document.

    The brief
    Credit · unclaimed
  2. 02

    Privilege calls, scored

    A published score for AI tools on finding privileged documents: messages a party may withhold because they are confidential legal advice between lawyer and client, or work prepared for litigation.

    The brief
    Credit · unclaimed

Open 01 · the brief

Blind test for AI review

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A shared test in which any document-review tool, generative or not, is run on the same public collection and the same document requests, and scored by assessors who do not know which tool marked which document.

Why it is still open

The 2026 studies of generative review used one matter's documents and disagree about what they measured, and in Schulte v. LinkedIn the court did not require accuracy measures. The TREC Legal Track, the shared test whose data gave TAR its evidence, ran from 2006 to 2011; its collections and answer keys are free to use.

complexdiscovery.comtrec-legal.umiacs.umd.edu

Built with what exists

the TREC Legal Track collections (scanned tobacco-industry records, and the EDRM email collection used in 2010 and 2011) with their answer keys; new document requests written for this test; review tools built on Claude, Gemini, GPT and open-weight models; random sampling to estimate recall. (see the Note "A thousand questions, one pass")

  1. First step

    Download the 2010 collection and its answer keys, rerun one past request with a current model, and check that the scoring reproduces the published results.

  2. Done looks like

    Five new requests, each with an answer key from a random sample of at least 1,500 documents judged by two lawyers, and recall and precision with 95% confidence intervals for at least four tools, published with the code to rerun them.

Open 02 · the brief

Privilege calls, scored

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A published score for AI tools on finding privileged documents: messages a party may withhold because they are confidential legal advice between lawyer and client, or work prepared for litigation.

Why it is still open

The 2010 TREC Legal Track added a privilege review topic, scored against the calls of an expert called the Topic Authority. I found no published test of current AI models on it.

trec-legal.umiacs.umd.edu

Built with what exists

the 2010 privilege topic and its judgments; a model asked to label each document privileged or not and give its reason; a classifier that returns a probability for each fixed answer, such as Gemini Decision 1. (see the Note "A thousand questions, one pass")

  1. First step

    Get the 2010 privilege topic and its judgments from the TREC Legal Track site and score one model on 200 judged documents.

  2. Done looks like

    Recall and precision for at least three models on the topic's judged documents, each run five times to show how often a label changes from one run to the next.