BRENNEN

Industries / Finance and business services / Legal and professional services / LegalTech

Discovery

Finding, keeping, reviewing and handing over the documents and data a lawsuit or investigation needs, while holding back what is privileged or private.

7

Milestones achieved, from 2003 to 2025.

3

To watch for: announced, not yet confirmed outside.

3

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 2003

    Email is evidence too

    In Zubulake v. UBS Warburg, Judge Shira Scheindlin held in May 2003 that "electronic documents are no less subject to disclosure than paper records," and set out factors for deciding who pays to restore hard-to-reach data such as backup tapes. In a 2004 ruling she held that sending a litigation hold, a notice telling staff to keep relevant records, is not enough: "Counsel must take affirmative steps to monitor compliance."

    LessonA hold notice that no one checks does not keep evidence.

    hallapproved.comediscoverylaw.com

    Credit

    Judge Shira Scheindlin, US District Court for the Southern District of New York

  2. 2006

    A shared map of discovery

    George Socha and Tom Gelbmann gathered nearly 40 practitioners in May 2005 to draw a common model of the discovery process, and published the Electronic Discovery Reference Model (EDRM) on 1 January 2006. Its stages run from identifying and preserving data through collection, processing, review and analysis to production and presentation, and vendors and lawyers use them to describe the work.

    revealdata.comzapproved.com

    Credit

    George Socha, Tom Gelbmann

  3. 2006

    Rules name electronic information

    Amendments to the US Federal Rules of Civil Procedure that took effect on 1 December 2006 added "electronically stored information" (ESI), anything kept in electronic form such as email, and confirmed that its discovery "stands on equal footing" with paper.

    law.cornell.edu

    Credit

    Judicial Conference of the United States, Advisory Committee on Civil Rules

  4. 2015

    Courts abroad accept TAR

    Courts outside the US accepted technology-assisted review (TAR), software that learns from a lawyer's calls on sample documents and ranks the rest. In Ireland, Mr Justice Fullam approved it in March 2015 for 680,809 documents although the parties had not agreed on it. In England, Master Matthews approved it in February 2016 for about 3.1 million documents, writing that "no English court has given a judgment which has considered the use of predictive coding." The Supreme Court of Victoria, in Australia, wrote it into a practice note in January 2017.

    LessonCourts accepted TAR when its process could be checked, a test the Irish court called "sufficient transparency.".

    ie.vlex.comcaselaw.nationalarchives.gov.uksupremecourt.vic.gov.au

    Credit

    Fullam J (High Court of Ireland); Master Matthews (High Court of England and Wales, Chancery Division); Supreme Court of Victoria

  5. 2019

    Redactions that could be copied

    In January 2019 lawyers for Paul Manafort filed a court document whose blacked-out text anyone could read by copying and pasting it. A 2022 study by University of Illinois researchers tested 11 PDF redaction tools and found that hidden text can often be partly worked out from the positions of the characters around it, and the authors recovered text from hundreds of real documents.

    LessonA black box drawn over text does not remove it; redaction has to delete the text from the file.

    cjr.orgarxiv.org

    Credit

    Mathew Ingram (Columbia Journalism Review) for the Manafort report; Maxwell Bland, Anushya Iyer and Kirill Levchenko for the study

  6. 2023

    Deleted chats bring sanctions

    In the Google Play antitrust case, Judge James Donato found that about 360 Google employees were under a legal hold, but Google did not turn chat history on for them, so their one-to-one chats were kept for only 24 hours. He ordered Google to pay the plaintiffs' fees and costs for the motion and left further sanctions open.

    LessonA legal hold has to change the system's settings, not rely on each person to save chats.

    sidley.com

    Credit

    Judge James Donato, US District Court for the Northern District of California, March 2023

  7. 2025

    Privilege logs planned early

    Since 1 December 2025, parties in US federal cases must agree at the start how and when they will describe the documents they withhold as privileged, usually in a privilege log, a list of withheld items with the reason for each. The committee's note says a log produced "near the close of the discovery period can create serious problems."

    law.cornell.edu

    Credit

    Advisory Committee on Civil Rules

Watch for

  1. Questions across a whole case

    Relativity says claiR will let lawyers ask plain-language questions "across an entire RelativityOne matter", the whole case file in its review platform, with answers that cite documents. A&O Shearman, Foley & Lardner and K&L Gates have early access, general release is planned for early 2027, and no accuracy figures are given.

    Achieved whenan outside party checks a sample of its answers and citations against the record and publishes the error rate.

    lawnext.com

    Claimed by

    Relativity, August 2026

  2. Agents for fact investigation

    DISCO says an "agentic" upgrade to its Cecilia tool, which plans and runs several search and reasoning steps on its own, can investigate matters with "millions of documents and terabytes of data." The report gives no accuracy figures and names no outside test.

    Achieved whenan independent test on a collection with known answers, or a court-reviewed use, reports how often its findings are right and complete.

    lawnext.com

    Claimed by

    DISCO, February 2026

  3. England reviews disclosure rules

    Disclosure is the English term for discovery. A working group of judges chaired by Mr Justice Butcher says that "simplification of the existing regime" for disclosure in England's Business and Property Courts, set by Practice Direction 57AD, "is desirable", and is studying how TAR and AI affect the work.

    Achieved whenthe Civil Procedure Rule Committee adopts amended disclosure rules and they take effect.

    judiciary.uk

    Claimed by

    Disclosure Review Working Group, May 2026

Open

  1. 01

    A modern chat test collection

    A free public test collection of invented workplace chat, so that review tools can be scored on today's data instead of old email: Slack- and Teams-style channels and direct messages with threads, edits, deletions, emoji reactions and file links, plus at least 20 mock document requests and relevance labels made by people.

    The brief
    Credit · unclaimed
  2. 02

    Personal data test for filings

    A scored test of automatic redaction on realistic US court filings, including scanned exhibits such as bank statements, covering what federal privacy rules say filers must redact or shorten: Social Security and taxpayer numbers, birth dates, minors' names and financial account numbers. It also checks that each redaction really removed the text. It uses public filings with invented numbers added.

    The brief
    Credit · unclaimed
  3. 03

    Hidden instructions for AI reviewers

    A test set of documents that hide instructions aimed at AI review tools, each meant to change a relevance or privilege call, paired with clean copies. This kind of attack is called prompt injection. The hiding places include white text, tiny fonts, comments, metadata, image descriptions and the hidden text layer of scanned pages. Score how often review tools obey the hidden text, and how often a pre-check finds it.

    The brief
    Credit · unclaimed

Open 01 · the brief

A modern chat test collection

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A free public test collection of invented workplace chat, so that review tools can be scored on today's data instead of old email: Slack- and Teams-style channels and direct messages with threads, edits, deletions, emoji reactions and file links, plus at least 20 mock document requests and relevance labels made by people.

Why it is still open

Craig Ball, writing for EDRM in August 2025, says the shared test data predates "smartphones, Teams, Slack, Zoom, linked attachments" and that "The only large, lawful, realistic corporate dataset we all share is woefully out of date." EDRM's data-set page lists no chat set. KLDiscovery, a review provider, wrote in 2021 that there was "no Enron equivalent data set for chat messages."

edrm.netedrm.netkldiscovery.com

Built with what exists

OrgForge (March 2026), an open-source tool that simulates a company and writes its Slack messages, email and meeting transcripts while keeping a log of what really happened, but has no document requests or relevance labels; Relativity Short Message Format (RSMF), a documented format that review tools use for chat; Claude, Gemini, GPT or open-weight models to write message text; two or more people to label relevance.

arxiv.org

  1. First step

    Generate one small company with OrgForge, convert one channel to RSMF, write five requests, and have two people label 500 messages to measure how often they agree.

  2. Done looks like

    At least 50,000 messages in at least 100 channels and at least 20 requests, each request labeled by two people with their agreement reported; released in Slack-export JSON and RSMF under an open license; and recall and precision for at least two review methods. Recall is the share of relevant messages a method finds; precision is the share of what it finds that is relevant.

Open 02 · the brief

Personal data test for filings

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A scored test of automatic redaction on realistic US court filings, including scanned exhibits such as bank statements, covering what federal privacy rules say filers must redact or shorten: Social Security and taxpayer numbers, birth dates, minors' names and financial account numbers. It also checks that each redaction really removed the text. It uses public filings with invented numbers added.

Why it is still open

In April 2026 the Federal Judicial Center (FJC), the federal courts' research agency, found 22,391 unredacted Social Security and taxpayer numbers, belonging to about 8,300 people, in nearly 4.7 million PACER documents; filers had tried and failed to redact a third of them. For account numbers its researchers "were unable to estimate their prevalence in the timeframe allotted." The one public test of this kind I found, TAB (2022), uses 1,268 judgments of the European Court of Human Rights, not US filings.

fjc.govarxiv.org

Built with what exists

public filings from the RECAP archive on CourtListener, with invented identifiers added; Microsoft Presidio, an open-source detector of personal data; Tesseract to read scans; pdfplumber or Docling; Free Law Project's X-Ray, which finds boxes drawn over text that is still there; Claude, Gemini or GPT for comparison, and an open-weight model such as Mistral Large 4 that a team can run on its own machines (see the Note "A trillion open parameters"). Any real identifier found goes to the court, never into the published set.

free.law

  1. First step

    Pull 200 public exhibits of the kinds the FJC flagged, add invented identifiers, and run Presidio and X-Ray for a first score.

  2. Done looks like

    A public labeled set of at least 1,000 documents, at least a third of them scans, with recall and precision for each kind of identifier for at least three tools, and a count of redactions whose text can still be pulled out.

Open 03 · the brief

Hidden instructions for AI reviewers

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A test set of documents that hide instructions aimed at AI review tools, each meant to change a relevance or privilege call, paired with clean copies. This kind of attack is called prompt injection. The hiding places include white text, tiny fonts, comments, metadata, image descriptions and the hidden text layer of scanned pages. Score how often review tools obey the hidden text, and how often a pre-check finds it.

Why it is still open

In Elliott v. New York Bariatric Group (August 2026), a Connecticut court sanctioned a self-represented plaintiff who put tiny white text in his filings telling any AI reviewing them to favor him; the court does not use AI to review filings, and the judge read a printed copy. Research published in September 2025 by Pangea, a security company, showed instructions hidden in legal fine print steering major models, but in code review, not document review. I found no test of this kind for legal review tools.

dechert.comtheregister.com

Built with what exists

invented emails, Word files and PDFs; Claude, Gemini, GPT and open-weight models running a standard review prompt; pdfplumber or Docling to surface hidden text.

  1. First step

    Build 50 pairs using white text and metadata, run them through two models with a review prompt, and record every call that changes.

  2. Done looks like

    At least 500 documents across at least six hiding methods, each with a clean twin; the rate at which each model or tool obeys the hidden text; and a scanner that flags at least 95% of documents with hidden text while wrongly flagging no more than 2% of clean ones.