BRENNEN

Industries / Government, education and community / Government / Libraries

Archives and preservation

Keeping old, rare and fragile material usable: special collections and the papers libraries hold, repairing damaged books, copying them to film or digital files, long-term digital preservation and archiving the web.

7

Milestones achieved, from 1938 to 2019.

3

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1938

    Newspapers move to microfilm

    Microfilm is photographs of each page shrunk onto a reel of film. In summer 1938 Harvard's library began filming foreign newspapers this way, paid for partly by selling the films and partly by the Rockefeller Foundation, and by 1940 it was filming papers from 37 countries. The film outlasted newsprint and took far less shelf space. In 1942 Eugene B. Power, founder of what became University Microfilms, set up a filming operation at the British Museum, and that film later became the base of the Early English Books Online database.

    LessonA cheap copy kept somewhere else saves the text after the paper crumbles.

    thecrimson.comblogs.helsinki.fi

    Credit

    Harvard College Library (directed by Frederick G. Kilgour); Rockefeller Foundation; Eugene B. Power

  2. 1966

    Florence flood remakes conservation

    In November 1966 the river Arno flooded Florence's national library, the Biblioteca Nazionale Centrale di Firenze, damaging about 1.3 million items, a third of its holdings. British bookbinder Peter Waters led the rescue with Anthony Cains and Christopher Clarkson, sorting books by urgency and treating them in stages. The Library of Congress hired Waters in 1971, and in the 1970s his office introduced "phased conservation": stabilizing and boxing many items now and fully repairing a few later.

    LessonWhen the damage is bigger than the staff, sort and stabilize first and repair later.

    loc.govblogs.libraries.indiana.educool.culturalheritage.org

    Credit

    Peter Waters; the Florence rescue teams; Library of Congress Conservation Office

  3. 1988

    Brittle books get a plan

    Acidic paper, common since the mid-1800s, turns brittle and breaks apart. In 1988 the US National Endowment for the Humanities (NEH) planned to microfilm the contents of about 3 million brittle books over 20 years; grants from 1989 to 1991 went to 49 libraries and consortia, and more than 750,000 books had been filmed by the end of 1996, behind plan. Separately, in 1984 the standards body NISO issued Z39.48, rules for paper meant to last several hundred years.

    LessonFilming fell behind schedule, so the longer-term fix was printing new books on paper that lasts.

    clir.orgcrln.acrl.orggroups.niso.org

    Credit

    NEH; Commission on Preservation and Access; NISO

  4. 1996

    Libraries archive the web

    The Internet Archive, a nonprofit library founded by Brewster Kahle, began saving web pages in 1996 and opened them to the public as the Wayback Machine in October 2001, holding more than 10 billion pages. On 6 April 2013 UK legal deposit, the legal duty to give national libraries a copy of every publication, was extended to online material, letting the British Library and five other libraries collect "the entire UK web domain" without asking each site owner. The Internet Archive says it passed one trillion saved pages in October 2025.

    LessonCollecting the web at scale needs a legal mandate as well as software, and in the UK the price was that archived sites can be read only in library reading rooms.

    salon.comgov.ukblog.archive.org

    Credit

    Brewster Kahle and the Internet Archive; the British Library and the other UK and Irish legal deposit libraries

  5. 1999

    Many copies, many libraries

    LOCKSS, "Lots of Copies Keep Stuff Safe", has each library keep its own copy of the e-journals it pays for on networked machines that compare copies with each other and repair damage, so no single copy has to be trusted. More than 50 libraries joined a test from 2000 to 2002, and the software went into production in April 2004.

    LessonNo single copy is safe, however well it is kept.

    lockss.orgdigitalpreservation.gov

    Credit

    Victoria Reich and David S. H. Rosenthal, Stanford Libraries

  6. 2017

    Saving every tweet, reversed

    In April 2010 the Library of Congress agreed with Twitter to receive the text of every public tweet since 2006. In December 2017 it announced that from 1 January 2018 it would collect tweets only selectively, citing the growing volume and the fact that it received text only, without images, video or links. The 2006–2017 collection stays closed, with "no projected timetable" for public access.

    LessonPromising to keep everything means promising storage and access too, and an archive no one can use stays closed.

    blogs.loc.govinformatieprofessional.nl

    Credit

    Library of Congress

  7. 2019

    Software reads old handwriting

    Handwritten text recognition is software that turns images of handwriting into text, and it became a routine archive and library tool through Transkribus, built by the EU-funded READ project (2016 to 2019, €8.2 million, led by the University of Innsbruck) and run since 1 July 2019 by the cooperative READ-COOP. By its own figures for October 2024 it had 227 member institutions in 30 countries, 235,000 registered users and 90 million page images processed. An independent 2025 test measured its Text Titan I model at a 7.07% character error rate on the Bentham papers, the best score on that historical English set, but 40.63% on a historical German set.

    LessonAccuracy depends heavily on script and language, and the tool outlived its grant because its users co-own it.

    eprints.whiterose.ac.ukarxiv.org

    Credit

    Günter Mühlberger and the University of Innsbruck; READ-COOP; Transcribe Bentham at University College London, which supplied early training data

Watch for

  1. Gemini 3 reads old handwriting

    Historian Mark Humphries reports that Gemini 3 Pro transcribed 50 handwritten English documents from the 18th and 19th centuries, about 10,000 words, at a 1.67% character error rate and a 4.42% word error rate under strict scoring, against about 8% character error for Transkribus without extra training. The test set is the authors' own and is not public.

    Achieved whenan independent team gets similar error rates on a public test set known to be outside the model's training data, including non-English scripts, and publishes its scripts.

    generativehistory.substack.com

    Claimed by

    Mark Humphries, with Lianne Leddy, November 2025

  2. A small open model leads

    Stanford's CHURRO, a vision-language model with 3 billion parameters and public weights, scores 70.1% normalized Levenshtein similarity on handwriting (a measure of how closely output text matches the correct text) on its authors' own test set of 99,491 pages from 155 collections. That is 6.5 points above Gemini 2.5 Pro, at about one-fifteenth of the cost.

    Achieved whenoutsiders reproduce the result on material outside the authors' set, or a library runs it in production and reports its accuracy.

    arxiv.org

    Claimed by

    Semnani, Zhang, He, Tekgürler and Lam, Stanford, September 2025

  3. Transkribus's own 5% figure

    Board members of READ-COOP write that "the average CER for handwritten text is currently 5%", meaning a 5% character error rate. A peer reviewer, Thibault Clérice, points out that Transkribus reports scores on the data used while tuning models, not on a held-back test set, and the independent 2025 test found 7.07% to 40.63% depending on the material.

    Achieved whenan outside party measures about 5% on held-back test sets across several scripts.

    eprints.whiterose.ac.uk

    Claimed by

    Melissa Terras and colleagues (READ-COOP), October 2025

Open

  1. 01

    Machine versus volunteer transcripts

    A public, scored comparison of current vision-language models with the Library of Congress's "By the People" volunteer transcriptions of handwritten pages, on at least 1,000 pages from at least 5 finished campaigns. Run each page through Gemini, Claude, Transkribus and an open model run locally, score each against the volunteer text, then have two people judge the 100 pages where machine and volunteer disagree most and count who was right. Report errors in names, dates and numbers, and skipped lines, separately.

    The brief
    Credit · unclaimed
  2. 02

    Draft folder lists for archivists

    A test of whether a vision model can draft the folder-level part of a finding aid, an archive's guide to what is in each box and folder, with dates. Use collections that are already digitized and already have an archivist's folder list, so no originals are handled. Hide the list, give the model each folder's page images, and ask for a folder title, a date range and the main names; an archivist marks each draft accepted, lightly edited or rewritten, and times the corrections.

    The brief
    Credit · unclaimed

Open 01 · the brief

Machine versus volunteer transcripts

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A public, scored comparison of current vision-language models with the Library of Congress's "By the People" volunteer transcriptions of handwritten pages, on at least 1,000 pages from at least 5 finished campaigns. Run each page through Gemini, Claude, Transkribus and an open model run locally, score each against the volunteer text, then have two people judge the 100 pages where machine and volunteer disagree most and count who was right. Report errors in names, dates and numbers, and skipped lines, separately.

Why it is still open

The Library's May 2025 post describes machine reading only for typed pages and says mixed handwritten collections "don't make good candidates for batch OCR processing". Published benchmarks use models from October 2024 or standard research sets, and Humphries' test used 50 private documents. Searches for a comparison of AI against By the People volunteers found none.

blogs.loc.govarxiv.org

Built with what exists

By the People's published datasets of volunteer transcriptions, with page images from loc.gov; Gemini, Claude and GPT vision models; Transkribus; CHURRO or another open model run through llama.cpp (see the Note "A first stable number for local AI"); a standard error-rate scoring library.

  1. First step

    Download one finished handwritten campaign, pick 50 pages, run two models and compute the character error rate against the volunteer text.

  2. Done looks like

    a public repository with page identifiers, every model's output, a table of character and word error rates per campaign, the 100 judged pages with who-was-right counts, and a script to rerun when a new model ships.

Open 02 · the brief

Draft folder lists for archivists

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A test of whether a vision model can draft the folder-level part of a finding aid, an archive's guide to what is in each box and folder, with dates. Use collections that are already digitized and already have an archivist's folder list, so no originals are handled. Hide the list, give the model each folder's page images, and ask for a folder title, a date range and the main names; an archivist marks each draft accepted, lightly edited or rewritten, and times the corrections.

Why it is still open

In 2010 OCLC found that only 44% of archival collections at 169 US and Canadian research libraries had an online finding aid, and counted 26% as backlog with none. The nearest tests describe single items, not folders: the University of Kansas used Claude Sonnet 4 to describe 2,263 digitized photographs and its reviewers found major problems in 2% to 6% of a 100-item pilot, and JSTOR's Seeklight tool, tested by Eastern Michigan University, drafts titles and descriptions but its April 2026 report gives no acceptance rate. A 2026 test of five general AI services found them "unsuitable for automating precise technical description processes".

oclc.orgemerald.comabout.jstor.orgjournal.almamater.si

Built with what exists

Claude or Gemini vision models, with Claude reading long runs of pages for problems (see the Note "Seven more flagged"); Library of Congress manuscript collections that have online finding aids; a spreadsheet in the import format of ArchivesSpace, a common archives system; archivist reviewers.

  1. First step

    Choose one digitized Library of Congress collection with an online finding aid, draft folder lists for 3 boxes, and score them with one archivist.

  2. Done looks like

    at least 500 folders from at least 20 boxes scored, with the acceptance rate, how often date ranges match, and correction minutes per folder compared with writing from scratch, all published.