BRENNEN

Industries / Technology / Software and SaaS

Customer support

Answering customers' questions and fixing their problems after they buy, by staff, through help pages and public forums, and now by AI agents.

7

Milestones achieved, from 1955 to 2025.

2

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1955

    Users form SHARE

    Users of IBM's 704 computer in the Los Angeles area set up SHARE, the Society to Help Avoid Redundant Effort, to swap the programs and manuals they wrote and to deal with IBM as one group. Forty-six people from seventeen 704 installations came to the first meeting, in the RAND Corporation's basement in the week of 22 August 1955. The Smithsonian's archive calls SHARE the first organized user group in the computer industry.

    sirismm.si.edu

    Credit

    Jack Strong and Frank Wagner of North American Aviation, Lee Amaya of Lockheed and Paul Armer of RAND, who phoned the other 704 customers from a booth in Los Angeles

  2. 2001

    Clippy switched off

    Microsoft Office 97 came with an animated helper, the Office Assistant, best known as a paper clip called Clippy, that offered help unasked with lines such as "It looks like you're writing a letter". On 11 April 2001 Microsoft said the assistant would be off by default in Office XP, and from Office 2007 it could not be called up at all.

    LessonHelp that interrupts people who did not ask for it gets switched off.

    tomshardware.com

    Credit

    Microsoft

  3. 2008

    Answers kept in public

    Stack Overflow opened to the public in September 2008 as a free site where programmers ask questions, others answer, and votes move the best answers to the top. Each question is kept as one page and repeats are pointed to the original, so the answers build up into a public reference. Good answers earn points, called reputation, that let a user edit other people's posts.

    infoq.com

    Credit

    Jeff Atwood and Joel Spolsky

  4. 2023

    AI help for support staff

    Economists followed 5,172 support agents at a Fortune 500 firm that sells business software as it brought in an AI assistant that suggests replies during chats. With the assistant, agents resolved 15% more issues per hour on average. The least experienced agents gained the most in speed and quality, and the most experienced saw small gains in speed and small drops in quality.

    arxiv.org

    Credit

    Erik Brynjolfsson, Danielle Li and Lindsey Raymond, published in the Quarterly Journal of Economics in 2025

  5. 2024

    The bot's word binds

    Air Canada's website chatbot told a grieving customer that a reduced bereavement fare could be claimed after the trip, which the airline's written policy did not allow. Air Canada argued the chatbot was in effect a separate legal entity responsible for its own actions. On 14 February 2024 British Columbia's Civil Resolution Tribunal called that "a remarkable submission", found negligent misrepresentation and awarded the customer CA$650.88.

    LessonA company answers for what its chatbot tells customers, as it does for any other page on its website.

    dentonsdata.com

    Credit

    Jake Moffatt, who brought the claim; tribunal member Christopher C. Rivers

  6. 2024

    Fewer questions in public

    Researchers compared Stack Overflow with similar question-and-answer sites where ChatGPT was harder to use or less useful. Within six months of ChatGPT's release in November 2022, activity on Stack Overflow fell by 25% relative to those sites.

    LessonWhen people take their questions to a private AI chat, fewer answers are left in public for the next person, or for the next AI model, to learn from.

    inet.ox.ac.uk

    Credit

    R. Maria del Rio-Chanona, Nadzeya Laurentsyeva and Johannes Wachs, in PNAS Nexus

  7. 2025

    A support bot invents policy

    In April 2025 a user of Cursor, an AI code editor, asked support why switching computers logged them out. A reply signed "Sam" said Cursor was "designed to work with one device per subscription as a core security feature". No such policy existed, Sam was an AI model, and users posted that they were canceling. Cursor co-founder Michael Truell apologized, said the logouts came from a security change, and said AI replies to support email would now be labeled as such.

    LessonAn AI agent that answers customers needs to be labeled as one and held to the company's written policies.

    arstechnica.com

    Credit

    Ars Technica, which reported it

Watch for

  1. Half of support by agents

    Salesforce's chief executive, Marc Benioff, said AI agents now handle about half of Salesforce's customer support conversations, and that he had cut support staff "from 9,000 heads to about 5,000". A Salesforce spokesperson said the company had stopped refilling support engineer roles.

    Achieved whenSalesforce or an outside party publishes how many of the agent-handled conversations solved the customer's problem, measured the same way as before the change.

    yahoo.com

    Claimed by

    Marc Benioff, Salesforce, on The Logan Bartlett Show, reported September 2025

  2. Three in four resolved

    Fin, the AI support agent sold by Intercom, is said to have an average resolution rate of 76% across more than 12,000 customers, with 2 million resolutions a week. By Intercom's own definition, a conversation counts as resolved when the customer confirms the answer helped or leaves without asking for more help.

    Achieved whenan outside party reads a random sample of conversations counted as resolved and reports how many really solved the customer's problem.

    fin.aiintercom.com

    Claimed by

    Intercom, on fin.ai, 2026

Open

  1. 01

    Support bots against their terms

    A public test of the AI support bots run by 30 software companies. For each company, write 15 questions whose answers are stated in its own published terms or help pages (refund window, number of devices or seats, how to cancel, how to export data). Ask its public bot each question and grade the reply as correct, wrong or invented. Publish every transcript beside the sentence of policy it was graded against.

    The brief
    Credit · unclaimed
  2. 02

    Chat widgets with screen readers

    An independent test of the support chat widgets that software companies put on their websites, tried with the screen readers that blind people use, which read the screen aloud. Each widget gets the same six tasks: find and open it, hear that it opened, send a question, hear the reply arrive, ask for a person, and close it. Publish a pass or fail for every widget, screen reader and task, with a screen and audio recording of each.

    The brief
    Credit · unclaimed

Open 01 · the brief

Support bots against their terms

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

A public test of the AI support bots run by 30 software companies. For each company, write 15 questions whose answers are stated in its own published terms or help pages (refund window, number of devices or seats, how to cancel, how to export data). Ask its public bot each question and grade the reply as correct, wrong or invented. Publish every transcript beside the sentence of policy it was graded against.

Why it is still open

Cursor's "Sam" and Air Canada's chatbot both stated policies that did not exist. In September 2026 NBC Chicago reported that "there are no specific studies measuring the accuracy of AI customer service chatbots". Benchmarks such as τ-bench, from the support-agent company Sierra, test AI agents against simulated customers and made-up policies, not the bots companies run.

nbcchicago.comarxiv.org

Built with what exists

a spreadsheet of questions, each with the policy sentence and its web address; the companies' public chat windows, used by hand at a normal pace and within each site's terms; a language model such as Claude or Gemini as a first-pass grader that compares each reply with the policy text (see the Note "Seven more flagged"), with a person checking every reply graded wrong or invented.

  1. First step

    Pick five software companies whose support pages open with an AI bot, write 15 questions from each one's refund and cancellation terms, and ask them by hand.

  2. Done looks like

    450 graded replies from 30 companies, each with its transcript, date and policy text. A second person regrades 50 at random and agrees on at least 45. Each company is sent its results 14 days before they are published.

Open 02 · the brief

Chat widgets with screen readers

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

An independent test of the support chat widgets that software companies put on their websites, tried with the screen readers that blind people use, which read the screen aloud. Each widget gets the same six tasks: find and open it, hear that it opened, send a question, hear the reply arrive, ask for a person, and close it. Publish a pass or fail for every widget, screen reader and task, with a screen and audio recording of each.

Why it is still open

Widget makers publish their own conformance reports. An article on Silktide's blog, by a firm that sells accessibility testing, said in July 2026 that "there aren't many published studies yet that test chatbot accessibility". The academic study found, by Leporini, Buzzi and Della Penna in 2025, tested ChatGPT, Copilot, Perplexity and Gemini, not support widgets, and found problems in all four.

silktide.comricerca.univaq.it

Built with what exists

free or trial accounts for eight widgets installed on a test page you own; the free NVDA screen reader on Windows and VoiceOver on a Mac and an iPhone; Accessibility Insights or axe DevTools for automated checks; WCAG 2.2, the web's accessibility standard, and MITRE's Chatbot Accessibility Playbook as the yardsticks; OBS Studio for recordings.

  1. First step

    Put two widgets on a blank test page and run the six tasks with NVDA, recording screen and sound.

  2. Done looks like

    8 widgets, 3 screen readers and 6 tasks, 144 results in all, each with a recording. A blind screen-reader user repeats 36 of them chosen at random, and the published table notes every result where the two disagree. Makers get 30 days' notice before publication.