Industries / Technology / Software and SaaS
Customer support
Answering customers' questions and fixing their problems after they buy, by staff, through help pages and public forums, and now by AI agents.
7
Milestones achieved, from 1955 to 2025.
2
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1955
Users form SHARE
Users of IBM's 704 computer in the Los Angeles area set up SHARE, the Society to Help Avoid Redundant Effort, to swap the programs and manuals they wrote and to deal with IBM as one group. Forty-six people from seventeen 704 installations came to the first meeting, in the RAND Corporation's basement in the week of 22 August 1955. The Smithsonian's archive calls SHARE the first organized user group in the computer industry.
Credit
Jack Strong and Frank Wagner of North American Aviation, Lee Amaya of Lockheed and Paul Armer of RAND, who phoned the other 704 customers from a booth in Los Angeles
2001
Clippy switched off
Microsoft Office 97 came with an animated helper, the Office Assistant, best known as a paper clip called Clippy, that offered help unasked with lines such as "It looks like you're writing a letter". On 11 April 2001 Microsoft said the assistant would be off by default in Office XP, and from Office 2007 it could not be called up at all.
LessonHelp that interrupts people who did not ask for it gets switched off.
Credit
Microsoft
2008
Answers kept in public
Stack Overflow opened to the public in September 2008 as a free site where programmers ask questions, others answer, and votes move the best answers to the top. Each question is kept as one page and repeats are pointed to the original, so the answers build up into a public reference. Good answers earn points, called reputation, that let a user edit other people's posts.
Credit
Jeff Atwood and Joel Spolsky
2023
AI help for support staff
Economists followed 5,172 support agents at a Fortune 500 firm that sells business software as it brought in an AI assistant that suggests replies during chats. With the assistant, agents resolved 15% more issues per hour on average. The least experienced agents gained the most in speed and quality, and the most experienced saw small gains in speed and small drops in quality.
Credit
Erik Brynjolfsson, Danielle Li and Lindsey Raymond, published in the Quarterly Journal of Economics in 2025
2024
The bot's word binds
Air Canada's website chatbot told a grieving customer that a reduced bereavement fare could be claimed after the trip, which the airline's written policy did not allow. Air Canada argued the chatbot was in effect a separate legal entity responsible for its own actions. On 14 February 2024 British Columbia's Civil Resolution Tribunal called that "a remarkable submission", found negligent misrepresentation and awarded the customer CA$650.88.
LessonA company answers for what its chatbot tells customers, as it does for any other page on its website.
Credit
Jake Moffatt, who brought the claim; tribunal member Christopher C. Rivers
2024
Fewer questions in public
Researchers compared Stack Overflow with similar question-and-answer sites where ChatGPT was harder to use or less useful. Within six months of ChatGPT's release in November 2022, activity on Stack Overflow fell by 25% relative to those sites.
LessonWhen people take their questions to a private AI chat, fewer answers are left in public for the next person, or for the next AI model, to learn from.
Credit
R. Maria del Rio-Chanona, Nadzeya Laurentsyeva and Johannes Wachs, in PNAS Nexus
2025
A support bot invents policy
In April 2025 a user of Cursor, an AI code editor, asked support why switching computers logged them out. A reply signed "Sam" said Cursor was "designed to work with one device per subscription as a core security feature". No such policy existed, Sam was an AI model, and users posted that they were canceling. Cursor co-founder Michael Truell apologized, said the logouts came from a security change, and said AI replies to support email would now be labeled as such.
LessonAn AI agent that answers customers needs to be labeled as one and held to the company's written policies.
Credit
Ars Technica, which reported it
Watch for
Half of support by agents
Salesforce's chief executive, Marc Benioff, said AI agents now handle about half of Salesforce's customer support conversations, and that he had cut support staff "from 9,000 heads to about 5,000". A Salesforce spokesperson said the company had stopped refilling support engineer roles.
Achieved whenSalesforce or an outside party publishes how many of the agent-handled conversations solved the customer's problem, measured the same way as before the change.
Claimed by
Marc Benioff, Salesforce, on The Logan Bartlett Show, reported September 2025
Three in four resolved
Fin, the AI support agent sold by Intercom, is said to have an average resolution rate of 76% across more than 12,000 customers, with 2 million resolutions a week. By Intercom's own definition, a conversation counts as resolved when the customer confirms the answer helped or leaves without asking for more help.
Achieved whenan outside party reads a random sample of conversations counted as resolved and reports how many really solved the customer's problem.
Claimed by
Intercom, on fin.ai, 2026
Open
01
Support bots against their terms
A public test of the AI support bots run by 30 software companies. For each company, write 15 questions whose answers are stated in its own published terms or help pages (refund window, number of devices or seats, how to cancel, how to export data). Ask its public bot each question and grade the reply as correct, wrong or invented. Publish every transcript beside the sentence of policy it was graded against.
The briefCredit · unclaimed02
Chat widgets with screen readers
An independent test of the support chat widgets that software companies put on their websites, tried with the screen readers that blind people use, which read the screen aloud. Each widget gets the same six tasks: find and open it, hear that it opened, send a question, hear the reply arrive, ask for a person, and close it. Publish a pass or fail for every widget, screen reader and task, with a screen and audio recording of each.
The briefCredit · unclaimed
Open 01 · the brief
Support bots against their terms
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
A public test of the AI support bots run by 30 software companies. For each company, write 15 questions whose answers are stated in its own published terms or help pages (refund window, number of devices or seats, how to cancel, how to export data). Ask its public bot each question and grade the reply as correct, wrong or invented. Publish every transcript beside the sentence of policy it was graded against.
Why it is still open
Cursor's "Sam" and Air Canada's chatbot both stated policies that did not exist. In September 2026 NBC Chicago reported that "there are no specific studies measuring the accuracy of AI customer service chatbots". Benchmarks such as τ-bench, from the support-agent company Sierra, test AI agents against simulated customers and made-up policies, not the bots companies run.
Built with what exists
a spreadsheet of questions, each with the policy sentence and its web address; the companies' public chat windows, used by hand at a normal pace and within each site's terms; a language model such as Claude or Gemini as a first-pass grader that compares each reply with the policy text (see the Note "Seven more flagged"), with a person checking every reply graded wrong or invented.
First step
Pick five software companies whose support pages open with an AI bot, write 15 questions from each one's refund and cancellation terms, and ask them by hand.
Done looks like
450 graded replies from 30 companies, each with its transcript, date and policy text. A second person regrades 50 at random and agrees on at least 45. Each company is sent its results 14 days before they are published.
Open 02 · the brief
Chat widgets with screen readers
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
An independent test of the support chat widgets that software companies put on their websites, tried with the screen readers that blind people use, which read the screen aloud. Each widget gets the same six tasks: find and open it, hear that it opened, send a question, hear the reply arrive, ask for a person, and close it. Publish a pass or fail for every widget, screen reader and task, with a screen and audio recording of each.
Why it is still open
Widget makers publish their own conformance reports. An article on Silktide's blog, by a firm that sells accessibility testing, said in July 2026 that "there aren't many published studies yet that test chatbot accessibility". The academic study found, by Leporini, Buzzi and Della Penna in 2025, tested ChatGPT, Copilot, Perplexity and Gemini, not support widgets, and found problems in all four.
Built with what exists
free or trial accounts for eight widgets installed on a test page you own; the free NVDA screen reader on Windows and VoiceOver on a Mac and an iPhone; Accessibility Insights or axe DevTools for automated checks; WCAG 2.2, the web's accessibility standard, and MITRE's Chatbot Accessibility Playbook as the yardsticks; OBS Studio for recordings.
First step
Put two widgets on a blank test page and run the six tasks with NVDA, recording screen and sound.
Done looks like
8 widgets, 3 screen readers and 6 tasks, 144 results in all, each with a recording. A blind screen-reader user repeats 36 of them chosen at random, and the published table notes every result where the two disagree. Makers get 30 days' notice before publication.