Industries / Finance and business services / Call centers and BPO
Quality
Quality work checks how well each contact was handled: recording calls and chats, scoring some or all of them against a checklist, searching recordings for patterns, keeping card numbers and personal data out of what is stored, and asking customers what they thought.
7
Milestones achieved, from 1996 to 2024.
3
To watch for: announced, not yet confirmed outside.
2
Open deliverables. Nobody has shipped them.
The road
AchievedWatch forOpen
Achieved
1996
Buyers write a center standard
By COPC's own account, large buyers of outsourced customer service, including Microsoft, Dell, American Express, Motorola, Adobe and Intel, agreed on shared metrics and performance requirements for call centers, which became the COPC standard (now the COPC CX Standard); Customer Operations Performance Center Inc. was founded in 1996 to audit and certify centers against it. By May 2018 the outsourcer Concentrix held the certification at 20 sites in ten countries, among them the Philippines, India, Brazil, Nicaragua and Portugal.
Credit
COPC Inc. and its founding buyer companies
2003
One question for loyalty
In the December 2003 Harvard Business Review, Fred Reichheld of Bain & Company wrote that companies were "measuring the wrong thing" with complex satisfaction surveys and proposed what became the Net Promoter Score: customers answer "How likely are you to recommend us to a friend or colleague?" from 0 to 10, and the score is the percentage answering 9 or 10 minus the percentage answering 0 to 6. In July 2007 the Journal of Marketing published a study by Timothy Keiningham, Bruce Cooil, Tor Wallin Andreassen and Lerzan Aksoy that found "no support" for the claim that the score predicts company growth better than other measures.
LessonA score that is simple to collect still has to be tested against later results before it is trusted to predict them.
Credit
Frederick Reichheld (Bain & Company); Keiningham and co-authors for the challenge
2010
Speech analytics spreads slowly
Speech analytics software makes recorded calls searchable, so a center can look across all its calls instead of the few a supervisor hears. The analyst firm DMG Consulting counted growth in installations of 39% in 2006, 50% in 2007 and 106% in 2008, slowing to 22% between 2009 and 2010, and put the share of contact centers using it at 11.6% at the end of 2010; it estimated that for every successful project there were at least twice as many used mainly for reports, without the expected return.
LessonBeing able to search every call changes nothing until someone acts on what the search finds.
Credit
DMG Consulting (Donna Fluss) for the count; no single vendor is credited
2011
Card codes out of recordings
In March 2011 the PCI Security Standards Council issued guidance on telephone payments and call recordings, revised in November 2018: the card security code must not be stored after authorization, even encrypted, and a recording that captures it must be deleted or made unrecoverable. In "pause-and-resume" the agent stops the recorder while the caller reads out the card, and because agents forget, the guidance recommends checking recordings for card data, preferably weekly; in "DTMF masking" the caller types the card on the keypad and the tones are replaced with flat sounds, though the start of a tone can slip through ("DTMF bleed").
LessonA safeguard that depends on an agent pressing a button on every call will miss some calls, and the guidance says manual methods need "much greater" oversight than automatic ones.
Credit
PCI Security Standards Council, Protecting Telephone-Based Payments Special Interest Group
2017
A world contact center standard
ISO 18295-1:2017 sets service requirements for customer contact centers, in-house or outsourced, of any size and in every sector and channel, and specifies performance measures (KPIs, key performance indicators) where needed; with its second part, ISO 18295-2, it replaced the European standard EN 15838:2009, which was withdrawn on 2 August 2017. A revision of the European edition was approved as a new project on 18 April 2026.
Credit
ISO and CEN (European Committee for Standardization)
2019
A notice is not consent
On 11 April 2019 Denmark's data protection authority ruled that TDC A/S, the country's largest telecom company, broke the EU General Data Protection Regulation (GDPR) by telling callers only that "calls may be recorded for training purposes" with no way to opt in or out; it rejected better customer service as a legitimate interest and required affirmative consent. In April 2024 it relaxed its position, allowing recording for training without consent if the caller can refuse by a keypress before or during the call.
LessonA recorded announcement is not a choice; callers need a working way to say no.
Credit
Datatilsynet (Danish Data Protection Agency)
2024
Satisfaction hits nine-year low
The UK Customer Satisfaction Index stood at 76.0 out of 100 in January 2024, its lowest level since 2015, down from 78.4 in January 2022, and complaint handling fell further than any other measure. The Institute of Customer Service's chief executive, Jo Causon, had linked the fall to longer resolution times, as firms struggled to recruit and keep staff, and to new technology not yet "properly embedded and tested".
LessonCutting staff before the replacement technology works shows up in what customers report, most of all in how complaints are handled.
instituteofcustomerservice.cominstituteofcustomerservice.com
Credit
The Institute of Customer Service
Watch for
AI scorers against human labels
Researchers at Observe.AI, which sells automatic quality scoring ("auto-QA"), tested 18 large language models on 3,000 real contact center transcripts with 30 yes/no quality questions labelled by three trained in-house annotators; accuracy against the majority label ranged from 65.94% (Llama 3.2 3B) to 93.84% (Amazon Nova Premier), and changing details that should not matter, such as identity cues or speaking style, flipped 5.4% to 13.0% of answers. The paper does not report how often the annotators agreed with each other, and the data is proprietary.
Achieved whenan independent group runs the test on a public set of calls, reports human-to-human agreement as the baseline, and finds AI-to-human agreement as high.
Claimed by
Kawin Mayilvaghanan, Siddhant Gupta and Ayush Kumar (Observe.AI), February 2026
Nearly every call bot-scored
Verint says an unnamed North American industrial-equipment distributor used its Quality Bot to evaluate 97% of more than 200,000 calls a month, raising supervisor capacity 80% and saving $12.5 million a year; the release says nothing about how well the bot's scores match human evaluators.
Achieved whenthe customer or an outside auditor publishes how often the bot's scores match trained human evaluators on a blind sample of calls.
Claimed by
Verint, May 2026
Small models for QA
Level AI announced Latitude, seven models built for customer service work, two of which score each interaction against a company's own rubric and redact sensitive data in transcripts and audio. It claims "accuracy at par" with frontier models at up to 49 times lower cost and 96% accuracy on key entities for redaction, from its own production workloads, naming no public benchmark.
Achieved whensomeone other than Level AI scores the models on a public benchmark against human graders and against frontier models.
Claimed by
Level AI, July 2026
Open
01
A public AI scoring test
300 synthetic customer service calls (billing disputes, cancellations, address changes, complaints), some with errors planted on purpose, written as scripts and voiced by consenting volunteers or text-to-speech. At least three experienced QA analysts each score every call independently on a published 25-item yes/no scorecard; the audio, transcripts, scorecard and every individual score are published, and several AI auto-QA systems are run on the same calls.
The briefCredit · unclaimed02
Spoken card-number leak test
1,000 synthetic payment-call clips in at least five languages, with several accents in each, in which callers read made-up card numbers, expiry dates and security codes the way people really do (in groups, with corrections, saying "double four", mixed with small talk, names and addresses). Run each clip through speech-to-text and redaction pipelines and count what survives in the stored transcript.
The briefCredit · unclaimed
Open 01 · the brief
A public AI scoring test
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
300 synthetic customer service calls (billing disputes, cancellations, address changes, complaints), some with errors planted on purpose, written as scripts and voiced by consenting volunteers or text-to-speech. At least three experienced QA analysts each score every call independently on a published 25-item yes/no scorecard; the audio, transcripts, scorecard and every individual score are published, and several AI auto-QA systems are run on the same calls.
Why it is still open
Public call center data carries no human quality scores. CallCenterEN holds 91,706 real transcripts with no quality scores and withholds the audio; the scored set SITCC has 9,081 sentences written by GPT-4 and scored by a "simulated call center manager", not by people; Observe.AI's 3,000 scored transcripts are proprietary and its paper reports no human-to-human agreement.
Built with what exists
Kokoro-82M or Chatterbox text-to-speech with built-in voices or voices from consenting volunteers; transcripts from ElevenLabs Scribe v3 (see the Note "Fewer word errors"); Gemini Decision 1 to answer all 25 scorecard items for a call in one pass (see the Note "A thousand questions, one pass"); an open-weights scorer such as Mistral Large 4 for comparison (see the Note "A trillion open parameters"); Cohen's kappa, a statistic that discounts the agreement two graders would reach by chance.
First step
Write the scorecard and 20 pilot scripts with planted errors, have three QA analysts score them, and compute human-to-human kappa to find the items humans themselves cannot agree on.
Done looks like
300 calls with at least three human scores each, released under an open license; a table of per-item kappa, human against human and AI against human, for each AI system; and a count of how many of the 25 items reach AI-to-human agreement at least as high as human-to-human agreement.
Open 02 · the brief
Spoken card-number leak test
The first to publish is credited on this line, and the deliverable moves from Open to Achieved.
What to build
1,000 synthetic payment-call clips in at least five languages, with several accents in each, in which callers read made-up card numbers, expiry dates and security codes the way people really do (in groups, with corrections, saying "double four", mixed with small talk, names and addresses). Run each clip through speech-to-text and redaction pipelines and count what survives in the stored transcript.
Why it is still open
The nearest benchmark, VoicePII-Bench-EN, is English only, with 545 clips of one sensitive value each, unverified accent labels and its full release waiting on paper acceptance. A June 2026 survey of redaction benchmarks found no "open, multilingual, ASR-noise-aware benchmark" and no harness that tests the chain from audio to speech-to-text to redaction. PCI guidance says pause-and-resume fails when agents forget and that keypad masking can leak the start of tones.
Built with what exists
Kokoro-82M and Chatterbox text-to-speech; ElevenLabs Scribe v3 (see the Note "Fewer word errors"), NVIDIA Parakeet TDT v3 and Cohere Transcribe for speech-to-text; Microsoft's open-source Presidio, GLiNER-PII and OpenPipe PII-Redact for redaction; a small fine-tuned extractor such as Meta Muse Ember (see the Note "The jobs agents repeat") or a Liquid LFM3 model tuned on one GPU (see the Note "One GPU to fine-tune").
First step
Generate 100 English clips in three accents with free text-to-speech models, run two pipelines on them, and publish the leak tally.
Done looks like
Clips, labels and scoring code published, with, for each pipeline and language, the share of clips in which four or more consecutive card digits, or any security code, remain readable in the transcript.