BRENNEN

Industries / Finance and business services / Call centers and BPO

Quality

Quality work checks how well each contact was handled: recording calls and chats, scoring some or all of them against a checklist, searching recordings for patterns, keeping card numbers and personal data out of what is stored, and asking customers what they thought.

7

Milestones achieved, from 1996 to 2024.

3

To watch for: announced, not yet confirmed outside.

2

Open deliverables. Nobody has shipped them.

The road

AchievedWatch forOpen

Achieved

  1. 1996

    Buyers write a center standard

    By COPC's own account, large buyers of outsourced customer service, including Microsoft, Dell, American Express, Motorola, Adobe and Intel, agreed on shared metrics and performance requirements for call centers, which became the COPC standard (now the COPC CX Standard); Customer Operations Performance Center Inc. was founded in 1996 to audit and certify centers against it. By May 2018 the outsourcer Concentrix held the certification at 20 sites in ten countries, among them the Philippines, India, Brazil, Nicaragua and Portugal.

    copc.comcopc.comcopc.com

    Credit

    COPC Inc. and its founding buyer companies

  2. 2003

    One question for loyalty

    In the December 2003 Harvard Business Review, Fred Reichheld of Bain & Company wrote that companies were "measuring the wrong thing" with complex satisfaction surveys and proposed what became the Net Promoter Score: customers answer "How likely are you to recommend us to a friend or colleague?" from 0 to 10, and the score is the percentage answering 9 or 10 minus the percentage answering 0 to 6. In July 2007 the Journal of Marketing published a study by Timothy Keiningham, Bruce Cooil, Tor Wallin Andreassen and Lerzan Aksoy that found "no support" for the claim that the score predicts company growth better than other measures.

    LessonA score that is simple to collect still has to be tested against later results before it is trusted to predict them.

    hbr.orgnetpromotersystem.comipsos.com

    Credit

    Frederick Reichheld (Bain & Company); Keiningham and co-authors for the challenge

  3. 2010

    Speech analytics spreads slowly

    Speech analytics software makes recorded calls searchable, so a center can look across all its calls instead of the few a supervisor hears. The analyst firm DMG Consulting counted growth in installations of 39% in 2006, 50% in 2007 and 106% in 2008, slowing to 22% between 2009 and 2010, and put the share of contact centers using it at 11.6% at the end of 2010; it estimated that for every successful project there were at least twice as many used mainly for reports, without the expected return.

    LessonBeing able to search every call changes nothing until someone acts on what the search finds.

    dmgconsult.com

    Credit

    DMG Consulting (Donna Fluss) for the count; no single vendor is credited

  4. 2011

    Card codes out of recordings

    In March 2011 the PCI Security Standards Council issued guidance on telephone payments and call recordings, revised in November 2018: the card security code must not be stored after authorization, even encrypted, and a recording that captures it must be deleted or made unrecoverable. In "pause-and-resume" the agent stops the recorder while the caller reads out the card, and because agents forget, the guidance recommends checking recordings for card data, preferably weekly; in "DTMF masking" the caller types the card on the keypad and the tones are replaced with flat sounds, though the start of a tone can slip through ("DTMF bleed").

    LessonA safeguard that depends on an agent pressing a button on every call will miss some calls, and the guidance says manual methods need "much greater" oversight than automatic ones.

    listings.pcisecuritystandards.org

    Credit

    PCI Security Standards Council, Protecting Telephone-Based Payments Special Interest Group

  5. 2017

    A world contact center standard

    ISO 18295-1:2017 sets service requirements for customer contact centers, in-house or outsourced, of any size and in every sector and channel, and specifies performance measures (KPIs, key performance indicators) where needed; with its second part, ISO 18295-2, it replaced the European standard EN 15838:2009, which was withdrawn on 2 August 2017. A revision of the European edition was approved as a new project on 18 April 2026.

    genorma.comgenorma.comdinmedia.degenorma.com

    Credit

    ISO and CEN (European Committee for Standardization)

  6. 2019

    A notice is not consent

    On 11 April 2019 Denmark's data protection authority ruled that TDC A/S, the country's largest telecom company, broke the EU General Data Protection Regulation (GDPR) by telling callers only that "calls may be recorded for training purposes" with no way to opt in or out; it rejected better customer service as a legitimate interest and required affirmative consent. In April 2024 it relaxed its position, allowing recording for training without consent if the caller can refuse by a keypress before or during the call.

    LessonA recorded announcement is not a choice; callers need a working way to say no.

    natlawreview.comdanskerhverv.dk

    Credit

    Datatilsynet (Danish Data Protection Agency)

  7. 2024

    Satisfaction hits nine-year low

    The UK Customer Satisfaction Index stood at 76.0 out of 100 in January 2024, its lowest level since 2015, down from 78.4 in January 2022, and complaint handling fell further than any other measure. The Institute of Customer Service's chief executive, Jo Causon, had linked the fall to longer resolution times, as firms struggled to recruit and keep staff, and to new technology not yet "properly embedded and tested".

    LessonCutting staff before the replacement technology works shows up in what customers report, most of all in how complaints are handled.

    instituteofcustomerservice.cominstituteofcustomerservice.com

    Credit

    The Institute of Customer Service

Watch for

  1. AI scorers against human labels

    Researchers at Observe.AI, which sells automatic quality scoring ("auto-QA"), tested 18 large language models on 3,000 real contact center transcripts with 30 yes/no quality questions labelled by three trained in-house annotators; accuracy against the majority label ranged from 65.94% (Llama 3.2 3B) to 93.84% (Amazon Nova Premier), and changing details that should not matter, such as identity cues or speaking style, flipped 5.4% to 13.0% of answers. The paper does not report how often the annotators agreed with each other, and the data is proprietary.

    Achieved whenan independent group runs the test on a public set of calls, reports human-to-human agreement as the baseline, and finds AI-to-human agreement as high.

    arxiv.org

    Claimed by

    Kawin Mayilvaghanan, Siddhant Gupta and Ayush Kumar (Observe.AI), February 2026

  2. Nearly every call bot-scored

    Verint says an unnamed North American industrial-equipment distributor used its Quality Bot to evaluate 97% of more than 200,000 calls a month, raising supervisor capacity 80% and saving $12.5 million a year; the release says nothing about how well the bot's scores match human evaluators.

    Achieved whenthe customer or an outside auditor publishes how often the bot's scores match trained human evaluators on a blind sample of calls.

    verint.com

    Claimed by

    Verint, May 2026

  3. Small models for QA

    Level AI announced Latitude, seven models built for customer service work, two of which score each interaction against a company's own rubric and redact sensitive data in transcripts and audio. It claims "accuracy at par" with frontier models at up to 49 times lower cost and 96% accuracy on key entities for redaction, from its own production workloads, naming no public benchmark.

    Achieved whensomeone other than Level AI scores the models on a public benchmark against human graders and against frontier models.

    thelevel.ai

    Claimed by

    Level AI, July 2026

Open

  1. 01

    A public AI scoring test

    300 synthetic customer service calls (billing disputes, cancellations, address changes, complaints), some with errors planted on purpose, written as scripts and voiced by consenting volunteers or text-to-speech. At least three experienced QA analysts each score every call independently on a published 25-item yes/no scorecard; the audio, transcripts, scorecard and every individual score are published, and several AI auto-QA systems are run on the same calls.

    The brief
    Credit · unclaimed
  2. 02

    Spoken card-number leak test

    1,000 synthetic payment-call clips in at least five languages, with several accents in each, in which callers read made-up card numbers, expiry dates and security codes the way people really do (in groups, with corrections, saying "double four", mixed with small talk, names and addresses). Run each clip through speech-to-text and redaction pipelines and count what survives in the stored transcript.

    The brief
    Credit · unclaimed

Open 01 · the brief

A public AI scoring test

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

300 synthetic customer service calls (billing disputes, cancellations, address changes, complaints), some with errors planted on purpose, written as scripts and voiced by consenting volunteers or text-to-speech. At least three experienced QA analysts each score every call independently on a published 25-item yes/no scorecard; the audio, transcripts, scorecard and every individual score are published, and several AI auto-QA systems are run on the same calls.

Why it is still open

Public call center data carries no human quality scores. CallCenterEN holds 91,706 real transcripts with no quality scores and withholds the audio; the scored set SITCC has 9,081 sentences written by GPT-4 and scored by a "simulated call center manager", not by people; Observe.AI's 3,000 scored transcripts are proprietary and its paper reports no human-to-human agreement.

arxiv.orghuggingface.coarxiv.org

Built with what exists

Kokoro-82M or Chatterbox text-to-speech with built-in voices or voices from consenting volunteers; transcripts from ElevenLabs Scribe v3 (see the Note "Fewer word errors"); Gemini Decision 1 to answer all 25 scorecard items for a call in one pass (see the Note "A thousand questions, one pass"); an open-weights scorer such as Mistral Large 4 for comparison (see the Note "A trillion open parameters"); Cohen's kappa, a statistic that discounts the agreement two graders would reach by chance.

  1. First step

    Write the scorecard and 20 pilot scripts with planted errors, have three QA analysts score them, and compute human-to-human kappa to find the items humans themselves cannot agree on.

  2. Done looks like

    300 calls with at least three human scores each, released under an open license; a table of per-item kappa, human against human and AI against human, for each AI system; and a count of how many of the 25 items reach AI-to-human agreement at least as high as human-to-human agreement.

Open 02 · the brief

Spoken card-number leak test

Credit · unclaimed

The first to publish is credited on this line, and the deliverable moves from Open to Achieved.

What to build

1,000 synthetic payment-call clips in at least five languages, with several accents in each, in which callers read made-up card numbers, expiry dates and security codes the way people really do (in groups, with corrections, saying "double four", mixed with small talk, names and addresses). Run each clip through speech-to-text and redaction pipelines and count what survives in the stored transcript.

Why it is still open

The nearest benchmark, VoicePII-Bench-EN, is English only, with 545 clips of one sensitive value each, unverified accent labels and its full release waiting on paper acceptance. A June 2026 survey of redaction benchmarks found no "open, multilingual, ASR-noise-aware benchmark" and no harness that tests the chain from audio to speech-to-text to redaction. PCI guidance says pause-and-resume fails when agents forget and that keypad masking can leak the start of tones.

github.comgist.github.comlistings.pcisecuritystandards.org

Built with what exists

Kokoro-82M and Chatterbox text-to-speech; ElevenLabs Scribe v3 (see the Note "Fewer word errors"), NVIDIA Parakeet TDT v3 and Cohere Transcribe for speech-to-text; Microsoft's open-source Presidio, GLiNER-PII and OpenPipe PII-Redact for redaction; a small fine-tuned extractor such as Meta Muse Ember (see the Note "The jobs agents repeat") or a Liquid LFM3 model tuned on one GPU (see the Note "One GPU to fine-tune").

  1. First step

    Generate 100 English clips in three accents with free text-to-speech models, run two pipelines on them, and publish the leak tally.

  2. Done looks like

    Clips, labels and scoring code published, with, for each pipeline and language, the share of clips in which four or more consecutive card digits, or any security code, remain readable in the transcript.