V4N

Quality

How to check every call, not 1%: scorecards and speech AI for call-centre QA

By the V4N team · Published · 3 min read

The short answer

Manual call QA usually reviews well under 1% of calls, so rare but serious problems, like a missed disclosure, go unseen. Speech AI can transcribe and score every call against your own scorecard, so reviewers spend their time on flagged calls, disputes and coaching. The AI should flag and a person should decide, and you should test its accuracy on your own calls, languages and accents before relying on it.

The arithmetic of sampling

Take a team of 20 agents, each handling 40 calls a day, over 22 working days. That is 17,600 calls a month. If a reviewer scores 5 calls per agent per month, they check 100 calls: 0.57%. At ten minutes each, that is about 17 hours of work to look at fewer than six calls in a thousand. These numbers are illustrative, but most teams' sums come out the same way.

The bigger issue is what sampling cannot see. Suppose one agent skips a required disclosure on 2% of calls. The chance that 5 random calls contain none of them is 0.98⁵, about 90%. The reviewer sees nothing wrong, month after month, while the problem continues.

Start with a scorecard people can apply

Automated scoring is only as good as the scorecard behind it, and so is human scoring. A good one is short and unambiguous:

  • 8 to 12 criteria. More than that and reviewers drift and agents stop reading it.
  • Yes or no wherever you can. "Gave the required disclosure" is checkable. "Showed empathy" is where two reviewers disagree.
  • Critical items that fail the whole call, such as skipping identity verification or a mandatory statement, whatever else went well.
  • Weights for the rest, so greeting and closing do not outweigh resolving the problem.
  • Written examples of a pass and a fail for every criterion.

What speech AI does well

  • Transcribes every recorded call, so any call can be searched by what was said.
  • Checks for required phrases and prohibited ones across all calls, not a sample.
  • Summarises the call in a few lines, so a reviewer does not need to listen to the whole recording.
  • Scores against your scorecard and quotes the moment each score is based on, so a person can check it in seconds.

Where a person must decide

  • Accuracy varies with language and accent. Callers who switch between English, Swahili and Sheng within a sentence are harder to transcribe than a scripted English call. Test any tool on a few hundred of your own calls and measure how often it is right before trusting a score.
  • Tone and context are often misread. A flagged call is a prompt to listen, not a verdict.
  • Scores that affect pay or discipline should always be reviewable. Give agents a way to dispute a score and have a person look at it.

A workable review process

  1. AI scores every call against the scorecard.
  2. People review all calls with a failed critical item, a low score or an agent dispute.
  3. Add a small random sample of passed calls each week, to check the AI is not missing things.
  4. Turn findings into coaching items for specific agents, with the call attached.
  5. Each month, compare AI and human scores on the same calls and adjust the scorecard.

Where recordings are sent for transcription, they are personal data. See our note on recording calls under Kenya's Data Protection Act before you switch it on.

Common questions

How many calls should be reviewed for quality?

As many as you can. Manual sampling typically covers well under 1% of calls, which can miss problems that occur on a few percent of calls. Automated scoring of every call plus human review of flagged calls gives far better coverage.

Can AI replace human call reviewers?

No. It can score and flag every call, but a person should decide on anything that affects an agent's pay or discipline, and should check a sample to confirm the AI is accurate.

Does speech AI work with Swahili and mixed-language calls?

Accuracy varies by service, accent and how much languages are mixed. Test on a few hundred of your own recorded calls and measure the error rate before relying on it.

What makes a good call scorecard?

Eight to twelve criteria, mostly yes or no, with critical items that fail the call, weighted scoring for the rest, and written examples of a pass and a fail.

Every call seen, sorted and answered

See V4N on a setup like yours

A short call with our team. We look at your phone system and your team, then show you V4N working the way you would use it.