Home › Topics › VoIP & Call-Quality Monitoring › Call-Quality Program

Building a Call-Quality Monitoring Program


Somewhere in your phone system, right now, there is a record of every call made today. It records when the call started, how long it lasted, and which trunk it used. Very often it also records how it sounded — packets lost, jitter, delay, a quality score. Almost nobody looks at it. Complaints arrive vague and late ("calls were bad last week, especially with the warehouse"). The data that could answer them sits in a table nobody opens.


A call-quality monitoring program is the habit of collecting that data on purpose, knowing what normal looks like, and noticing degradation before users do. It does not require expensive tooling. It requires four data sources, a baseline for each site and trunk, a small dashboard, thresholds that fire on real change, and half an hour a week.


This guide builds that program step by step. If you have not read What Makes VoIP Different, the essentials are that voice cannot retransmit and that four numbers describe how a call sounded. They are loss, jitter, latency, and a quality score called MOS (Mean Opinion Score, 1 to 5). Everything below is about collecting those numbers and acting on them.


In this guide

  • Where quality data comes from
  • Baselining per site and per trunk
  • Dashboards: trend, worst calls, worst sites
  • Thresholds that catch degradation before users complain
  • The weekly worst-calls review
  • Correlating bad-call clusters with network events
  • The program-maturity ladder
  • Where to go next

Where quality data comes from


Four sources feed a quality program, and each sees something the others cannot. Start with whichever you already have; a mature program uses all four.


Per-call quality records from the PBX

Every phone system writes a CDR (call detail record) per call: start time, caller, callee, duration, trunk, outcome. Many also write quality fields, either in the same record or in a companion record. These are packets sent and lost, jitter, round-trip delay, codec, and a MOS estimate, usually per leg and per direction. This is the backbone of the program because it covers every real call. Its limits: the numbers arrive only after the call ends. For internal calls whose media went phone to phone the PBX may only know what the phones reported to it. Keep the records for a few months at least; the question "has the warehouse always been like this?" can only be answered with history.


Phone-reported statistics

Each phone measures the stream it receives. It shares those measurements with the other end through RTCP (the control protocol that travels alongside the audio). The phone often sends a summary to the PBX or a collector when the call ends. Many phones also show live statistics on their screen or web page and can send them as syslog. This is the closest thing to the user's actual experience, per direction. And it is the source that tells you "the far end heard loss even though we heard none."


Synthetic voice probes between sites

A synthetic voice probe is a simulated call. Two agents exchange a voice-like stream of packets every few minutes, marked exactly as real voice is. The agents are small pieces of software at each site, or a feature built into many routers. They measure loss, jitter, and latency in both directions, usually producing a MOS estimate as well. Probes run continuously, including at three in the morning when nobody is calling, and they test a path under controlled conditions. Their limit is that they are a simulation: a clean probe proves the path, not the phones, the PBX, or the codec negotiation.


The network's own counters on the voice paths

Switches and routers already count what happens to packets. They count errors and discards on phone ports and uplinks, drops in the QoS queue that voice uses, utilization on WAN links, and PoE draw on phone ports. Collected over SNMP, these are the cause side of the story. They do not know a call exists, but when a bad-call cluster lines up with a discard counter climbing on one uplink, the investigation is over.


SourceWhat it seesWhat it is blind toEffort to collect
PBX call quality recordsEvery real call, every trunk, with outcome and quality fieldsAnything that happens between calls; sometimes internal mediaLow: export or query the PBX
Phone-reported statisticsThe user's received stream, per directionWhat the network did to cause itLow to medium: needs the PBX or a collector to gather them
Synthetic voice probesEach path, continuously, under controlled conditionsPhones, the PBX, real codec behaviorMedium: place an agent at every site and toward the provider
Network countersErrors, discards, queue drops, utilization on the paths calls useCalls themselvesLow if you already monitor the network over SNMP

Baselining per site and per trunk


A baseline is a written-down description of normal, and in voice there is no single normal. Head office on fiber with wired phones will score differently from a branch on broadband behind a VPN, and both will differ from remote workers on home Wi-Fi. The external legs across each trunk have their own normal. Baselining means recording those normals separately, so that "worse than usual" means something.


Two numbers per group do most of the work: the median MOS and the worst five percent. The median tells you what a typical call sounds like. The tail is where the complaints live. A site can have a lovely median and a miserable tail, and the median will hide it completely. Add the share of calls below a floor. A common floor is a MOS of about 3.6, roughly where "fine" turns into "noticeable". And you have a baseline you can defend. Build it from two or three typical weeks, split by site, by trunk, by internal versus external, and by direction where the records allow. If your business has a daily shape, keep an hourly version too.


Baseline (three typical weeks)
Group          Calls/wk  Median MOS  Worst 5%  Below 3.6  Notes
HQ internal    6,120     4.3         4.0       0.4%       fiber, wired phones
HQ external    3,900     4.2         3.9       0.8%       via provider-main
Warehouse      740       4.1         3.5       3.1%       one uplink, Wi-Fi handsets
Branch B       1,310     4.2         3.8       1.0%       broadband + VPN
Probe HQ-B     --        loss 0.1%   jit 6 ms  rtt 38 ms  every 5 minutes

Notice what the baseline already reveals before any alerting exists. The warehouse's tail is far worse than everyone else's; nobody has complained loudly, because warehouse staff are used to it. That is your first project. Baseline the synthetic probes the same way, per path, so that a probe reading has a normal to be compared against. And re-baseline after anything that changes the path or the phones. That could be a new WAN circuit, a codec change, a firmware rollout, or an office move. That is because a baseline from before the change judges the new world by the wrong normal.


Dashboards: trend, worst calls, worst sites


A quality dashboard should answer three questions at a glance, one panel each, and nothing more. Big dashboards get ignored; three panels get looked at.


  • Is quality drifting? A trend panel: median and worst-five-percent MOS per site and per trunk, one point per day, several weeks wide. Drift is a slow slide you cannot see in any single day, and this panel is the only place it shows.
  • Who is suffering right now? A worst-calls panel: the twenty lowest-scoring calls of the day, with time, site, phone, trunk, direction, loss, and jitter. It is the fastest way to notice a cluster forming while it is still happening.
  • Where should effort go? A worst-sites panel: sites and trunks ranked by their share of bad calls this week against their own baseline. This is the panel that decides what you work on.

Put the synthetic probe results on the same screen, one row per path, showing loss, jitter, and latency against baseline. Then put the dashboard somewhere it will actually be seen: a wall screen, the first tab of the morning, the top of the weekly review. A dashboard nobody looks at is a report with extra steps.


Tip: show worst-five-percent MOS more prominently than the average. Averages of quality scores are flattering and nearly useless. A site where one in twenty calls is bad has a problem, and its average will say it does not.


Thresholds that catch degradation before users complain


Users complain when a call is bad enough to interrupt a conversation. The program's job is to notice the step before that: quality that is worse than normal but not yet awful. That means two kinds of thresholds. A broken threshold fires on the kind of failure that is already hurting people and should page someone. A degrading threshold fires on drift and produces a ticket or a line in the weekly review.


The single most important rule: threshold on rates, never on single calls. One call with a MOS of 2.9 is a mobile caller in a car park. Fifteen percent of a site's calls below 3.6 in the last hour, at a site whose baseline is under one percent, is an incident. Express thresholds relative to each group's own baseline. That way, the warehouse and head office are each judged against their own normal rather than a company-wide number that is wrong for both.


For synthetic probes, thresholds on the raw numbers work well because the probe is controlled. Common starting points are loss above one percent, one-way latency above roughly 150 milliseconds, and jitter above a few tens of milliseconds. Each must be sustained across several consecutive samples rather than a single reading. Jitter Explained covers what those jitter numbers mean and why sustained matters more than peak. Tune them per path after a few weeks of baseline. A table of sensible starting thresholds looks like this:


SignalDegrading (ticket / review)Broken (page)
Share of calls below MOS 3.6, per site, per hourTwo to three times the site's baselineA large jump, such as one call in ten, with real call volume behind it
Worst-five-percent MOS, per site, per dayDown by a few tenths against baseline for several daysNot a paging signal; it is a trend
Synthetic probe loss on a pathAbove the path's baseline for an hourAbove one percent for several consecutive samples
Synthetic probe jitter or latencySustained rise above baselineSustained above the starting points, during business hours
Trunk legs: external MOS per trunkTail drifting down over a weekSharp drop across all external calls at once

Whatever the numbers, require confirmation. Require two or three consecutive samples over the line before an alert fires. Require a clear margin below it before the alert clears. That way, a path hovering at the threshold does not page you every five minutes. Tie voice thresholds to dependencies like any other alert. If the WAN link to a branch is down, its probe will fail and its calls will vanish. In that case, you do not need three alerts to say so. If the PBX is down, no call records will arrive, and "no data" should not read as "perfect quality."


The weekly worst-calls review


Thresholds catch sharp changes. The weekly review catches everything else. It catches the slow drift, the site that is always a bit worse, and the phone that has been bad for a month. The review takes about thirty minutes, and it is the part of the program that turns data into fixes.


  • 1. Pull the worst calls of the week. The lowest fifty by MOS, or every call below the floor if that is fewer. Include time, site, phone, switch port if you have it, trunk, direction, codec, loss, and jitter.
  • 2. Group them. By site, by phone, by trunk, by hour of day, by internal versus external. Clusters are the entire point; a single bad call is noise, twelve bad calls sharing a switch are a cable.
  • 3. Ask what each cluster shares. Same phone: its port, its cable, its firmware. Same site and hour: an uplink, a scheduled job, the Wi-Fi at shift change. Same trunk: the provider or the internet link. External only: the trunk path. Internal only: the LAN.
  • 4. Check the trend panel and the worst-sites panel for drift that did not make the worst-calls list.
  • 5. Assign one action per cluster with an owner and a date, even if the action is "watch this for another week."
  • 6. Close last week's actions. Did the fix change the numbers? If not, the diagnosis was wrong, which is worth knowing.
  • 7. Write three lines in a log. What was worst, what you think caused it, what you did. In six months this log is the history nobody else has.

A real review might read like this. Fourteen of this week's fifty worst calls came from the second floor between nine and half past. They were all internal, all on phones fed by the same access switch. The trend panel shows the floor's tail drifting for three weeks. Action: check that switch's uplink utilization and discards at nine tomorrow; owner: you; due: Friday. That is the whole ceremony, and it beats a month of "the phones are bad."


Correlating bad-call clusters with network events


A cluster tells you when and where. The network monitoring tells you why, if you lay the two on the same timeline. Take the cluster's window and look at what the network was doing. Look at interface flaps and error growth on the phones' ports, and utilization and discards on the uplinks and WAN links they use. Look at QoS queue drops, PoE changes, routing changes, and configuration changes. Look at scheduled jobs such as backups and replication, and the synthetic probe on the same path.


One prerequisite is boring and decisive: every source must agree on the time. Sync the PBX, the phones, the switches, and the monitoring server to the same NTP source. Otherwise, a cluster stamped 09:04 by the PBX will land at 09:11 on the switch and the overlay will point at nothing.


A few patterns come up so often they are worth memorizing:


  • Same minutes every day. A scheduled job saturating a link, or shift change on the Wi-Fi. The probe on that path will show loss at the same minutes.
  • One site, every hour. That site's uplink or WAN link, or its QoS policy. Compare its probe against a healthy site's.
  • External calls only, all sites. The trunk path: the internet link, the firewall, or the provider. Internal calls in the same window will be clean.
  • One switch, mixed hours. A port, a cable, a duplex mismatch, or a PoE budget under strain.
  • Bad calls, clean probes. The path is fine; look at the phones, the PBX, and codec transcoding. This is the case the network team gets blamed for and did not cause.

The probe is the tiebreaker. When calls and the probe on the same path degrade together, it is the network, and you can name the segment. When calls degrade and the probe stays clean, it is not the path. The full isolation playbook, including how to build the evidence a provider will accept, is in Proving Where Call Problems Live.


The program-maturity ladder


Most organizations sit on one of five rungs. Find yours, and the next rung is the plan.


LevelWhat it looks likeWhat you can answerTo move up
0: We look when someone complainsNo quality data is kept; troubleshooting starts from a description of a symptomNothing about the pastTurn on quality fields in the call records and keep them for a few months
1: We keep the recordsCall and quality records are retained and can be queried after a complaint"Was that call actually bad, and how?"Write down baselines per site and per trunk; build the three-panel dashboard
2: We know what normal isBaselines exist, the dashboard is looked at, the worst calls are reviewed weekly"Is the warehouse worse than usual, and since when?"Add synthetic probes on every site-to-site path and toward the provider; set rate-based thresholds
3: We hear it before users doProbes and thresholds alert on degradation; clusters are correlated with network counters"Which segment, and what changed there?"Tie alerts to dependencies, keep the evidence file ready, make quality part of change review
4: Continuous quality monitoringEvery call scored, every path probed, alerts noise-free, provider disputes settled with data"Prove it" — to anyoneKeep the weekly review honest; revisit baselines after every major change

Two honest notes about the ladder. Moving from level 0 to level 2 costs almost nothing but discipline. It delivers most of the value. Knowing normal and reviewing the tail weekly heads off many "the phones are bad" tickets before they are opened. Level 3 is where probes and network monitoring need to be in place, and where a hosted PBX changes the picture. With a hosted PBX, you may not get per-call records at all. In that case, phone statistics and probes carry the program instead.


Where to go next


A call-quality program is four sources, a baseline for every site and trunk, and three dashboard panels. It has thresholds on rates rather than single calls, and a weekly half hour spent on the worst calls. Correlating the clusters with network counters and probes turns "the phones are bad" into "the second-floor uplink discards at nine." That is a sentence you can act on.


The trunk side of the picture is covered in SIP Trunk Health. The phones' ability to register at all is covered in Phones That Won't Register. All the guides in this topic live in VoIP & Call-Quality Monitoring.


Put this into practice with SysPeep Monitor

The call-scoring half of this program lives in your PBX and phones. The network half is what SysPeep Monitor collects over SNMP and similar collectors. It collects errors and discards on voice ports and uplinks, WAN latency and loss history, and utilization on the paths calls take. Its dashboards and thresholds make the drift visible. Put the two side by side and the bad-call clusters start explaining themselves. The free trial installs on a Windows machine in minutes.

Download Free TrialLearn About SysPeep