Measuring SLA Compliance From Monitoring Data
An internal service level agreement makes promises, and at the end of the month somebody has to say whether they were kept. Suppose that somebody is you, with a spreadsheet and a memory of which days were bad. You have a rough sense of how long things were down. In that case, the number you produce will be argued with, and the argument will be fair. The month has to be measured, not remembered.
The good news is that the measurement mostly already exists. Every monitoring system keeps a status history — a record of when each check went from up to down and back. That history, cut to the right hours and with the agreed exclusions removed, is your compliance figure. The work is in deciding, before the month starts, exactly how the cutting is done.
By the end of this guide you will be able to name the monitoring source behind every clause. You will be able to write a measurement specification that stops arguments before they start. You will be able to handle maintenance windows and upstream outages without fooling anyone, and run a query that produces the month's downtime per service. You will be able to avoid the three mistakes that make an honest number wrong.
In this guide
- Every clause needs a source
- Mapping clauses to checks
- The measurement specification
- Exclusions, written before the month
- The arithmetic
- A query that computes the month
- Measuring response and resolution
- Three traps
- The monthly compliance worksheet
- A short story from Harbourview Clinic
- Checklist for the measurement
- Where to go next
Every clause needs a source
A clause without a named measurement source is a sentence, not a promise. "The ERP will be available 99.5% of business hours" says nothing about what "available" is, who decides, or where the record lives. At month end three people will have three different figures. The fix is to attach a source to every clause before the agreement is signed. Specify this check, from this place, at this interval, recorded here.
The source has to be a machine, not a person. A person's recollection of the month is shaped by which outage they were shouted at about; a status history is shaped by nothing. It is also there at three in the morning. (People occasionally are too. They are not reliable witnesses.)
If a clause has no possible machine source, the clause waits. Set up the check, collect a month of data, then promote it to the agreement. An agreement built only on clauses you can measure is shorter, and every line in it can be defended.
Mapping clauses to checks
An availability clause maps to a service check: a test that does what a user does. Examples are fetching a login page, opening a file share, or running a small database query. It does not map to a ping. A ping proves the network stack answers; it does not prove the application does. A server can return pings all day while its database has stopped. The difference, and why it decides whether your number means anything, is in ping up versus service up. One service may really be several checks (a web front end, a database, a file share). Grouping those checks so the service counts as down only when the user-facing path fails is covered in service-based monitoring.
A response-time clause maps to an acknowledgement timestamp. That is the moment a named person marked the alert as acknowledged in the monitoring console. Or it is the moment the ticket changed from "new" to "assigned" in whatever tracks your tickets. Either works, as long as the agreement says which. Response time is that timestamp minus the alert's or ticket's creation time.
A resolution clause maps to the clear. For a monitored service, that is the moment the check returned to up and the alert cleared. For a ticket, it is the close timestamp. Prefer the alert clear where one exists. A ticket closed at five for a service that came back at two measures the paperwork, not the service. The full set of incident timestamps is in the incident clock.
| Clause | Measurement source | What it records | Not this |
|---|---|---|---|
| Availability of a service | Service check status history (HTTP, SMB, SQL, DNS, and so on) | Minutes in the down state within service hours | Ping of the host; server uptime counter |
| Response time by priority | Alert acknowledgement or ticket assignment timestamp | Minutes from alert or ticket creation to a named person starting | First reply email; the time someone "saw it" |
| Resolution time by priority | Alert clear timestamp, or ticket close for non-monitored requests | Minutes from creation to the service working again | Ticket close time for a monitored outage |
| Capacity or performance clause | Metric history (response time, disk free, link utilisation) | Share of samples within the stated bound | A single spot check on a quiet afternoon |
The measurement specification
A measurement specification is one table per service that answers every question a sceptic could ask about the number. It states what counts as down, from where the check runs, how often, and what is excluded. Write it once, attach it to the agreement, and the monthly figure stops being a matter of opinion.
"What counts as down" is the line most people leave vague, and the whole number depends on it. One failed check is not down; checks fail for a dropped packet. Three consecutive failures at a one-minute interval is down. The outage is timed from the first of them, or from the third, and the specification says which. The reasoning behind that choice is laid out in defining down; the specification just records the answer.
"From where" matters more than it looks. A check that runs on the monitoring server tells you whether the monitoring server can reach the service. If users sit in another building behind another switch, the check should run from there too, using a remote probe. A remote probe is a small piece of the monitoring system installed at another site that runs checks from that site's point of view. How probes fit in is in probes and collectors. The specification names the vantage point so that "up from the server room" and "up for the warehouse" are never confused again.
| Item | What to specify | Example |
|---|---|---|
| Service | The exact host, URL, or share being tested | https://erp.example.com/login |
| Check type | The test and what a pass looks like | HTTP GET; pass is status 200 and the text "Sign in" in the body |
| Vantage point | Which server or probe runs the check | Monitoring server (head office) and warehouse probe; down only if both fail |
| Interval | How often the check runs | Every 60 seconds |
| Down means | Consecutive failures required, and where the clock starts | 3 consecutive failures; outage timed from the first failure |
| Up means | Consecutive successes required to end the outage | 2 consecutive successes |
| Service hours | The hours during which downtime counts | Mon–Fri 07:00–19:00 local time, excluding listed public holidays |
| Exclusions | Categories of downtime that do not count, decided in advance | Announced maintenance (3 business days' notice); ISP outage confirmed by ISP ticket; user-side faults |
| Source of record | Where the numbers are read from | Status history in the monitoring database; availability report, monthly |
| Response source | Which timestamp is acknowledgement | Alert acknowledged in the console, or ticket set to "assigned" |
| Resolution source | Which timestamp is the fix | Alert clear; ticket close for requests that have no check |
Exclusions, written before the month
An exclusion is a category of downtime the target does not count against you, and the rule that keeps it honest is timing. Exclusions are written before the month, as categories. They are never written after an outage, as descriptions of it. "Announced maintenance windows" is an exclusion. "The Thursday the SAN firmware did that thing" is an excuse with a date on it.
Maintenance windows are the common case and the easiest to do properly. A window is announced in advance. It is entered into the monitoring system so that alerts are suppressed and the status history is marked. Its minutes are subtracted from the period. The whole discipline of planning windows and suppressing the noise inside them has its own pillar at maintenance windows and suppression. For compliance, the only rule that matters is that a window entered after the outage started is not a window.
Upstream outages are the second case, and they need evidence. If the agreement excludes ISP outages, the exclusion applies only when the ISP's own ticket confirms one. Monitoring makes that easier. A dependency rule marks everything behind the router as "unreachable because the router is" rather than "down". That gives a clean record of what was actually the ISP's fault. That mechanism is described in dependency suppression.
User-side faults are the third case and the most abused. A laptop with a broken wireless card is not a network outage. A whole floor that cannot reach the file server because the floor switch failed is, even though the file server was fine. Write the boundary into the specification — "down means the check fails from the named vantage point" — and let the vantage point settle it.
Keep the list short. Every exclusion is a piece of the month the number no longer describes, and a target with six exclusions describes a month that never happened. Three lines is right. If someone wants a fourth, ask what they are planning.
The arithmetic
The calculation is subtraction and one division, and the only skill is being strict about which minutes go where. The period is the month's minutes inside service hours. Exclusions are subtracted from the period, not from the downtime, because an excluded minute is one the promise did not cover at all. Downtime is the minutes the check was down, inside service hours, and not inside an exclusion.
period_minutes = minutes in the month that fall inside service hours
excluded_minutes = announced maintenance + confirmed upstream outages
(only the part inside service hours)
measured_minutes = period_minutes − excluded_minutes
downtime_minutes = minutes in DOWN state, inside service hours,
not inside an exclusion
availability = (measured_minutes − downtime_minutes) / measured_minutes × 100
Example: 22 business days × 10 h = 13,200 period minutes
one 60-minute announced upgrade at 07:00 = 60 excluded
measured = 13,140
two outages, 31 and 17 minutes = 48 downtime
availability = (13,140 − 48) / 13,140 × 100 = 99.63%
Do the calculation per service, and only per service. Adding services together before dividing produces a number that belongs to nothing. The rest of the rules, including partial months and services added mid-period, are in honest availability math.
A query that computes the month
Most monitoring products produce an availability report from a menu. Where yours does, use it, because it already knows about its own maintenance markers. The query below is for when you want the raw minutes, want to check the report's arithmetic, or need to feed a spreadsheet. It is illustrative. Table and column names differ by product. Some products store one row per check result rather than one row per state period. Check the product documentation for your schema before trusting anything it returns.
-- Illustrative only: table and column names vary by product.
-- Assumes status_history holds one row per state period:
-- service_name, state ('UP' or 'DOWN'), started_at, ended_at
-- Replace the two placeholders with the first and last moment of the month.
SELECT service_name,
COUNT(*) AS outages,
SUM(TIMESTAMPDIFF(MINUTE, started_at, ended_at)) AS down_minutes,
MAX(TIMESTAMPDIFF(MINUTE, started_at, ended_at)) AS longest_minutes
FROM status_history
WHERE state = 'DOWN'
AND started_at >= '<period start>'
AND ended_at < '<period end>'
GROUP BY service_name
ORDER BY down_minutes DESC;
That gives whole-month downtime, not business-hours downtime, and it does not yet subtract exclusions. Clipping each outage to service hours is fiddly in plain SQL. One outage can start at 18:40 and end at 19:30 and only fifty minutes of it count. The practical approach is to export the rows into a spreadsheet. Clip each outage to the service hours by hand (there are rarely more than a handful). Mark the ones that fall inside an exclusion. For a small team that is ten minutes a month. The date function differs between SQLite, MySQL/MariaDB, and SQL Server, so expect to adjust the TIMESTAMPDIFF line for your backend.
Measuring response and resolution
Response and resolution are measured per incident, then summarised as a share that met the target. A single average hides the one ticket that waited four hours, and that ticket's owner will remember it longer than the average. The figure the agreement wants is "how many priority-one incidents were acknowledged inside fifteen minutes, out of how many."
A spreadsheet does this without help. With one row per incident and columns for priority and response minutes, the compliance share for priority one is:
=COUNTIFS(priority, "P1", response_min, "<=15") / COUNTIF(priority, "P1")
Report the share, the count, and the worst case. "Nine of ten P1 incidents acknowledged within fifteen minutes; the tenth took forty-one" tells the reader everything. "Average response 11 minutes" hides the forty-one.
Three traps
Averaging across services
Ten services at 99.9% and one at 97% average to 99.6%. That looks like a pass and describes a month in which one service failed its promise badly. Availability is a per-service figure. Do not average it across services. Not for the summary, not for the slide, not because the number was "basically fine." The worked example of how much this hides is in SLA reporting without spin.
Counting a flap as one minute
A flap is a service going down and up repeatedly in a short time, usually because something underneath it is failing intermittently. Each dip may be shorter than your check interval. If so, the status history shows a handful of one-minute outages and the month looks like a rounding error. Users experienced two hours of a service that could not be trusted. The specification should say how flapping is counted. A common rule is that a service down more than a set number of times in an hour is down for the whole hour. How short an outage a check can even see is covered in granularity blind spots.
Measuring from inside the data centre
A check that runs from the same rack as the service will report it as up during any failure between that rack and the users. The core switch dies, the building loses the ERP, and the monitoring server, sitting next to the ERP, records a perfect month. Put at least one vantage point where the users are. A month measured from the wrong place describes nothing they experienced.
Tip: once a quarter, pick one outage users remember and check that the status history agrees with them about when it started and ended. If the history says twelve minutes and the helpdesk says an hour, the specification has a gap. A quiet quarter is the place to find that gap.
The monthly compliance worksheet
This sheet turns the arithmetic into a record. One row per service, filled in from the query and the exclusion log, kept every month in the same place. The example rows show one met target and one miss.
| Service | Service hours | Period min | Excluded min | Measured min | Downtime min | Achieved | Target | Met? | Longest outage | Exclusion evidence |
|---|---|---|---|---|---|---|---|---|---|---|
| erp.example.com | Mon–Fri 06:00–20:00 | 18,480 | 60 | 18,420 | 48 | 99.74% | 99.5% | Yes | 31 min | Window announced 5 days ahead, calendar entry |
| files.example.com | Mon–Fri 07:00–19:00 | 13,200 | 0 | 13,200 | 214 | 98.38% | 99.0% | No | 171 min | None claimed |
| <service> | <hours> | <n> | <n> | <period − excluded> | <n> | <calc> | <from SLA> | <Yes/No> | <n min, date> | <ticket or calendar reference> |
The last column is the one that earns trust. Every excluded minute points at a calendar entry or a vendor ticket that existed before the outage. So anyone who doubts the figure can check it without asking you. Keep the sheets; a year of them is the start of the uptime evidence file you will want when the target is next discussed.
A short story from Harbourview Clinic
Harbourview Clinic measured availability of its patient records system from the monitoring server. That server lived in the same rack, on the same switch, as the records server. The uplink from that switch to the rest of the building failed one Monday for three hours. Every consulting room lost the system. The monitoring server, still happily talking to its neighbour, recorded nothing at all. The monthly report showed 100%, and the clinic manager showed the report to the practice partners. The practice partners, who had spent the Monday morning on paper, described the figure as "creative." The following month a probe went into the reception office. That month, the exclusions list was rewritten so that nothing could be added to it after the fact. The reports have been less flattering since. They have also been believed.
Checklist for the measurement
Run through this when the specification is written, and again whenever a check changes.
- Every availability clause is backed by a service check, not a ping, and the check tests something a user does.
- Every clause names its vantage point, and at least one vantage point is where the users sit.
- The specification states the interval, the number of consecutive failures that mean down, and where the outage clock starts.
- Service hours are written per service, with a time zone and a holiday rule.
- Exclusions are three lines or fewer, are categories rather than incidents, and were written before the month began.
- Every maintenance window is entered in the monitoring system before it starts, and appears in a calendar the business can see.
- Every upstream exclusion is backed by the vendor's own ticket.
- Downtime is clipped to service hours and calculated per service, never averaged across services.
- Flapping has a written counting rule.
- Response and resolution are reported as "met the target, out of total," with the worst case named.
- The worksheet for the month is saved where last month's is, with its evidence column filled in.
- One outage a quarter is cross-checked against what the helpdesk remembers.
Where to go next
With the specification written and the worksheet filled in, the month is a fact rather than a memory. That fact is what the rest of the practice stands on. The agreement the measurements belong to is described in what an internal SLA is. The targets you are measuring against came from choosing what to promise. The next step is putting the figures in front of people in a form they will believe. That is SLA reporting without spin. Then comes handling the month the sheet says "No," which is when you miss the SLA. All of it is collected in Defining and Tracking Internal SLAs.
Put this into practice with SysPeep Monitor
The arithmetic in this guide needs a status history to run against. The monitoring software behind this site records every state change for every service check. It produces availability reports per service from one web console. It supports remote probes so you can measure from where the users actually are. The free trial installs on a single Windows machine.
Download Free TrialLearn About SysPeep