Home › Topics › It's Always DNS: Name Resolution Health › Monitoring DNS Health

Monitoring DNS Health: Resolution Time and Failure Rates


Most networks discover DNS problems the same way: a flood of confusing tickets, followed by an hour of denial, followed by the traditional confession — it was DNS. That is a terrible detection method. It is slow, it burns your credibility, and by the time humans notice, caches have been papering over the failure for so long that the trail has gone cold.


The fix is almost embarrassingly simple. Ask your own DNS servers a question every minute or two, time the answer, and alert when the answers get slow, wrong, or stop coming. This is called a synthetic check — synthetic because no real user is involved. The monitoring system plays the part of a user on a schedule and measures what happens. DNS is unusually well suited to this treatment, because a query is tiny, fast, and free to repeat forever.


This guide shows you how to do it properly. It covers what questions to ask, and which servers to ask them of (spoiler: every resolver, separately, plus the upstream layer). It covers what to measure and where to set thresholds that catch trouble without crying wolf. And it shows how to write the resulting alerts so they name the guilty server instead of announcing "DNS bad" to a confused on-call phone.


In this guide

  • Why DNS deserves its own checks
  • The synthetic check: a known question on a schedule
  • What to measure
  • Check every resolver separately — and both layers
  • Thresholds that catch trouble without crying wolf
  • Gradual degradation versus hard failure
  • Alerts that name the failing resolver
  • A starter setup you can build this week
  • Where to go next

Why DNS deserves its own checks


If you already monitor your domain controllers or DNS appliances with ping, you might reasonably ask what more there is to do. The answer is the oldest lesson in monitoring: a host that answers ping is not the same as a service that works. A DNS server can respond to ping beautifully while the DNS service on it is hung, misconfigured, out of memory, or forwarding into a black hole. Ping tests the box. A synthetic query tests the thing users actually need: a correct answer, quickly.


DNS also earns special treatment because of its position. As Why It's Always DNS explains, name resolution sits in front of nearly every other service. Its failures scatter into symptoms that don't mention DNS at all. A dedicated check turns that around: instead of deducing DNS trouble from a dozen weird downstream alarms, you get one direct signal from the source. And caching delays the user-visible pain. So a fresh synthetic query — which pays the full resolution price on schedule — often catches the break many minutes before the first ticket. Those minutes are the difference between "we're on it" and "what outage?"


The synthetic check: a known question on a schedule


The mechanics could not be simpler. On an interval — every minute is common for something this cheap — the monitoring machine sends a DNS query to a specific server. It records three things: did an answer come back, was it the kind of answer we expected, and how long did it take. Any scripting language can do it; so can most monitoring tools. The craft is entirely in what you ask and whom you ask.


Choose test names deliberately, because each name exercises a different part of the machinery:


  • An internal name you own, like dc1.corp.example — something stable in your own zone. This tests the resolver's knowledge of your world and involves no upstream at all.
  • A stable external name, like a major site that is effectively never down. This tests the full chain: your resolver, its forwarder, and the internet beyond.
  • Optionally, use a name with a known expected answer — a record you control whose value never changes. That way, you can check not just that an answer arrived but that it is correct. Wrong answers are rarer than slow ones, but they are far nastier.

And ask by address, not by role. Query 10.10.8.21 and 10.10.8.22 explicitly rather than "whatever resolver this machine happens to use." A check that rides the operating system's normal resolution path inherits the operating system's caches and failover. That is precisely the machinery you are trying to see through. On the command line, that means the form:


# Ask ONE specific resolver, bypassing the local stub and cache
dig @10.10.8.21 www.example.com +noall +answer +stats

www.example.com.   287   IN   A   203.0.113.80
;; Query time: 2 msec
;; SERVER: 10.10.8.21#53(10.10.8.21)

# The same idea in PowerShell
Measure-Command { Resolve-DnsName www.example.com -Server 10.10.8.21 }

That Query time line is your future resolution-time graph, one dot at a time. Two milliseconds means the resolver had the answer cached and handed it straight over — the healthy LAN case.


Failures, when they come, arrive in different shapes, and a good check records which. There might be a timeout (no reply at all) or a SERVFAIL (the server replied, admitting it could not complete the job). Or there might be an unexpected NXDOMAIN (the server confidently claims your known-good name does not exist). Each shape implicates a different suspect — the playbook article in this pillar decodes them in detail. So an alert that reports the shape, not just "failed," is already halfway to a diagnosis.


Tip: when you test an external name, you are also unavoidably measuring the internet and someone else's servers. Pick a boring, famously stable name. Treat a failure of only that check — with internal checks green — as "the outside world is having a day," not as your outage. The per-layer matrix below makes that distinction automatic.


What to measure


Resist the urge to collect everything. Four measurements carry nearly all the diagnostic weight:


MetricWhat it tells youWhat to watch for
Resolution timeHow long a query takes against one specific resolverCreep above the usual baseline; sudden jumps to seconds
Failure rateThe fraction of checks that fail (timeout, SERVFAIL, refused)Anything sustained above zero; occasional single drops are normal on UDP
Answer correctnessWhether a known record still returns its expected valueAny change you didn't make — hijacked, stale, or fat-fingered records
Per-resolver spreadThe same check compared across resolversOne server drifting away from its twin — the early sign of a half-broken pair

A word on failure rate, because DNS has a quirk here. Most queries travel over UDP, a fire-and-forget protocol with no delivery guarantee. So a single lost packet — and therefore a single failed check — is unremarkable in isolation. Failure rate matters as a pattern: three consecutive failures, or a steady few per hour where there used to be none. Track it per resolver and per test name. A lump-sum "DNS failure rate" across all servers averages the guilty and the innocent into a number that accuses nobody.


Resolution time, meanwhile, is really two different measurements wearing one name. A cached answer never leaves the resolver and should be near-instant on a LAN. An uncached answer pays for a trip upstream and costs whatever the chain costs. Your graph will show both mixed together — a low floor with regular spikes — and that is fine once you know what you are looking at. What matters is that the floor stays low and the spikes stay modest.


As for how often to check: err on the frequent side. A DNS query costs a few bytes and a few milliseconds. So a one-minute interval is cheap even across a dozen checks — and the interval is your detection floor. On a five-minute schedule, a resolver can be dead four minutes and fifty-nine seconds before you get your first hint. Frequent checks also make the confirmation rule in the next section workable: waiting for three consecutive failures at one-minute spacing still beats any human noticing.


Check every resolver separately — and both layers


Here is where most DNS monitoring falls short. A typical network has two internal resolvers (often Active Directory domain controllers) which forward external questions to an upstream — the provider's resolvers or a public service. That is at least three or four servers with distinct failure modes. A single "can I resolve a name?" check tells you almost nothing about which of them is sick. Worse: because clients quietly fail over between resolvers, a check that uses the normal client path can stay green while half your redundancy is already dead.


The cure is a small matrix: each resolver, asked directly, for each layer of question. It looks like this:


CheckTargetTest nameA failure here means
1Internal resolver 1 (by IP)Internal namedc1's DNS service or your zone is broken
2Internal resolver 2 (by IP)Internal nameSame, for dc2 — your redundancy is thinner than it looks
3Internal resolver 1External namedc1 can't reach or use its forwarder
4Internal resolver 2External nameSame, for dc2
5Upstream/forwarder target (by IP)External nameThe upstream itself, or the path to it, is the problem

Read the matrix by columns during an incident and it practically speaks. Checks 1-2 red, 5 green: the problem is inside your walls. Checks 3-4 red, 1-2 green: your resolvers are fine but their forwarding path is not — look at the firewall or the upstream. Only check 3 red: dc1 specifically has a forwarder problem its twin doesn't share. Everything red: condolences, and at least you know it's everything. The point of monitoring both layers separately is exactly this. The pattern of green and red localizes the fault before you have typed a single manual command. (Designing the layers so this stays true is the subject of Resolvers, Forwarders, and Caches.)


If your organization runs branch sites with their own resolvers, repeat the pattern there. A check running at the branch tests what branch users actually experience, including the WAN link a central check never touches.


Thresholds that catch trouble without crying wolf


Numbers first, then the reasoning. For resolvers on the same LAN as the checker, a cached answer should come back in a few milliseconds. For those resolvers, treat a floor that climbs into tens of milliseconds as worth a look. Uncached external lookups normally land in the tens-to-low-hundreds of milliseconds depending on where the answer lives. Sustained times near a second are a warning. Anything routinely hitting multiple seconds is effectively an outage already, because applications are stalling even when answers eventually arrive.


Those are starting points, not laws. The honest procedure is to run the checks for a week or two and look at what your normal looks like. Then set warning thresholds a comfortable margin above normal and critical thresholds where users genuinely feel pain. A resolver that answers external names in a leisurely quarter second through a slow provider link may simply live there; alerting on it daily helps no one.


For failures, the rule is confirmation before alarm. Because single UDP drops are routine, alert on consecutive failures rather than on any single miss. That means two or three in a row — a couple of minutes of genuine deadness at a one-minute interval. But do not soften correctness checks the same way: a known record returning the wrong value is alert-worthy on the first occurrence, because it never happens innocently.


  • Warning: resolution time sustained well above the resolver's own baseline; occasional repeated failures; one resolver drifting notably slower than its twin.
  • Critical: consecutive failures against any single resolver; SERVFAIL on internal names; any wrong answer for a controlled record.
  • Informational: the external-reference check failing alone — the outside world's problem until proven otherwise.

Gradual degradation versus hard failure


Hard failure is the easy case: the service stops, consecutive checks fail, the alert fires, everyone knows where they stand. The more valuable catch is the slow slide, because DNS rarely dies suddenly. A resolver under memory pressure answers a little slower each day. A failing WAN path drops a few more forwarded queries each week, and retries hide the loss as growing latency. An overloaded upstream stretches its answers gradually. Users feel this long before they can articulate it — "everything's a bit sluggish lately" — and no single check ever fails hard enough to page anyone.


Catching the slide takes history rather than cleverness. Keep the resolution-time series for months, not days, and actually look at the trend: this quarter's floor versus last quarter's, spike frequency now versus then. A useful trick is to alert on the gap between twins. Two identical resolvers doing identical jobs should perform nearly identically. So a growing gap between dc1 and dc2 is suspicious even when both remain inside absolute thresholds. It is also worth knowing that a resolver's cache empties when the service restarts. So expect a short, innocent period of slower answers after every reboot. A cold cache pays upstream prices until it rewarms. If your monitoring shows a step-change in resolution time, "did something restart?" is the first question. The uptime graph next to the latency graph answers it instantly.


One more slow failure deserves a guard if you host your own zones: drift between copies. Suppose a secondary server quietly stops transferring updates from the primary, or one domain controller misses changes the others have. Different resolvers begin giving different answers for the same name — partial failure at its most confusing. The simple defense is to query the same controlled record against every server that should hold the zone and compare the results. Identical zones have no excuse for different answers, so any disagreement is alert-worthy on the spot.


Alerts that name the failing resolver


All this careful measurement is wasted if the alert that reaches a human says only that something DNS-ish is unhappy. Compare:


BAD:   DNS check failed

GOOD:  Resolver dc2 (10.10.8.22): SERVFAIL for external names,
       3 consecutive checks. Internal names on dc2: OK.
       dc1: all checks OK. Upstream direct check: OK.
       => dc2's forwarding path is the likely fault.

The good alert does three things the bad one doesn't. It names the server, by hostname and IP, so nobody starts an investigation by asking which of the four boxes this is about. It states the pattern across the matrix — what failed and what still works — which, as we saw, is the diagnosis. And it ventures a conclusion the on-call person can immediately test. You built five checks precisely so the alert could talk like this; let it.


Two structural rules complete the picture. First, make downstream checks depend on DNS checks. When a resolver dies, every name-based check in your monitoring system is about to fail with it. Without declared dependencies you get an alert storm in which the one alert that matters — the resolver itself — arrives fifty-first. Dependency thinking covers the technique. Second, make sure the monitoring system itself does not secretly depend on the thing it is judging. Monitor your resolvers by IP address, never by a name those very resolvers must resolve. Otherwise, your DNS outage will take your DNS monitoring down with it — the watcher blinded by the crime it was hired to witness. The broader discipline of keeping your monitoring honest lives at Monitoring the Monitor.


Tip: put the resolution-time graphs for all resolvers on one dashboard panel, same scale, side by side. Half of DNS diagnosis is comparison, and twins that should match — but don't — jump off the screen in a way no table of numbers manages.


A starter setup you can build this week


Everything above compresses into an afternoon of setup:


  • 1. List every resolver that matters: each internal DNS server, and the upstream or forwarder targets they rely on. IPs, not names.
  • 2. Pick two test names — one stable internal, one stable external — and, if you can, create one never-changing record you own for correctness checking.
  • 3. Build the five-check matrix from this guide, each check aimed at one resolver by IP, each running every minute or two.
  • 4. Record resolution time and success for every check, and keep the history long enough to see seasons, not just days.
  • 5. Set provisional thresholds — a few milliseconds cached, modest fractions of a second uncached — and revisit them after two weeks of real data.
  • 6. Alert on consecutive failures per resolver, first-offense wrong answers, and sustained latency above baseline. Write the alert text to name the resolver and state the matrix pattern.
  • 7. Declare dependencies so name-based checks defer to the resolver checks, and confirm the monitor reaches every resolver by IP.

That is a modest amount of work for what it buys. The next DNS incident announces itself, names its own culprit, and hands you a head start measured in minutes. That is often before the first user has noticed anything worth a ticket.


Where to go next


Monitoring tells you that and where; someone still has to fix it. When a check goes red, the DNS troubleshooting playbook picks up exactly where the alert leaves off. It walks the same layers by hand with nslookup and dig. Your check matrix may keep implicating the same architectural sins — a single resolver, mystery forwarding paths. In that case, Resolvers, Forwarders, and Caches shows the layout that fails loudly instead of weirdly. The whole series lives at It's Always DNS: Name Resolution Health.


Put this into practice with SysPeep Monitor

Your resolvers are servers too, and they deserve the same watchful eye as everything behind them. SysPeep Monitor auto-discovers them and tracks the health of the Windows machines your DNS roles run on over WMI, SNMP and SSH. Its dependency-aware alert rules make sure a sick resolver pages you once, by name — not fifty times through everything downstream. The free trial sets up on one Windows machine in minutes.

Download Free TrialLearn About SysPeep