Home › Topics › Email Flow Monitoring › Mail Queue Monitoring

Mail Queues: The Metric That Predicts the Stuck-Mail Incident


Every stuck-mail incident looks the same in the post-mortem. Somebody pulls up the relay's queue graph, and there it is. The line started climbing at nine in the morning. It kept climbing all day. It was finally noticed at four in the afternoon when the sales team asked why customers had stopped replying. The information was sitting on the server the whole time. Nobody was looking, because nobody had told the monitoring system that this number mattered.


Mail queues are the rare metric that is genuinely leading rather than lagging. Ping tells you a server died. Queue depth tells you mail stopped moving hours before anyone notices, while every other light on the dashboard is still green. If you own a relay, a gateway, or a mailbox server, the queue is the first thing worth monitoring about it.


This guide explains what a queue depth number actually means, and why the age of the oldest waiting message matters more than the count. It explains how to work out what normal looks like on your own relays. It covers the classic causes of a growing queue and how to tell them apart. It explains how to collect the numbers and set thresholds that fire in time without crying wolf. Finally, it covers a worked incident from first alert to fix. If you have not read how mail flows, skim it first.


In this guide

  • What a queue depth of N really means
  • Why age matters more than count
  • Normal versus alarming: baseline your own relays
  • Deferred, bounced, and stuck
  • The classic causes of a growing queue
  • Collecting queue metrics
  • Thresholds and alert design
  • A worked stuck-mail incident
  • Where to go next

What a queue depth of N really means


A mail server's queue is its list of messages it has accepted responsibility for but not yet handed to the next hop. Each entry is a file or database record on disk containing the message and some bookkeeping. That includes arrival time, sender and recipients, how many delivery attempts have been made, and why the last one failed. Queue depth is simply how many entries are in that list right now. It is a snapshot value, a gauge in monitoring terms, that rises when mail arrives faster than it leaves.


Most mail servers keep several queues, and the names vary, but the roles are consistent. The active queue holds messages being delivered at this moment. The incoming queue holds messages just accepted and awaiting their first attempt. The deferred queue holds messages that have failed at least once and are waiting for their next scheduled retry. Some servers also have a hold queue for messages an administrator has frozen deliberately. The number worth watching most closely is the deferred queue: it is the pile of things that already went wrong once.


Here is the uncomfortable truth about a depth of N on its own: it means almost nothing. Five hundred messages at eight in the morning on a relay that handles the company newsletter is a healthy burst that will drain in minutes. Five hundred on a relay that normally holds three is a disaster in progress. Depth only becomes information when you add how long the messages have been waiting and what this relay normally looks like.


Why age matters more than count


The second number every queue can give you is the age of the oldest message — how long the longest-waiting entry has been sitting there. Age is what the customer experiences. Nobody ever complained that a relay had four hundred messages in its queue; they complained that the invoice took six hours to arrive. Compare two relays. One holds two thousand messages, all under a minute old, because a batch job just dumped them in. The other holds twelve, but the oldest has been waiting since early morning. The first is busy. The second is broken, in a way no depth threshold would catch.


Age has a second property that makes it a better alarm: it compounds. Mail servers retry deferred messages on a back-off schedule, after a few minutes at first and then at longer and longer intervals. A message that has been failing for hours is retried rarely. So even after you fix the problem it may sit a while longer unless you force a retry. A growing oldest-age number says two things at once: something has been wrong for a while, and it will not clean itself up quickly.


The practical rule: collect depth and oldest age together on every queue-holding server you own, and treat age as the primary alarm. Depth tells you the size of the problem; age tells you it is real. If your server can also break deferred mail down by destination, collect that too. "The queue is growing because everything to one domain is failing" is half the diagnosis.


Normal versus alarming: baseline your own relays


You will not find a universal "alarming queue depth" number in this guide, because there is not one. Consider a relay behind a busy web application and a relay that carries a handful of scanner-to-mailbox messages a day. Add an edge gateway for a thousand users. All have completely different normal. The only honest source of thresholds is your own history.


Baselining a queue is not hard. Collect depth and oldest age every minute or so, keep the data, and wait a couple of weeks. That is long enough to see every scheduled batch job, the Monday-morning surge, and whatever the finance system does at month-end. Then look at the graph and write down three things for each relay:


  • The resting depth. What the queue holds on an ordinary quiet afternoon; on most relays a small number, often single digits.
  • The normal peak and its drain time. The largest depth reached during routine bursts, and how long it takes to return to resting. This is your most useful number: a queue still above resting well after the usual drain time is abnormal, whatever its absolute size.
  • The normal maximum age. During a routine burst, how old does the oldest message get before delivery? Usually a few minutes at most; a handful of long-lived deferrals to flaky destinations is normal background noise.

Once you have those, the shape of the graph becomes readable at a glance. A sawtooth — sharp rise, steady drain, repeat — is a scheduled batch sender doing its job. A spike that drains is a burst. A steady climb that never turns down means mail is arriving normally and leaving abnormally: the next hop is refusing or unreachable. A flat plateau with nothing arriving or leaving means the queue-processing service has stopped. And a vertical wall of thousands of messages from one sender is a loop or a compromised account.


Deferred, bounced, and stuck


Three words describe a message that is not being delivered, and they mean different things for your metrics. A deferred message got a temporary failure — a response code starting with 4, a timeout, or a failed DNS lookup. The message is waiting for its next retry. Deferred mail is what makes depth and age grow. A bounced message got a permanent failure (a code starting with 5) or ran out of retry time. The server removes it from the queue and sends a non-delivery report (NDR) back to the sender. Bounces do not grow the queue; they empty it. That is why a blocklisting incident can look like a perfectly healthy queue while mail is being thrown away at the far end.


A stuck message is neither being retried nor bounced. Usually the mail transport service has hung, crashed without exiting, or lost access to its queue directory. The signature is a queue whose depth is frozen or growing while the oldest age climbs and the delivery log goes quiet. Stuck catches people out because the server still answers on port 25 and looks alive to every simple check.


The classic causes of a growing queue


The list of reasons a queue grows is short, and each leaves a recognizable fingerprint in the queue and the log. Learn these six and you will diagnose most incidents from your chair.


CauseQueue signatureLog clueFirst action
Next hop is downEverything bound for one next hop (a gateway, a smart host, one domain) deferred; other mail flows"Connection refused," "connection timed out," or a 4xx greeting from that hostCheck the next hop's service and reachability; fix it if yours, watch age if not
DNS failureDeferrals for every external domain at once"Host not found," "name service error," "temporary lookup failure"Test resolution from the relay itself; check its configured resolvers
TLS negotiation failureDeferrals to one or a few domains; others fineLines mentioning TLS, handshake, or certificate verificationCheck certificate validity on both ends and any strict-TLS policy for that domain
Blocklisted IPQueue may look healthy; bounces and NDRs spike instead5xx rejections mentioning blocks, reputation, or policyLook up your sending address on blocklists; see the reputation guide
Disk fullDepth freezes or drops because new mail cannot be accepted; age climbs"No space left," write failures, the service stopping abruptlyFree space on the queue and log volumes; restart the service if it stopped
Runaway senderDepth climbs vertically; most entries share one sender, subject, or recipientThousands of submissions from one account or host; loop warnings such as "too many hops"Hold that sender's mail, disable the account or connector, then clean the queue

A note on the runaway sender, because it turns a monitoring problem into a security one. A compromised account or a web form under attack can push tens of thousands of messages into a queue in minutes. So can a forwarding rule bouncing a message endlessly between two systems. That burst gets your relay's IP address blocklisted — two incidents for the price of one. The vertical-wall shape is the earliest warning you will get; treat it as urgent.


Collecting queue metrics


How you get the numbers depends on what holds the queue. Every case boils down to the same two values — depth and oldest age — handed to your monitoring system on a schedule.


Command-line relays

Unix-style relays have a queue listing command; the classic name is mailq. It lists each queued message with its arrival time, sender, recipients, and — the useful part — the reason the last attempt failed. Illustratively:


$ mailq
-Queue ID-  --Size-- ---Arrival Time--- -Sender/Recipient--------
7F2A1C0B3*     4213  Tue 09:47:12       alice@example.com
    (connect to gw1.example.net[203.0.113.10]:25: Connection timed out)
                                        bob@example.net
9B31D77E1      1802  Tue 09:51:40       app-noreply@example.com
    (connect to gw1.example.net[203.0.113.10]:25: Connection timed out)
                                        orders@example.net
...
-- 1240 Kbytes in 312 Requests.

You could read that by hand during an incident. But the point of monitoring is to have a script read it every minute and hand two numbers to the monitor. Run a script like this from the monitoring server over SSH, or expose its output through an SNMP extension alongside the server's other SNMP values:


#!/bin/sh
# Report mail queue depth and the age (in minutes) of the oldest deferred message.
QUEUE_DIR=/path/to/queue/deferred
DEPTH=$(mailq | grep -c '^[0-9A-F]\{8,\}')
OLDEST=$(find "$QUEUE_DIR" -type f -printf '%T@\n' | sort -n | head -1)
if [ -n "$OLDEST" ]; then
  AGE_MIN=$(( ( $(date +%s) - ${OLDEST%.*} ) / 60 ))
else
  AGE_MIN=0
fi
echo "depth=$DEPTH oldest_age_min=$AGE_MIN"

Adjust the pattern and directory for your relay. The last line prints two labeled numbers, easy for any monitoring system to parse and graph.


Windows mailbox servers and Windows-based relays

Windows mail servers generally expose their queue lengths as performance counters — the same mechanism that exposes CPU and disk figures. A monitoring system can read those counters over WMI without any script at all. Those mail servers also have a management console or shell that lists queues with message counts, status, and the last error. That is where you go during an incident. If the counters give you depth but not age, a scheduled shell command that prints the oldest arrival time fills the gap.


Gateway and filter appliances

Edge gateways and filtering appliances almost always publish inbound and outbound queue counts through their SNMP agent (in the vendor's own MIB) or a management API. Walk the device once to find the queue objects. Then poll them like any other SNMP value. A hosted filtering service has no queue you can read; your visibility there comes from the synthetic tests in the latency testing guide.


Tip: a missing reading is not a zero. The script might fail, the SSH session might time out, or the counter might vanish because the service stopped. In those cases, your monitoring should show "no data" and alert on that. It should not quietly plot zero and declare the queue empty. A dead mail service with an "empty" queue graph is exactly how a stuck-mail incident hides all day.


Thresholds and alert design


With depth and age flowing in and a baseline written down, you can design alerts that fire in time. The common mistake is a single alert on depth. Use three signals, each catching what the others miss.


  • Oldest age above the normal drain time. The primary alarm. Pick a value comfortably above the longest age you saw during routine bursts. Require it to hold for two consecutive polls so a single slow destination does not page you. This alert means "mail is measurably late," which is what users care about.
  • Depth above the baseline peak. Set this above the largest routine burst, again with two-poll confirmation. It catches the loop or compromised account, where the queue explodes in minutes and age has not had time to grow.
  • Rate of growth. Alert when depth has risen on every poll for a set number of polls in a row. Also alert when it rises by more than a chosen amount per minute. This is the earliest of the three. A queue that has climbed steadily for ten straight minutes says the next hop stopped accepting long before either absolute threshold is near.

Around those three, add the supporting checks that explain them. Check that the mail transport service is running and free space on the queue volume is above a healthy margin. Check that the relay can resolve an external name. Make the queue alerts depend on those, so that when the disk fills you receive one alert about the disk rather than three about symptoms. Finally, pair the metric with the log. A rule that watches for the deferral reasons in the table above turns a queue alarm into a queue alarm with a diagnosis attached. The log alerting guide shows how to build that side without drowning in noise.


Severity should follow age. A growing queue is a warning worth a look within the hour. An oldest message older than your business would tolerate is the critical alert that interrupts someone. Only you know whether that is thirty minutes or four hours. Route both to whoever owns mail, and make sure the notification path does not depend on the failing relay. Sending "the mail relay is down" through the mail relay is a mistake every administrator makes exactly once.


A worked stuck-mail incident


Here is how the three-signal design plays out. The organization runs a single outbound relay, relay-01, that all users and applications send through. Its baseline: resting depth of two or three, routine peaks around one hundred during the morning batch. Those peaks drain within five minutes. Oldest age is rarely beyond four minutes.


  • 08:41 — rate-of-growth alert. Depth has risen on eight consecutive one-minute polls, from four to ninety-one. Not yet above the baseline peak, and the oldest message is only eight minutes old, but the steady climb is the tell.
  • 08:44 — look at the queue. Every deferred entry, to dozens of different domains, carries the same reason: Host or domain name not found. Name service error. Everything external is failing at once — the DNS fingerprint from the table, not a next-hop outage.
  • 08:46 — confirm from the relay itself. An MX lookup for any external domain from the relay's own shell fails; the same lookup against a public resolver succeeds. The relay's configured resolver is the problem, not the internet.
  • 08:48 — find the cause. The relay has a single resolver configured: an internal server that was rebooted for maintenance just before the alert and is slow coming back. The second internal resolver was never added to the relay's configuration.
  • 08:52 — fix and flush. The administrator adds the second resolver and confirms lookups work. The administrator issues the relay's "retry everything now" command so deferred mail does not wait for its back-off timers. Depth drops from one hundred forty to under ten within minutes; oldest age falls to zero.
  • 09:05 — verify end to end. A synthetic test message sent through the relay arrives at an external mailbox in normal time. Incident closed, customer-visible delay roughly twenty minutes, and no user ever opened a ticket.

Now picture the same morning with no queue alerts at all. The first signal would have been a user at lunchtime asking why a customer never received a quote. That would be followed by an afternoon of log reading, hundreds of hours-old messages trickling out through back-off timers, and apologetic emails. Same root cause, same five-minute fix, very different day.


The follow-up matters as much as the fix. The relay now has two resolvers. The monitoring system gained a resolver-reachability check that the queue alerts depend on. So next time the notification will say "the relay's resolver is down" rather than "the queue is growing." And the incident note records the fingerprint that gave it away, so the next person on call recognizes it in seconds.


Where to go next


The queue is the mailbag at every hop, and the mailbag is where stuck mail lives long before anyone knows it is stuck. Collect depth and oldest age from every relay, gateway, and mailbox server you own. Baseline them for a couple of weeks. Alert on age first, depth second, and rate of growth earliest of all. Make those alerts depend on the service, the disk, and the resolver. Pair them with a log rule that names the cause. The stuck-mail incident then shrinks from an afternoon-long mystery to a footnote.


Queue metrics have one blind spot: they only see the hops you can log into. The rest of the path includes the hosted service, the recipient's gateway, and the return direction. For those, you need the synthetic round-trip test in the next guide. You also need the relay, MX, and gateway checks that keep the path healthy in the first place. All of it lives under Email Flow Monitoring.


Put this into practice with SysPeep Monitor

The two numbers this guide is about — queue depth and oldest age — are exactly the kind of custom metric SysPeep Monitor is built to collect. It can collect them from a script over SSH, from performance counters over WMI, or from an appliance's SNMP agent. Graph them next to the relay's disk and service state. Make the queue alerts depend on both. Let the rate-of-growth rule page you before the users do. The free trial installs on any Windows machine in minutes.

Download Free TrialLearn About SysPeep