The Polling Load Budget: What Monitoring Costs Your Network
There is a small irony at the heart of monitoring. The system you installed to watch your network is itself traffic on that network, CPU load on your devices, and work for a server somewhere. Every check is a real packet that a real switch has to answer with its modest little management processor. Most of the time this cost is trivial — monitoring is one of the cheapest tenants on a healthy LAN. But "usually trivial" is not "always trivial." When polling load does become a problem, it is a sneaky one: the mystery CPU spike nobody suspects. After all, who suspects the smoke detector of starting fires?
The fix is not to monitor less out of vague guilt. The fix is to know your numbers. Polling load obeys simple arithmetic — multiply three numbers, divide by one. Once you can compute it, you can budget it. You can spend generously where speed pays and spend nothing where it does not. You can prove to anyone who asks that your monitoring costs less than one user watching cat videos.
This guide walks the whole ledger. It covers the polls-per-second math and the three places the cost actually lands (the polled device, the poller, and the WAN). It explains how to keep polling cycles from stampeding and how to measure what your monitoring really consumes. It also covers the priority order for when the budget runs out. It ends with a worked budget for a real-sized network and a worksheet you can fill in for yours.
In this guide
- The arithmetic, made friendly
- What the polled device pays
- What the poller pays
- What the WAN pays
- Spreading and jitter: avoiding the thundering herd
- Measuring your actual load
- When the budget runs out: the priority order
- A worked budget, end to end
- The budget worksheet
The arithmetic, made friendly
One formula runs this whole article: devices × metrics per device ÷ interval = polls per second. A "poll" here is one question-and-answer exchange — one SNMP request for an interface counter, one WMI query for a disk, one ping. Count how many values you collect per polling cycle and divide by the cycle length in seconds. And you have the sustained rate of questions your monitoring system is asking the world.
60 devices x 40 metrics each = 2,400 values per cycle 2,400 values / 60 seconds (1-minute interval) = 40 polls per second
Forty polls per second sounds industrious. Now price each poll. A simple SNMP get is one small UDP packet out and one small packet back — on the order of a couple hundred bytes each. So call a round trip somewhere near half a kilobyte on the wire (tools that fetch several values per request do better per value). Forty polls per second is therefore roughly 20 KB/s of traffic — about 0.16 Mbps. On a gigabit LAN that is a rounding error on a rounding error. This is the first honest conclusion of the math: on a local network, polling bandwidth is almost never your problem. The interesting costs are elsewhere — which is exactly where the next three sections go. But keep the formula handy, because every "elsewhere" is this same number wearing a different hat. The rate of 40 polls per second is also 40 answers per second that devices must compute. It is 40 results per second the poller must schedule and store. And, across a WAN link, that rate means bytes that are suddenly not free.
One refinement before moving on: not all polls cost the same. A ping is the cheapest question in networking — a tiny packet the device answers almost reflexively. An SNMP get is nearly as cheap. A WMI query (the Windows management interface) is heavier. It rides on a chattier protocol, sets up more state per conversation, and costs noticeably more on both ends per value collected. An SSH-based check is heavier still — a full encrypted login session to run a command. None of this changes the formula; it changes the exchange rate. A budget of "40 polls per second" is comfortable when it is mostly SNMP. It is worth a second look when a big slice of it is WMI or SSH. When you have a choice of collector for the same metric, the lighter one is usually the right default.
What the polled device pays
Answering your questions is work, and it is work done by the least glamorous part of a network device: the management CPU. A switch might shovel traffic through specialized hardware at line rate all day. Yet it answers SNMP queries with a small, busy general-purpose processor. That processor is also running spanning tree, logging, and everything else on the control plane. SNMP answers rank low on its priority list — which is correct design, but it means a modest device can be visibly loaded by an enthusiastic poller.
The classic symptom is the sawtooth. A device's CPU graph shows a neat spike every N minutes, perfectly aligned with your polling cycle. That is because each cycle delivers a burst of dozens of questions at once. Usually the spike is harmless. It stops being harmless when the device is old, tiny, or busy — or when the questions are expensive. The single most expensive question is the full table walk. Asking a device to enumerate an entire table (every route, every ARP entry, every interface with all its counters) makes it iterate through potentially thousands of entries. A scheduled full walk every five minutes is how monitoring becomes the mystery load. The guide on troubleshooting SNMP covers the slow-walk problem from the diagnostic side. From the budgeting side, the rule is simply: walk tables during discovery, poll individual values during monitoring.
Beyond that, per-device politeness is three habits. Spread your questions: forty metrics from one switch as one burst per cycle is a spike; trickled across the cycle, it is background hum. Many tools do this automatically; it is worth knowing whether yours does. Use bulk operations: SNMP's GetBulk fetches many values in one exchange, cutting both packets and processing per value — prefer it where offered. Ask only for what you use: every metric nobody graphs or alerts on is pure cost. A device answering ten pointed questions a minute is fine; a device being interrogated like it owes you money is a support ticket waiting to happen.
What the poller pays
Your monitoring server runs a scheduling engine. Every check has a next-due time. Worker threads pick due checks, send the queries, wait for answers, and store the results. Healthy polling is mostly waiting, so the server handles surprising volumes — until the waiting goes wrong.
The expensive event on the poller is not the answer; it is the timeout. A healthy device answers in milliseconds and releases the worker instantly. A dead or unreachable device holds that worker for the full timeout, then for each retry. A three-second timeout with two retries ties up a worker for around nine seconds to learn precisely nothing. A handful of unreachable devices can quietly consume more poller time than a hundred healthy ones. This is why a monitoring server that sailed through normal operations can struggle during a big outage, exactly when you need it most. Suddenly half its workers are standing at doors nobody will ever answer.
The failure mode to know by name is the overrunning cycle: the round of checks takes longer than the interval that scheduled it. A five-minute cycle that needs six minutes never catches up; checks start late, run later, and the backlog compounds. The symptoms are recognizable once you have seen them. There are gaps in graphs where samples should be and sample timestamps drifting late. There are "check skipped" or queue-depth warnings in the tool's own logs. If graphs from many unrelated devices all develop holes at the same times, do not blame the network first. Ask whether your poller is falling behind. The remedies are the budget levers in the priority-order section below, plus one poller-specific one. Keep timeouts tight and retries modest. And let dependency logic pause checks behind a dead router instead of timing out on thirty of its children individually.
What the WAN pays
Everything changes at the branch office. The 20 KB/s that vanished into your LAN's noise floor is real money on a thin site-to-site link. Polling remote devices from headquarters sends every question and answer across that link, all day, forever. The math is the same formula scoped to one site. If the branch has 12 devices × 20 metrics on a 1-minute interval, that is 240 values per minute crossing the WAN, roughly 2 KB/s. That is harmless on fiber, noticeable on a saturated tunnel, and genuinely rude over a metered LTE backup link. Latency compounds it. Each answer takes longer to arrive, checks hold workers longer, and timeouts (there are always more across a WAN) cost their full price at distance.
The architectural answer is to stop shipping questions across the WAN and ship results instead. A small remote probe or collector at the site polls its local devices over the LAN and forwards compact results. And, helpfully, it keeps watching even when the WAN link itself is down. That is the one outage headquarters-based polling can never see past. Sizing and placing probes is its own topic, covered in the Scaling Monitoring pillar. For budget purposes, just remember that WAN polling cost is optional cost.
Spreading and jitter: avoiding the thundering herd
Load has a rate and a shape, and shape is where schedules go wrong. Left to their defaults, scheduled systems synchronize. Every 5-minute check fires at :00 and :05, and the hourly jobs pile onto :00. Your monitoring becomes a metronome — silence, then a stampede, sixty times an hour. The stampede is the thundering herd. Poller CPU spikes, hundreds of devices get their question-burst simultaneously, and WAN links hiccup in unison. And any device that was going to struggle, struggles now. Worse, the herd pollutes your data. Latency measured during your own self-inflicted burst reads high. You end up investigating phantom slowness that is really just your own reflection.
And the herd is not only yours. The top of the hour is when everyone else's scheduled work fires too — backup jobs, scheduled tasks, report generation, log rotation. A latency check that runs at exactly :00 is measuring the network at its artificially worst minute, every hour, and dutifully recording that the network is "spiky."
The fix is deliberate desynchronization. Stagger start times so checks with the same interval are distributed across it. Five hundred 5-minute checks should fire as a steady ~1.7 checks per second, not as five hundred at :00. Jitter adds a small random offset each round so schedules that drift into alignment drift back out. Most monitoring engines do some of this automatically. The way to find out is to look at your poller's CPU and network graphs at one-minute resolution or finer. A smooth plateau means spreading is working. A picket fence of spikes means everything is marching in step. If your tool exposes scheduling spread or jitter settings, turning them on is the cheapest load win in this entire article: same questions, same answers, no stampede.
Measuring your actual load
Everything so far is estimation. The good news: you own a measurement instrument, and it is the monitoring system itself. Monitoring the monitor is a small discipline of its own, and its load-related core is three habits.
- Watch the poller's own resource graphs. Monitor the monitoring server like any other important server: CPU, memory, disk I/O and disk fill (metric databases grow). Also monitor the tool's self-reported stats — checks per second, queue depth, cycle duration — if it exposes them. Cycle duration creeping toward the interval length is your overrun early warning.
- Count its traffic. The poller's own network interface counters tell you exactly what monitoring sends and receives. Compare the observed rate against your formula's prediction. For a closer look, a mirror/SPAN port on the switch will show you the polling conversation packet by packet. Sorting by destination also reveals which devices you question most, which is often a surprise.
- Review load when you add load. Onboarding forty new devices or enabling a chatty new check template is a load event. Note the before, apply the change, compare the after. Thirty seconds of looking, and load growth stays a decision instead of a discovery.
Tip: record your polls-per-second number and cycle duration somewhere when the system is healthy. When something later feels off, "we normally run 45 polls/sec with 70-second cycles" turns a vague suspicion into a ten-second comparison.
When the budget runs out: the priority order
Someday the numbers stop fitting: cycles overrun, a WAN site complains, the poller's CPU flattens at the top. There is a correct order for spending your way out, and it is cheapest-first:
- 1. Lengthen Tier-3 intervals. The bulk of most estates is low-stakes devices polled faster than anyone needs. Moving them from 5 to 15 minutes cuts their load by two-thirds and costs approximately nothing. Nobody was reacting to a lab switch in five minutes anyway.
- 2. Trim hoarded metrics. Every estate accumulates checks nobody reads: metrics with no alert, no dashboard, no question they answer. Deleting them is free capacity. The Monitoring Scope pillar covers how to prune without losing anything you will miss.
- 3. Spread the schedule. If load is spiky rather than high, staggering and jitter buy headroom without dropping a single check.
- 4. Then hardware. Only after the free levers: more cores and faster storage for the poller, or moving its database to a beefier backend. Buying hardware to sustain unread metrics at unneeded speeds is the expensive way to avoid three cheap decisions.
- 5. Then architecture. Remote probes take WAN sites off the central budget; additional pollers split the estate. And if some system genuinely needs second-by-second visibility, polling is the wrong instrument entirely. In that case, streaming telemetry, where devices push readings continuously instead of answering questions, is the endgame for high-cadence needs.
A worked budget, end to end
Here is the arithmetic run honestly for a 120-device shop that tiered its intervals the way the interval guide recommends. There are 10 Tier-1 devices watched closely, 40 Tier-2 devices watched normally, and 70 Tier-3 devices watched economically.
Tier 1: 10 devices x 20 metrics @ 1 min = 200 values/cycle = 3.3 polls/sec
Tier 2: 40 devices x 15 metrics @ 5 min = 600 values/cycle = 2.0 polls/sec
Tier 3: 70 devices x 8 metrics @ 15 min = 560 values/cycle = 0.6 polls/sec
Total ~ 6 polls/sec
Wire cost: 6 polls/sec x ~0.5 KB round trip = ~3 KB/s (~0.025 Mbps)
Daily samples: 288,000 + 172,800 + 53,760 = ~515,000 samples/day
Six polls per second. A device answers a question every few minutes on average, and the poller idles. And the whole conversation would fit through a dial-up modem with room to spare. Compare the flat-rate alternative: all 120 devices × all metrics at 1 minute would run over 40 polls per second and about 2.4 million samples a day. That is seven times the load for almost no additional insight, since the extra speed lands on disks and lab gear that did not need it. The half-million samples a day are the one number that keeps growing after the packets are forgotten. Where those samples go, and for how long, is the retention story told in rollups and retention.
The budget worksheet
To budget your own estate, fill one row per tier or check class and add it up. Everything in this article reduces to this table:
| Line item | How to get it | Example (above) |
|---|---|---|
| Values per cycle, per tier | Devices × metrics per device | 200 / 600 / 560 |
| Polls per second, per tier | Values ÷ interval in seconds | 3.3 / 2.0 / 0.6 |
| Total polls per second | Sum of tiers | ~6 |
| Wire cost | Polls/sec × ~0.5 KB | ~3 KB/s |
| Per-WAN-site cost | Same formula, that site's checks only | compute per site |
| Samples per day | Values per cycle × cycles per day, summed | ~515,000 |
| Poller headroom | Cycle duration vs interval; CPU at peak | watch the trend |
Run it once and keep it with your documentation; rerun it when the estate grows. The point of the exercise is calm: monitoring load is real, bounded, and yours to spend on purpose. For the rest of the granularity story — what your intervals can and cannot see, and how long the samples live — head back to the Polling Intervals & Data Granularity pillar.
Put this into practice with SysPeep Monitor
If your budget math says the WAN is paying too much, the fix is architectural. SysPeep Monitor runs as a central service on one Windows machine. It has optional lightweight remote probes that poll branch sites locally. Its dashboards make watching the monitor's own load a one-glance habit. The free trial is at /download/.
Download Free TrialLearn About SysPeep