Per-Tunnel Throughput: Seeing Inside the Pipes
Your VPN gateway carries five site-to-site tunnels, and its dashboard shows one big healthy number: total traffic, comfortably below capacity. Meanwhile the branch office on tunnel three is calling to say everything crawls. You have no way to argue with them. That is because your one big number is an average — and averages are where individual suffering goes to hide. Four quiet tunnels and one drowning tunnel add up to "everything looks fine."
The fix is to stop watching the VPN as a single pipe and start watching each tunnel as its own pipe. Each needs its own traffic graph, in each direction, with history. Once you have that, a whole family of mysteries becomes readable. You can see which branch actually uses its tunnel and which barely does. You can spot the one transfer that flattens everyone else every afternoon. You can answer "the VPN is slow at 2 pm" with a graph instead of a shrug.
This guide shows you where per-tunnel numbers live. It shows why tunnel interfaces are the friendliest way to get them. It shows how tunnel load relates to the circuit underneath and the crypto hardware behind it. And it shows how to run the conversation when one tunnel starves the rest. No vendor specifics required — every capable gateway exposes some version of what follows.
In this guide
- Why the aggregate view hides trouble
- Where per-tunnel numbers live
- Tunnel interfaces: tunnels as monitorable objects
- The aggregate still matters: crypto load
- The tunnel rides the WAN
- Crypto overhead in plain language
- One tunnel starving the rest
- Building the graphs: a short recipe
- The 2 pm conversation
Why the aggregate view hides trouble
A quick definition to anchor everything: throughput is how much data actually moves through something per second. That is different from capacity, which is how much could. When a gateway reports throughput for the whole VPN, it is summing every tunnel together. A sum answers only one question — is the box as a whole busy? It cannot tell you that tunnel three has been pinned at its practical limit since lunch while tunnels one, two, four, and five idle along. That is because the sum of one screaming pipe and four sleepy ones is a perfectly moderate number.
This matters because almost every real complaint arrives per-site. Users at the branch on tunnel three do not experience your aggregate; they experience their tunnel. If your monitoring cannot see at the same granularity that your users complain at, every ticket starts with an investigation just to locate the problem. And "the VPN is slow" stays unfalsifiable. Per-tunnel visibility is what turns that sentence into "tunnel three is moving ten times its normal traffic; the others are normal." That is the beginning of an answer instead of an argument.
Where per-tunnel numbers live
Gateways expose per-tunnel traffic in two broad shapes, and yours offers at least one. The first shape is per-peer counters. Somewhere in the gateway's status pages, command output, or SNMP tree, there is a table of tunnels. SNMP is the standard question-and-answer protocol monitoring tools use to read numbers off devices. The table has one row per remote peer, with bytes sent and bytes received for each. Your monitoring system polls those counters on a schedule, subtracts the previous reading, and divides by the interval. The result is per-tunnel throughput, graphed over time. That subtract-and-divide dance is the same one used for any interface statistics; the monitoring tool does it for you.
Two honest caveats about counters of this shape. On some platforms the counters are tied to the tunnel's current encryption keys. So on those platforms, they reset to zero whenever the tunnel renegotiates its keys. That produces a sawtooth graph and occasional weird spikes if your tool is not expecting the reset. And counters count only what entered the tunnel machinery. Traffic dropped before reaching the tunnel (by routing or policy) never appears. That is worth remembering the day a tunnel's counter flatlines. A frozen counter with users still trying is evidence, not a glitch — the "lying tunnel" diagnosis in Monitoring Site-to-Site Tunnels covers exactly that.
While you are in the counter business, prefer the 64-bit counters if the device offers both widths. Old-style 32-bit counters can only count up to about 4 GB before wrapping back to zero. At modern speeds that happens absurdly fast. A tunnel moving a gigabit per second wraps a 32-bit byte counter in roughly half a minute. If your polling interval is longer than the wrap time, the arithmetic produces nonsense: impossible spikes, negative rates, graphs that look like modern art. The 64-bit versions of the same counters take longer than your career to wrap, which is the correct amount of time.
And keep your directions straight. Every counter pair is labeled from the gateway's point of view. "Out" on your headquarters gateway is traffic toward the branch, and "in" is traffic coming from it. The two directions routinely tell different stories. A branch that mostly pulls files from headquarters shows heavy out and light in from where you sit. So graph both, separately. Plenty of real problems are one-directional (a saturated branch upload, a one-way blackhole), and a graph that sums the directions hides exactly the asymmetry that would have told you.
Tunnel interfaces: tunnels as monitorable objects
The second shape is nicer to live with: on many gateways, each tunnel can exist as a tunnel interface. That is a virtual network interface, like a port made of software, that all traffic for that tunnel passes through. This is typical of route-based VPN configurations, where you steer traffic into tunnels with ordinary routes. The gift for monitoring is that a tunnel interface shows up in the device's standard interface table, right next to the physical ports. The tunnel interface has the same standard counters every interface has: bytes in, bytes out, errors, up/down state.
That means your monitoring system does not need to know anything special about VPNs to watch it. Whatever your tool already does for switch ports, it can do for each tunnel. That includes auto-discovering them, polling their counters, drawing day/week/month graphs, and alerting on state changes and utilization. Each tunnel is just another interface as far as SNMP is concerned. A discovered interface listing might look like this:
Interface Description Status In (avg) Out (avg) wan1 Internet circuit up 214 Mbps 52 Mbps tunnel.1 Branch-A vpn up 18 Mbps 6 Mbps tunnel.2 Branch-B vpn up 84 Mbps 3 Mbps tunnel.3 Warehouse vpn up 2 Mbps 1 Mbps lan1 Core switch uplink up 171 Mbps 48 Mbps
Suddenly the pipes are visible from the inside: Branch-B is pulling serious traffic while the warehouse barely sips. If your gateway supports route-based tunnels and you have a choice, this is the monitoring-friendly way to build them. If your tunnels are policy-based (no interface, traffic selected by rules), you fall back to per-peer counters — same information, slightly more assembly required. Either way, give each tunnel a human name in its description field; "tunnel.2" tells nobody anything at 2 am.
The aggregate still matters: crypto load
Per-tunnel graphs do not retire the aggregate view; they give it a proper job. All tunnels share one gateway. That gateway has a finite ability to encrypt and decrypt. That work is done by dedicated crypto hardware on larger boxes and by the main CPU on smaller ones. Watch the gateway's CPU and, where the device exposes it, the crypto-engine load, alongside your per-tunnel graphs. The two views separate cleanly during trouble. When one tunnel is the problem, its graph spikes while the others and the CPU stay sane. When the box is the problem, CPU pins and every tunnel degrades together. In that case, each one's throughput sags even though no single tunnel looks dramatic. Same user complaint, opposite fixes — one is a conversation with a branch, the other is a bigger gateway.
The tunnel rides the WAN
Here is the mental model that prevents a whole category of misdiagnosis: a tunnel is not its own circuit. It is encrypted traffic riding your ordinary internet circuit, sharing that circuit with everything else you send — web traffic, cloud backups, the lot. Tunnel throughput is a slice of circuit throughput, and tunnel experience inherits circuit experience. When the circuit saturates or gets lossy, every tunnel on it shows rising latency and retransmissions at the same moment, regardless of how little each tunnel itself is carrying. The circuit's own health — utilization, errors, latency — is a subject of its own, covered properly in WAN Circuit Health: The Metrics That Describe a Line. The point here is to graph tunnel and circuit together, because neither graph makes sense alone.
Remember both ends, too. Tunnel three's traffic rides your headquarters circuit and the branch's little circuit, and the branch end is usually the thinner pipe. A tunnel limited to a modest rate might simply be a branch circuit doing its honest best. Keep each site's circuit capacity written down next to its tunnel graph, so "this tunnel never exceeds its branch's line rate" reads as physics, not as a fault.
The cleanest way to see the relationship is to overlay a through-tunnel latency probe on the circuit's utilization graph. On a healthy day the probe line is flat and boring. As the circuit climbs toward full, the probe's latency starts to climb with it — queues forming at the bottleneck. The two curves rise together like a duet. That picture is worth keeping. It converts "the circuit is at 90%" from an abstract number into "and here is what it is doing to every tunnel's response time." That is the version that wins budget discussions.
Crypto overhead in plain language
One more piece of physics for the file: a tunnel always moves somewhat less useful data than the raw line rate. The reason is packaging. Every packet that enters the tunnel gets sealed inside a new packet — an envelope inside an envelope. The outer envelope has its own headers (the addressing and bookkeeping that gets it across the internet), and those bytes ride the circuit without carrying any of your data. On top of that, the sealing itself — the encryption — costs gateway processing on both ends, which is the crypto load from earlier.
The overhead tax is not flat. Big packets amortize the envelope nicely. Tiny packets — the kind chatty applications and voice traffic send in swarms — pay the tax on every single one. So a tunnel full of small packets delivers noticeably less useful throughput than the same tunnel moving large file transfers. There is also a size limit wrinkle: the envelope makes packets bigger. And a packet that grows past what the path can carry must be split or resent smaller, which adds its own drag. The practical takeaways: expect tunnel throughput visibly below line rate and treat that as normal. And when comparing "circuit says X, tunnel says Y," remember the gap is partly envelopes, not necessarily loss.
One tunnel starving the rest
Now the classic incident this whole setup exists for. All tunnels share the headquarters circuit and the gateway's crypto budget, and nothing in a default configuration stops one tunnel from eating both. The usual villain is not malice but a schedule. It could be a replication job, an off-site backup, or a software deployment pushing images to a branch. It is something big, legitimate, and indifferent to the fact that it is flattening four other sites' afternoon.
On your graphs the signature is unmistakable once you know it. One tunnel's throughput climbs to dominate the circuit, while the other tunnels' throughput drops at the same timestamp. That is not because their users went quiet, but because they lost the bandwidth contest. Latency on through-tunnel probes rises everywhere. A stacked view of all tunnels against the circuit's capacity line makes it readable at a glance: one color swelling, the rest compressed beneath it. That picture also hands you the fix, which is almost never "buy more bandwidth" as a first resort. Name the flow. The tunnel tells you which site; flow data or the far site's own graphs tell you which host. Then either reschedule the flow to off-hours or, if your gateway supports traffic shaping, cap the greedy class so interactive traffic keeps breathing. The graph then verifies the cure the next afternoon — same job, no starvation.
Tip: set a gentle utilization warning on each tunnel — not to page anyone, but to timestamp the moment a tunnel starts running abnormally hot. When the slowness tickets arrive twenty minutes later, the warning is your head start: you already know which pipe and since when.
Building the graphs: a short recipe
If you are starting from nothing, here is the whole setup as a sequence. None of it is hard; the value is in doing all of it, once, deliberately.
- 1. List the tunnels. Every site-to-site tunnel that should exist, one line each, with its far site and purpose — the same expected-tunnel inventory the tunnel-health guide builds.
- 2. Find each tunnel's numbers. Tunnel interface if you have one; per-peer counters if not. Name the interface or check after the site, not the peer address.
- 3. Poll both directions at a steady interval. One to five minutes; use 64-bit counters; let the tool turn deltas into rates.
- 4. Add the context checks. The WAN circuit's utilization at each end, gateway CPU and crypto load, and a through-tunnel latency probe per tunnel.
- 5. Build one page per site and one overview. The site page: that tunnel, both directions, plus probe latency. The overview: all tunnels stacked against the circuit's capacity line.
- 6. Let it run for a few weeks, then write down normal. Typical busy-hour rates and latency per tunnel, noted in the inventory. Baselines are what turn a graph into a verdict.
- 7. Add gentle warnings. Tunnel unusually hot, circuit near capacity, CPU pinned — warnings for context, not pages, tuned so they mark the start of trouble.
The one-page-per-site habit earns its keep the first time a branch manager calls. You open their page while they talk, and the conversation is immediately about the same picture instead of competing impressions. Which brings us to that conversation.
The 2 pm conversation
Finally, the payoff scene. A manager reports: "the VPN is slow every day around 2 pm." Without per-tunnel history this begins a week of vague testing. With it, you pull three graphs for the last several days and read them side by side. They are the complaining site's tunnel, the WAN circuit, and gateway CPU. The pattern almost always falls into one of a few shapes:
| What the graphs show at 2 pm | Verdict | What you do |
|---|---|---|
| Their tunnel pinned at a flat ceiling; others normal; CPU fine | That site is saturating its own tunnel or branch circuit | Find the 2 pm job at that site; reschedule or shape it |
| One other tunnel swollen; circuit near full; everyone's latency up | A neighbor is starving them (and everyone) | Name the greedy flow; reschedule, shape, or isolate it |
| Circuit near capacity but no single tunnel dominant | Ordinary afternoon congestion; the circuit is simply full | Capacity conversation, with the graph as evidence |
| CPU or crypto load pinned; all tunnels sag together | The gateway is the bottleneck | Plan a hardware or design change |
| Everything flat and modest; probes clean | The slowness is not in the tunnel path | Look at the application, server, or the site's local network |
Notice what the table really is: every row is the same three graphs, read at the same timestamp. That is the whole method. The history does the arguing for you — calmly, with timestamps. The last row matters as much as the rest, because "it is not the tunnels, and here is the graph" is how you exit other people's incidents gracefully. Two polling rules keep the history trustworthy. Poll often enough to catch short bursts (a five-minute average can smooth a violent two-minute spike into innocence). Keep enough history to compare this week's 2 pm against last month's. That is because "slow lately" only means something against a baseline.
Where to go next
Seeing inside the pipes is the middle layer of VPN visibility. Beneath it sits the truth-telling layer — is each tunnel actually passing traffic at all — in Monitoring Site-to-Site Tunnels: Up, Down, and Lying. Above it sits the incident layer, the ordered triage of A VPN Troubleshooting Playbook, where these graphs become your evidence. The rest of the pillar lives at VPN & Remote-Access Monitoring. Give every tunnel a name, a graph, and a baseline, and the next "slow at 2 pm" call will be the short kind.
Put this into practice with SysPeep Monitor
If your tunnels exist as interfaces, SysPeep Monitor treats them like any other port. In that case, it auto-discovers them, polls their counters over SNMP, and graphs each pipe's history on dashboards you can pull up mid-phone-call. Add gateway CPU and WAN circuit checks and the whole 2 pm story sits on one screen. The free trial installs on any Windows machine.
Download Free TrialLearn About SysPeep