Throughput Testing Done Right: iperf-Style Tests Without Breaking Things
"Copying to the file server is slow." "The backup to the data center never finishes." "Is the new link actually giving us what we ordered?" Every one of those questions is really asking for a number. How many bits per second can this path move, right now, between these two points? Interface graphs tell you how much traffic a link carried; they cannot tell you how much it could carry. For that you have to push traffic through it on purpose and measure what arrives.
That is what an iperf-style throughput test does, and it is one of the most useful and most dangerous tools an administrator owns. Useful, because it produces a hard number in ten seconds. Dangerous, because it gets that number by filling the path completely, and a path filled by you is a path unavailable to everyone else. This guide explains how the tests work in plain language and what TCP and UDP tests each tell you. It shows how to schedule them on production paths without becoming the outage. It explains how to turn occasional tests into a per-path baseline you can lean on when the next "it's slow" ticket arrives.
In this guide
- What an iperf-style test actually does
- TCP tests: what one flow can achieve
- UDP tests: loss and jitter at a target rate
- Single stream versus parallel streams
- Scheduling on production paths without breaking things
- Building a per-path throughput baseline
- Interpreting results: LAN, WAN, or host?
- Where to go next
What an iperf-style test actually does
The idea is simple. One machine runs as a server and waits. Another runs as a client, connects to it, and sends data as fast as it possibly can for a fixed number of seconds. The receiving side counts what arrived and divides by the time. The result is the throughput of everything between the two machines: their network cards, every switch and router in the path, and the machines themselves. Think of it as pouring water into one end of a pipe as fast as you can and holding a bucket at the other end. The bucket tells you the pipe's capacity, not the tap's.
That last point matters. A throughput test measures the narrowest thing on the path, and the narrowest thing is not always a link. A busy virtual machine, an old network card, a duplex mismatch, or a firewall inspecting every packet can each be the bottleneck. Interpreting a result means working out which. The tool itself is free, runs on every operating system you have, and has a handful of options you will use constantly:
# On the far end (the server): listen, handle one test, then exit iperf3 -s -1 # On the near end (the client): send TCP for 10 seconds, report each second iperf3 -c 10.20.0.10 -t 10 -i 1 Connecting to host 10.20.0.10, port 5201 [ 5] local 10.10.0.25 port 51344 connected to 10.20.0.10 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 11.2 MBytes 94.0 Mbits/sec 0 212 KBytes [ 5] 1.00-2.00 sec 11.3 MBytes 94.6 Mbits/sec 0 212 KBytes ... [ 5] 9.00-10.00 sec 11.3 MBytes 94.5 Mbits/sec 0 212 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 113 MBytes 94.5 Mbits/sec 0 sender [ 5] 0.00-10.04 sec 112 MBytes 93.8 Mbits/sec receiver
Read the last two lines first. The sender line is what the client pushed; the receiver line is what actually arrived, and it is the number to record. Here a path over a 100 megabit WAN link delivered about 94 megabits per second, which is what you expect once protocol overhead is subtracted. The Retr column counts TCP retransmissions — packets sent again because they were lost or badly delayed — and zero is the number you want. By default the client sends and the server receives. Add -R to reverse the direction. Always test both, because links and paths are routinely asymmetric.
TCP tests: what one flow can achieve
TCP is the reliable transport that carries file copies, web pages, database connections, and backups. It guarantees delivery by numbering every byte, waiting for acknowledgments, and resending anything that goes missing. A TCP throughput test therefore answers a practical question: how fast can one ordinary connection — one file copy, one backup stream — move data over this path?
The answer is often less than the link speed, for a reason that has nothing to do with bandwidth. TCP can only have a limited amount of data in flight before it must stop and wait for an acknowledgment. That amount is the window. On a long path the acknowledgments take a while to come back, so the sender spends time waiting. A single flow's throughput is capped at roughly the window divided by the round-trip time. With a 64 kilobyte window and a 20 millisecond round trip, that is about 26 megabits per second, no matter how fat the link is. Stretch the round trip to 80 milliseconds and the same window manages about 6.5 megabits per second. Modern operating systems grow the window automatically, often to several megabytes. So this ceiling is mostly a long-distance and satellite problem. But it is the first thing to suspect when a fast link tests slow over a long path.
Loss makes it worse. Every retransmission makes TCP slow down and cautiously speed back up. On a high-latency path even a small fraction of lost packets can halve a single flow's throughput. This is why the Retr column matters as much as the bitrate. A test that shows 40 megabits per second with hundreds of retransmissions is describing a lossy path, not a slow one. And the fix is finding the loss, not buying bandwidth. Useful TCP options:
# Reverse direction: the server sends, the client receives iperf3 -c 10.20.0.10 -t 10 -R # Skip the first 2 seconds (TCP ramp-up) so the average reflects steady state iperf3 -c 10.20.0.10 -t 12 -O 2 # Ask for a larger socket buffer (window) on a long, fat path iperf3 -c 10.20.0.10 -t 10 -w 2M
UDP tests: loss and jitter at a target rate
UDP is the other transport: no acknowledgments, no retransmission, no slowing down. It carries voice, video, and other real-time traffic where a late packet is as useless as a lost one. A UDP test asks a different question from a TCP test. Instead of "how fast can this go?", it asks "if I send at exactly this rate, how much arrives, and how evenly?" You choose the rate with -b. The server reports how many datagrams were lost and the jitter. Jitter is the variation in arrival spacing — the thing that makes voice calls sound like a robot underwater.
# Send UDP at a steady 50 Mbit/s for 10 seconds iperf3 -c 10.20.0.10 -u -b 50M -t 10 [ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams [ 5] 0.00-10.00 sec 59.6 MBytes 50.0 Mbits/sec 0.000 ms 0/43140 (0%) sender [ 5] 0.00-10.02 sec 59.5 MBytes 49.8 Mbits/sec 0.412 ms 86/43140 (0.2%) receiver
Here the path carried 50 megabits per second of UDP with a little loss and under half a millisecond of jitter. That is fine for a data link and worth investigating for one that carries phone calls. Run the same test at a rate closer to the link's capacity and watch loss climb. That shows how the path behaves under pressure, which is exactly what TCP hides by backing off.
The warning that goes with UDP testing is the most important sentence in this guide. Because UDP never backs off, a UDP test at or above the link's capacity floods the link for the whole test. It starves everything else on it, including the phone calls you were trying to protect. Never run a UDP test at a rate you have not deliberately chosen. Never run one at line rate on a production path during business hours. Start well below the link speed and work up in short steps.
Single stream versus parallel streams
A single TCP stream tells you what one connection gets. Real traffic is many connections at once, and a link can carry more than any single connection can achieve. Adding -P 4 opens four streams in parallel and reports each one plus a SUM line. Comparing the single-stream result with the parallel total is one of the most informative things you can do. The relationship tells you where the limit lives:
| Single stream | Four parallel streams (SUM) | What it usually means |
|---|---|---|
| Near link speed | Near link speed | The path is healthy and the link is the limit; there is nothing to fix |
| Well below link speed | Near link speed | A per-flow limit: TCP window versus latency, or per-flow shaping or policing; single copies will always be slow here unless the window grows |
| Well below link speed | About the same as one stream | Something shared caps everything: the link is smaller than you think, a device in the path, or one of the hosts |
| Erratic, with many retransmissions | Erratic, more retransmissions | Loss on the path: duplex mismatch, a bad cable or optic, a congested queue, an overloaded firewall |
Parallel streams also load a path harder, which is both their value and their danger. Four streams saturate a link faster and more completely than one, so they belong to the same off-hours discipline as UDP tests. And more is not better: sixteen streams on a modest link mostly measure how well your host can juggle sockets.
Scheduling on production paths without breaking things
A throughput test is, by design, a short flood. On a lab network that is harmless. Now consider the WAN link that carries the warehouse's order scanners. A ten-second flood at two in the afternoon is ten seconds during which scanners time out, calls break up, and the branch manager learns your name. The discipline below separates a testing program from a series of self-inflicted incidents.
- 1. Check the link first. Look at the interface utilization graph for the path you are about to test. If the link is already busy, your test will both hurt users and produce a meaningless number. Never test against a saturated link during business hours; the result is worthless and the complaints are not.
- 2. Prefer off-hours. Early morning, late evening, or a maintenance window. For sites in other time zones, off-hours means theirs, not yours.
- 3. Keep it short. Ten seconds is enough for a steady-state TCP number; use
-Oto discard the ramp-up if you want precision. Nobody needs a five-minute flood. - 4. Announce it. A line in the operations channel or a change record: which path, when, how long, which direction. Then a spike on someone's graph matches a note instead of opening an incident.
- 5. TCP by default, UDP by intent. TCP backs off when the path is busy and shares reasonably with real traffic. UDP does not. Use UDP only with a chosen rate, well below capacity to begin with.
- 6. One direction at a time, one test at a time. Do not run tests on two paths that share a link simultaneously; you will measure the collision, not the paths.
- 7. Do not leave servers listening. Start the server for the test and stop it afterwards (
-1exits after one client). A permanently open test port is an invitation for someone else's "quick test" to flatten your link.
Tip: Watch the interface graph during the test. If your test reports 60 megabits per second and the link graph shows the interface at 100, the missing 40 was real traffic you were competing with. In that case, your number is pessimistic. If the graph shows 60, the path really does top out there, and you have a bottleneck to find.
Building a per-path throughput baseline
A single test answers today's question. A baseline answers the far more valuable one: "is this path slower than it used to be?" You build one by testing the same paths, with the same parameters, on a schedule, and writing the results down where you will find them again. This is the ongoing internal program. The one-time acceptance test you run when a carrier hands over a new circuit is covered in the guide on circuit throughput testing.
Start by listing the paths that matter, which is not every pair of machines. Three families cover most organizations:
- Site to site. Each branch to head office, and any branch pair that talks directly. This is where carrier problems and misconfigured WAN routers show up.
- Site to data center. Each site to the servers it depends on, including cloud-hosted ones where you can place a test endpoint. This is what users actually experience.
- Host to storage, host to host. Inside the data center: hypervisor to storage, backup server to the systems it backs up, database to application server. These paths are fast, and problems here hide behind "the server is slow."
Test each path in both directions, single-stream TCP plus a four-stream run. For paths that carry voice or video, add a UDP run at a deliberately modest rate. Use identical parameters every time; tests with different durations, window sizes, and stream counts make a pile of numbers, not a baseline. Monthly is a reasonable cadence for WAN paths, plus a run after any change to the path. Changes include new firmware, a new firewall rule, a carrier maintenance, a rerouted circuit. Keep a results log; a spreadsheet is fine, as long as each row records enough to reproduce the test.
| Path | Direction | Test | Expected | Measured | Retr / loss | Note |
|---|---|---|---|---|---|---|
| HQ → Branch North | HQ sends | TCP, 1 stream, 10 s | ~94 Mbit/s | 94.5 Mbit/s | 0 | Baseline run, 06:30 local |
| HQ → Branch North | Branch sends (-R) | TCP, 1 stream, 10 s | ~94 Mbit/s | 93.9 Mbit/s | 0 | Symmetric, as ordered |
| HQ → Branch South | HQ sends | TCP, 1 stream, 10 s | ~47 Mbit/s | 31.2 Mbit/s | 412 | Down from 46.8 last month; retransmissions new |
| HQ → Branch South | HQ sends | TCP, 4 streams, 10 s | ~47 Mbit/s | 33.0 Mbit/s | 1,380 | Parallel does not help: shared cap plus loss |
| Branch South → HQ | UDP 20M, 10 s | UDP, 20 Mbit/s | 0% loss | 19.9 Mbit/s | 1.1% loss, 4.8 ms jitter | Voice complaints from this site match |
| Hypervisor A → Storage | Host sends | TCP, 4 streams, 10 s | ~9.4 Gbit/s | 9.3 Gbit/s | 0 | Healthy |
| Backup srv → DB srv | Backup sends | TCP, 1 stream, 10 s | ~940 Mbit/s | 612 Mbit/s | 0 | No loss; DB server CPU pegged during test |
Two rows in that log are doing real work. Branch South has lost a third of its throughput since last month and started retransmitting. That is a carrier conversation backed by evidence rather than a feeling. And the backup path shows a host limit, not a network one: no retransmissions, but the database server was out of CPU. Neither finding is possible without a previous row to compare against.
Interpreting results: LAN, WAN, or host?
A throughput number is only useful once you know what limited it. Narrow the path and read the retransmission column, in this order:
- Compare with the expected figure. A link's usable TCP throughput is a little under its nominal speed once headers and acknowledgments are paid for. Roughly 94 out of 100 megabits is normal, and 60 is not. Know the ordered speed of every WAN link and the port speed of every host before you test.
- Test a shorter segment. If host A to host B across the WAN is slow, test host A to a machine on its own LAN. Then test across the WAN to a machine at the far site. The segment where the number collapses is the segment to investigate.
- Check the hosts. Look at CPU on both ends during the test. A virtual machine with one busy vCPU, or a host with network offloads disabled, cannot fill a fast link. Confirm negotiated speed and duplex on both network cards and the switch ports facing them. A port stuck at 100 megabits half duplex is a throughput-test classic.
- Read the retransmissions. Zero retransmissions with a low number means a clean but narrow path or a slow host. Many retransmissions mean loss, and loss has a physical cause: a cable, an optic, a mismatched port, an overloaded queue, or a firewall at its limit.
- Look at the graphs for the same minutes. Interface utilization, errors, and discards on every port in the path, plus CPU on any firewall or router the traffic crossed. A device that ran out of CPU while you tested has told you where the bottleneck is.
| Pattern | Likely limit | First place to look |
|---|---|---|
| Slow across the WAN, fast on both LANs, no retransmissions | The WAN link or carrier shaping | Ordered speed, WAN router utilization, carrier ticket with the results log |
| Slow with many retransmissions, any segment | Loss on the path | Interface errors and discards, duplex settings, cables and optics |
| Single stream slow, parallel streams fine, long path | TCP window versus latency | Round-trip time; window tuning on the hosts that need single-stream speed |
| Slow even on the local LAN, one host involved | The host | CPU during the test, NIC speed and offloads, virtual machine resources |
| Slow only when traffic crosses one device | That device | Firewall or router CPU, inspection features, session limits |
Where to go next
An iperf-style test fills a path on purpose and measures what arrives. TCP tells you what one ordinary connection can achieve and, through retransmissions, whether the path is losing packets. UDP tells you how the path behaves at a chosen rate and how much it jitters. The gap between one stream and several tells you whether the limit is per flow or shared. Run tests short, off-hours, announced, and TCP by default. Log every result per path. Read a slow number by narrowing the path until you know whether the LAN, the WAN, or a host is the limit.
Throughput is one dimension of the path; what users experience over it is another. The guides on scripted user journeys and where to run transaction tests from cover the experience side. The guide on alerting on experience shows how to act on it. All of them live under the Synthetic Transactions & End-to-End Testing pillar.
Put this into practice with SysPeep Monitor
A throughput test tells you what a path can do. The interface graphs tell you what it was doing while you tested, and what it does every other hour of the month. SysPeep Monitor discovers your switches, routers, and firewalls. It polls interface utilization, errors, and discards over SNMP, and keeps the history. So a disappointing test number can be checked against the link's real load in the same console. The free trial installs on any Windows machine.
Download Free TrialLearn About SysPeep