The Slowness Evidence Kit: Data to Grab Before the Problem Fades
Network slowness is a perishable crime scene. A dead switch waits for you to examine it; a slow afternoon does not. The backup finishes and the congestion eases. The flaky link steadies, and the overloaded server catches its breath. You are left investigating a problem that no longer exists, armed with the memory of how annoyed everyone was. Then it happens again next Tuesday, and you start from zero.
The way out is a habit borrowed from every discipline that deals with fleeting events: collect evidence first, interpret it later. While the slowness is live, you are the only person who can capture what it actually looks like. That means the timestamps, the numbers, the graphs, and the test results. Twenty minutes of disciplined collection turns “it was bad yesterday, trust me” into a folder of facts that supports diagnosis today, escalation tomorrow, and pattern-hunting next month.
This guide is the packing list. You will learn what to grab, in what order, and how to grab it properly. That includes why timestamps outrank everything and how to run throughput tests that mean something (and what they can never tell you). It includes how to save monitoring graphs before they blur into averages and which live outputs to capture at the command line. And it includes how to organize the pile so that future-you, or the ISP's support engineer, can actually use it.
In this guide
- Evidence first, diagnosis second
- Timestamps: the master key
- Throughput tests done properly — and their limits
- Monitoring graphs at the right resolution
- Live captures: pings, paths, and counters
- Concurrent activity: what else was happening?
- The kit, on one page
- Organizing evidence for diagnosis or escalation
- Where to go next
Evidence first, diagnosis second
It feels backwards. The problem is live, people are waiting, and instinct says fix it now, document later. But consider what each path costs. If you collect for twenty minutes and the problem persists, you have lost nothing. In that case, you now diagnose with data. If you skip collection, dive in, and the problem fades mid-investigation (they so often do), you have lost everything. You have no measurements, no graphs at fine resolution, and no proof it happened at all. Collection is the only step that cannot be done later. That asymmetry is the entire argument.
Evidence also changes your standing with everyone downstream. To your ISP, “the internet felt slow Thursday” is an anecdote to be politely deflected. A timestamped log of latency to their gateway tripling during a specific window is a case they have to answer. Same with application vendors, and same — gently — with the colleague who insists it must be the network. The four-zone tests that acquit or convict each layer (covered in isolating where the slowness lives) only settle arguments if their output was saved.
At Tollgate Foods the warehouse system went slow, and two positions formed at once. The vendor said the network was dropping packets, and the network team said it was not. The first meeting on the subject ran for two hours and ended where it started. Before the second, somebody ran the kit. It included three hundred logged pings to the application server at a steady 2 ms with none lost, and a traceroute. It also included the server's disk graph pinned flat across the whole window, and a one-paragraph summary at the top of the folder. The second meeting took five minutes, most of it finding the projector.
One caution before the list: collecting evidence must not worsen the event. Almost everything below is passive or feather-light. The exceptions — throughput tests, which deliberately load the network — get their own etiquette in a moment.
Timestamps: the master key
Every other piece of evidence is indexed by time. Graphs, logs, change records, backup schedules, the ISP's own records — the only way any of them can be joined together later is a shared clock. That makes the humble timestamp the highest-value, lowest-effort item in the kit, and the one most often skipped.
Capture four times, as precisely as you can get them. Note when the user first noticed the problem, when it was reported, and when you confirmed it yourself. And — do not forget this one — note when it stopped. The end time is the most useful of the four, because whatever caused the slowness often ended at the same moment. Examples are a backup window closing, a transfer completing, or a process dying. An event with a crisp start and end can be laid over every graph you own like a stencil. Write the end time down. Yes, even when it ended by itself. Especially then.
Two habits make timestamps trustworthy. First, note times immediately, not at write-up time, because memory keeps the order of events and loses the minutes. Second, know whose clock you are quoting. Devices with drifting clocks produce logs that cannot be correlated. This is why networks run NTP (Network Time Protocol), the service that keeps every device's clock synced to a common reference. If your switches and servers do not all agree on the time, fixing that is itself one of the best investments in future troubleshooting you can make. And when users report times, anchor them: “before or after your ten o'clock call?” sharpens a vague “mid-morning” wonderfully.
The graduate version of the habit is a running log. It is one scratch file, opened the moment an event starts, where you note each observation and action with the clock time as you go. 10:41 confirmed slow from my desk. 10:44 gateway ping clean. 10:52 WAN graph pinned since ~10:15. It feels like overhead for about two minutes. Then it becomes the spine of the whole investigation. The write-up, the escalation email, and the answer to “how long was it actually broken?” fall straight out of it.
Throughput tests done properly — and their limits
A throughput test (the familiar “speed test” is one kind) measures how much data per second a path can actually carry right now. Done thoughtfully, it is a legitimate piece of evidence. Done as a panicked reflex — one browser speed test on Wi-Fi, screenshot, case closed — it is close to meaningless. Knowing why makes you better at every measurement you take.
What a browser speed test really measures: the path from that machine, over its connection, through your LAN and internet link, to one nearby test server, at that moment. Every link in that chain is part of the result. Run it on Wi-Fi and you are measuring the radio link as much as the network. Run it during the congestion you are investigating and it adds a flood of its own. That both distorts the number and briefly worsens the thing your users are suffering. And no public speed test tells you anything about the path your users actually complain about, the one to your file server or business application. It answers one question only: what can this machine push to that test server right now?
So use throughput tests like a professional:
- Test wired whenever possible, to take the radio out of the equation — or test both and label which was which.
- Test the path that hurts. For internal paths, a timed large-file copy to the relevant server is a crude but honest instrument. Better, a tool like
iperfrun between two of your own machines measures a specific leg of your network with nothing else in the frame. - Run it more than once and keep all the numbers — a single sample of a fluctuating system is a dice roll.
- Record the context with the result: time, machine, wired or wireless, which server, what else was running. A number without context cannot be compared with anything later.
- Compare against normal, not against the contract. A result only means something next to the same test run on a healthy day. That is an argument for running a quick throughput check periodically before trouble, as part of baselining.
Here is the iperf pattern for the internal legs. Run it in server mode on a machine at one end of the path you care about. Point the client end at it, and read the sustained rate:
On the far machine: iperf3 -s On the near machine: iperf3 -c 10.20.0.15 -t 30 [ ID] Interval Transfer Bitrate [ 5] 0.00-30.00 sec 3.29 GBytes 941 Mbits/sec
Thirty seconds, one number, and it measures your wire between your machines: no internet, no test server, nothing in the path you did not put there. Run the same pair on a good day and a bad day and you have a before-and-after that no browser speed test can give you.
Tip: during a live congestion event, hold the heavy tests. A throughput test on a saturated link mostly measures the saturation (informative once!) while piling more load onto suffering users. Capture the passive evidence — graphs, pings, counters — first, and save the load tests for after hours or for confirming a fix.
Monitoring graphs at the right resolution
If you run a monitoring system, it has been quietly collecting evidence all along — interface utilization, errors, device CPU, response times. During a slowness event, two facts about that data matter urgently. First, graphs answer the isolating questions: was the WAN link pinned? Did the server's disk max out? Did latency to the app jump? Pull up the relevant graphs for the complaint window and save exactly what you see. For reading a utilization graph honestly — and why a link can hurt long before it reads full — see how full is too full.
Second, and less obvious: monitoring data ages into mush. To save space, most systems keep fine-grained recent data (a point every minute or five) but progressively average older data into coarser summaries. That process is called rollup or consolidation. A five-minute spike to 100% that explains today's complaint is vivid in today's one-minute data. It becomes a modest bump in next week's hourly averages and vanishes entirely from next month's daily ones. Averages flatten exactly the short, sharp events that cause slowness complaints. The practical rule: export or screenshot the fine-resolution graphs now, while the detail still exists. Capture the complaint window plus a few hours around it, and the same window on a normal day for contrast. Label each capture with device, interface, metric, and time range; a folder of anonymous graph screenshots is a puzzle nobody will solve later.
If you are unsure which graphs matter, this shortlist covers most slowness events. Capture utilization on the WAN or internet interface (both directions — upstream saturation is the classic blind spot). Capture utilization and errors on the uplinks serving the affected area, and CPU and memory on the edge router or firewall. Also capture CPU, memory, and disk activity on the implicated server, and any latency or response-time probes you run. Five families of graphs, one bad window, one normal window each — a complete portrait in ten minutes of clicking.
Live captures: pings, paths, and counters
Some evidence only exists while the problem is happening, and the command line collects it cheaply. The workhorse is a logged, sustained ping: not four packets, but a few hundred over several minutes, saved to a file. It documents latency (the round-trip time), its variation, and any packet loss — the three numbers behind most bad experiences. On Windows:
ping -n 300 192.168.1.1 > ping_gateway.txt ping -n 300 10.20.0.15 > ping_appserver.txt ping -n 300 1.1.1.1 > ping_internet.txt
Run three targets — gateway, key internal server, outside world — from an affected machine. You have captured the shape of the network as the victim saw it. When you read the logs later, look past the average. A steady 2 ms with three lost packets, a clean 2 ms punctuated by spikes into the hundreds, and a uniform sluggish 80 ms are three different diseases that can share an average. The pattern of the bad readings — constant, periodic, or random — is itself a clue worth a sentence in your notes. If loss shows up, the deeper chase continues in the packet loss guides. If it is high round-trip times, troubleshooting high latency takes the diagnosis from here. Alongside the pings, save a traceroute to the slow destination (the path, hop by hop, with per-hop timings) so you have a record of the route the traffic took. Routes can change, and a during-the-problem path is evidence you cannot re-create afterwards.
Two more captures earn their place. Interface counters: switches and routers count errors, discards, and traffic per port. Reading the numbers twice, a few minutes apart, shows what is climbing right now. A single reading cannot make that distinction, since totals accumulate from the day the device booted. Your monitoring system does this continuously, but a manual before-and-after works anywhere. A short packet capture: if you are comfortable with a tool like Wireshark, capture sixty seconds of traffic from an affected machine during the slowness. That preserves the event in perfect microscopic detail for later analysis. Retransmissions, stalls, and slow server replies all leave fingerprints. Do not detour to learn it mid-incident; if it is not already a familiar tool, take the pings and move on.
Concurrent activity: what else was happening?
The final evidence category is context. That means the things that were also true during the window, any of which might be the cause or a clue. Note them while they are checkable: Was a backup, replication job, or big sync running? Were updates being pushed to machines? Was there a company-wide video call, a training session, an office full of visitors on the guest Wi-Fi? Did any change happen recently — config, firmware, a new device, a new application — whether or not it seems related? Who else complained, from where, and about what? (Their reports are scope evidence; the interview technique in the right questions to ask users turns each one into two or three usable facts.)
Context is what elevates a mystery into a pattern. “Slow every weekday at 2 PM” plus “the offsite sync kicks off at 2 PM” is nearly a diagnosis by itself — but only if somebody wrote both halves down.
Give your own team's activity the same scrutiny you give everyone else's. Check the change log, the ticket queue, and the group chat for anything IT did in the day before the window. Examples include a firmware update, a policy push, a new rule on the firewall, or a cable moved during other work. Nobody connects their own Tuesday change to Wednesday's slowness without being prompted. That is precisely why the note gets written down now, while the memory of who touched what is fresh enough to be honest.
The kit, on one page
| Evidence | Why it matters | How perishable? |
|---|---|---|
| Four timestamps (noticed, reported, confirmed, ended) | Indexes every other source | Memory blurs within hours |
| Scope notes (who, what, where) | Aims the whole investigation | Users disperse quickly |
| Logged pings to 3 targets | Latency, variation, loss — as experienced | Only exists during the event |
| Traceroute to the slow destination | Records the path taken | Routes can change |
| Fine-resolution graphs, saved | Shows which resource misbehaved | Rolls up into averages |
| Interface counters, read twice | Shows errors climbing now | Deltas unreadable later |
| Throughput test, labeled | Quantifies capacity right now | Meaningless after recovery |
| Concurrent activity notes | Turns mysteries into patterns | Checkable only near the event |
| Optional: 60-second packet capture | Microscopic record for deep analysis | Only exists during the event |
Organizing evidence for diagnosis or escalation
Evidence you cannot find is evidence you never took. The organizing system does not need to be clever — it needs to exist. Use one folder per event, named by date and a word (2nd-floor-slowness does fine). Label every file inside with what, where, and when. At the top, put one summary file written before you forget. Write a paragraph stating the window, the scope, what was observed, what was measured, and what was ruled out. That paragraph is the piece everyone will actually read. Write it as if the reader knows nothing, because in six months that reader is you.
Tailor the bundle to the audience when escalating. An ISP wants the window in exact times, ping and traceroute logs to their edge from your edge, and your circuit identifiers. They will ignore complaints about your internal apps. An application vendor wants the opposite: transaction timings, evidence the network path was clean (your fast pings during slow transactions are exactly that), and the server's resource graphs. A manager wants the summary paragraph and one graph with the bad window circled. Same folder, three different covers — and every one of them lands harder because it is specific. If the event repeats before you have a diagnosis, do not despair. A second evidence folder to lay beside the first is often the moment the pattern jumps out. That is especially true when you are deciding whether the problem is a one-off breakage or a slow trend. That is the exact question chronic versus acute slowness is built to answer.
Where to go next
Timestamps, scope, pings, paths, graphs, counters, context: collected in that spirit, twenty minutes of evidence work outlives any slow afternoon. An unrecorded test is just another opinion by Friday; a saved one is still a fact next Tuesday, when the problem comes back and finds you ready. Practice the kit on small complaints so it is automatic during big ones. Pair it with the rest of the diagnostic sequence in “The Network Is Slow”: A Diagnostic Playbook, the intake routine that starts the clock and the four-zone isolation the evidence ultimately feeds.
Put this into practice with SysPeep Monitor
The best evidence is the kind that was already being collected before anyone complained. SysPeep Monitor records interface utilization, errors, latency, and server health around the clock over SNMP and WMI. So when slowness strikes, the fine-grained graphs of the bad window (and a normal day to compare) are already on your dashboard. The free trial installs on any Windows machine.
Download Free TrialLearn About SysPeep