Home › Topics › Application Slowness vs. Network Slowness › Working With App Owners

Working With App Owners Instead of Against Them


Somewhere in your organization there is an application owner who thinks your network is slow. You think their application is chatty. You are both partly right, and neither of you can prove it, because your tools do not speak to each other. Their profiler shows the network as a grey box labeled "I/O." Your monitoring shows their server as a green dot that answers pings. Every incident starts with each of you staring at your own half of the picture and waiting for the other to blink. I have sat in both chairs, and the view is equally unhelpful from either.


The politics of that standoff includes presenting findings without starting a war, writing neutrally, and turning a rivalry into a partnership. It is covered in sharing data with app teams without starting a war. This guide is the plumbing underneath the politics: the technical mechanisms that let two teams work on one problem. A vocabulary table so the same words mean the same things. A dashboard that puts both teams' metrics on one clock. A handoff package with what the other side needs and nothing else. This guide also provides a test harness both sides trust, a written format for findings, and a fixed agenda for the joint call. And it provides a way to close the loop after the fix.


In this guide

  • Two teams, one clock
  • A shared vocabulary table
  • Joint dashboards: both layers on one timeline
  • Evidence handoff etiquette
  • A test harness both sides trust
  • Blameless framing, in writing
  • The joint-triage call format
  • Closing the loop after the fix
  • Where to go next

Two teams, one clock


Every mechanism in this guide serves one technical idea. The two teams must look at the same request, in the same time window, on the same clock. They must use words that mean the same thing. Almost every failed cross-team investigation breaks one of those. The app team profiles a request at 10:14 while you graph the WAN link at 10:40. They measure from the data center while you measure from the branch. They say "latency" meaning four seconds and you say "latency" meaning four milliseconds. None of that is bad faith. It is two measurement systems that were never lined up.


Lining them up is mostly agreement in advance, and that is the good news. It can be done on a quiet Tuesday rather than during an outage. Agree on vocabulary once. Build the shared dashboard once. Write the handoff template once. Set up the synthetic check once. After that, an incident is a matter of running a process both sides already know instead of inventing one under pressure.


A shared vocabulary table


Network people and application people use overlapping words for different things, and a surprising number of arguments are two teams agreeing violently in different dialects. The table below is the translation layer. Print it, pin it to the top of the shared incident document, and refer to rows by name when a conversation starts drifting.


Network saysApp saysThey meanWatch out
Latency (round-trip time)Latency (how long the request took)Two different things that share a wordAlways say "round trip" or "response time," never bare "latency"
Round tripCall, requestOne question and its answerOne app "call" may be dozens of network round trips
Packet lossTimeouts, retries, stallsLost packets that TCP recovers after a pauseLoss does not feel like "slower"; it feels like random freezes
Throughput, bandwidth (bits per second)Throughput (requests per second)Different units of "how much"A link can be nearly idle while requests per second is at its ceiling
Connection (a TCP session)Connection (a pooled database handle)Different scopes"Out of connections" in app-speak is pool exhaustion, not a firewall limit
Session (firewall state entry)Session (a logged-in user)Different thingsA firewall session limit hits new TCP connections, not logins
RetransmissionSlow requestA recovery pause inside a requestRetransmissions are visible only in a capture, never in app logs
Time to first byteServer response timeRoughly server think time, plus a round trip for the client viewAgree on which vantage point the number came from
Utilization (percent of a link)Load (CPU, users, queue depth)"How busy," measured on different thingsState which thing
The server (an IP and a port)The service (app, database, cache, queue)One address versus a whole tier"The server is fine" and "the service is slow" can both be true
JitterTail latency, the slowest one percentVariability rather than averageAverages hide both; ask for distributions

Two rows cause most of the damage. "Latency" should simply be banned from cross-team conversation in favor of "round trip" and "response time." And "connection" needs a qualifier every time. An application owner saying "we ran out of connections" is describing a pool inside their process. Meanwhile, a network admin hearing it starts checking firewall session tables. Two words, an hour saved.


Joint dashboards: both layers on one timeline


A joint dashboard is one screen where the network metrics and the application metrics share an x-axis. It is the single most effective technical tool for cross-team work. That is because "our graph is green" stops being an argument when both graphs are on the same screen. The viewer can see, at 10:14, what every layer was doing at once. The build is less about software than about agreement on what goes on it.


RowMetricOwnerWhy it is there
NetworkRound-trip time and loss from the affected site to the application serverYouThe flat line everything else is compared against
NetworkWAN link and server switch port utilization; firewall session countYouShows saturation or a middlebox limit in the same minute the app slowed
ServerCPU, memory, disk queue, established connectionsServer owner (often you)Resource exhaustion lines up with think time, or visibly does not
ApplicationRequests per second, response time (average and slowest percentiles), error rateApp ownerThe user's experience, as the app measures it
ApplicationDatabase query time, pool wait time, garbage-collection pausesApp ownerThe usual residents of long think time, in their own words
BothSynthetic transaction timing, split into segmentsSharedThe one measurement both teams agreed to trust (next sections)
AnnotationsDeployments, configuration changes, network changes, restartsEveryone"Since when" answered visually

Three rules make it work. One time zone, stated on the dashboard, because a one-hour offset between two teams' tools has derailed more investigations than any router. One window, meaning that when the incident says "10:00 to 11:00," every panel shows exactly that. Use one resolution, or at least be aware of it. A five-minute average will hide a thirty-second spike that a ten-second sample shows clearly. And the two teams' tools rarely sample at the same rate. If you cannot integrate the application team's metrics into your dashboard, do not let perfect kill useful. In that case, agree the window, export both sets, and paste them into one document with the timestamps aligned. The point is the shared x-axis, not the software.


At Ashgrove Housing the tenancy system's slowness spent a month as a network ticket and then a month as an application ticket, in that order. The first joint call put both graphs on one timeline. Round trip was flat at 12 ms, application think time was flat at 300 ms, and users were still waiting five seconds. The gap belonged to a reverse proxy in front of the application, on nobody's dashboard. It was doing certificate checks against an address that no longer existed and timing out on every request. Both teams had been right for two months. The proxy has a row on the dashboard now.


Evidence handoff etiquette


A handoff is the moment your investigation becomes their investigation, and the package you send decides whether they start working or start arguing. The etiquette is simple: send exactly what the other team needs to reproduce your reasoning. Use a form they can act on in ten minutes. Send nothing that requires them to defend themselves.


  • The scoping sentence. Who, where, which action, since when, how slow. One sentence at the top.
  • Timestamps with a time zone. The window you measured, and the exact times of the samples you are citing.
  • Client and server pairs. Source and destination addresses, ports, and hostnames for the transaction you measured. "The CRM" is not a pair; 10.20.5.31 to 10.0.0.40:443 is.
  • The decomposition. The six segments of the slow request as a small table, from at least one vantage point. The method is in this topic's response-time decomposition guide.
  • Think time, from two places. The request-received-to-first-byte figure from the affected site and from near the server, with the round trip alongside, as described in server think time.
  • A capture trimmed to the transaction. One TCP conversation, not a day of traffic. Filter it to the client-server pair and the minute in question, note where it was captured, and strip anything unrelated. A forty-kilobyte file they can open is worth more than a four-gigabyte file they will not.
  • What you ruled out, and how. The diagnosis log lines: ping results, link utilization, error counters, the loopback test. This is the part that earns trust, because it shows you audited your own layer first.
  • What you did not check. Honesty about gaps prevents the other team from assuming you covered them.
  • One theory, labeled as a theory. "Pattern and idle CPU suggest a wait on the database; not verified" is helpful. Three theories are noise.

And what to leave out: the full-day capture, the forty screenshots, the history of previous incidents, and any sentence that begins with "as we said." If the recipient has to search the package for the number that matters, the package is too big.


Tip: keep the package as a template with headings already in place, and fill it in during the incident rather than after. Filling it is the same work as investigating; writing it afterwards from memory is a second job nobody does well.


A test harness both sides trust


"Works for me" is the sentence that ends cooperation, and it is usually true. The developer tried it from a desk near the server and it was fast. The antidote is a measurement that neither team runs by hand. A synthetic transaction is a scripted version of the real user action — log in, open the customer record, log out. A probe performs it on a schedule from a fixed location. The timing is recorded and split into segments every time. The transaction is the same action, from the same place, on the same clock, before and after any change, and it is nobody's opinion. The broader topic of building these checks has its own pillar at synthetic transactions. Here the point is what the two teams must agree on for the harness to count as evidence.


  • Which action. The one users complain about, scripted end to end. Not a ping, not a page load of the login screen: pings and logins exercise almost nothing.
  • From where. A probe at the affected site and a probe near the server. The difference between them is the network's contribution, measured continuously.
  • How often. Frequent enough to catch the spiky pattern, gentle enough not to become load itself; every few minutes is a common compromise.
  • What it records. Segments, not just a total, so a regression can be attributed without re-investigating.
  • Where the results live. On the joint dashboard, visible to both teams, with the same time zone as everything else.
  • What "fixed" means. A number, agreed in advance: "customer-open under two seconds from the branch, measured by the harness, for five consecutive business days." Without this, the incident ends in a shrug.

Blameless framing, in writing


The write-up outlives the incident and gets read by people who were not there, so its structure matters more than its tone. The other guide covers tone; here is the structure, which does most of the tone's work for it. Every finding gets the same seven fields. The discipline of filling them keeps verdicts about teams from leaking in, because none of the fields has room for one.


FINDING
Observation:   Opening a customer record takes 7.5-8.2 s from the warehouse.
Measurement:   Time to first byte 7.4 s (browser), 7.5 s (capture, client side);
               round trip 40 ms, 0 loss over 100 pings; WAN link peak 31%.
Vantage point: Warehouse PC 10.20.5.31 to crm 10.0.0.40:443, 09:12-09:48 local.
Rules out:     Loss, saturation, and DNS on the warehouse-to-DC path; client
               (fast from neighbor's PC? no - same result, so client not cleared).
Leaves:        Time between request received and first response byte, at the
               server; also fast-from-loopback not yet tested.
Next test:     Loopback timing on the server; slow-query log for the window.
Owner / by:    App team (loopback + query log), today 14:00; network team retests
               from warehouse after any change.

The format forces every claim to carry a measurement and a vantage point, so it can be checked. "Rules out" is scoped to what was actually tested, and where a test was inconclusive it says so instead of quietly dropping it. "Leaves" describes where the time is, not whose fault it is. The next test is named, owned, and dated. There is no field for "as we told you," and no field for a conclusion about someone else's code. The format simply has nowhere to put them. Use the same seven fields for your own layer's findings, including the ones that convict the network. A format you only use when you are right is a weapon, and everybody can tell.


The joint-triage call format


A joint call without an agenda becomes two teams presenting slides at each other. The format below fits in thirty minutes, assigns roles, and ends with written actions. It works for a first meeting on a new incident and for a follow-up. The only thing that changes is what the evidence section contains.


MinutesSegmentWhat happens
0–3Symptom and windowRead the scoping sentence aloud; agree on the time window everyone will look at. Nobody proposes a cause yet.
3–8Evidence, one screen eachNetwork first: round trip, loss, utilization, think time for the window. Then the app side: response time, errors, query time. One screen per team, no narration beyond the numbers.
8–15Joint dashboard walkScroll the shared timeline through the window together, noting what moved at the same minute and what did not.
15–25Hypotheses, cheapest test firstList candidate causes; for each, name the test that would confirm or kill it and how long it takes. Run anything under five minutes on the call; schedule the rest.
25–30Written next stepsOwner, action, time for each test; who updates the finding document; when the harness will be checked; time of the next call if needed.

Three roles keep it moving. A driver shares one screen for the whole call, so everyone looks at the same thing. A scribe types the finding document live, so nothing has to be reconstructed later. And the two owners — one network, one application — are the only people who commit their teams to actions. One rule protects the clock: nobody debugs live for more than five minutes. If a test needs longer, it gets an owner and a time, and the call moves on. A thirty-minute call that ends with four owned tests beats a two-hour call that ends with a theory.


Closing the loop after the fix


Most cross-team incidents end badly not at the diagnosis but at the close. The app team deploys a change, the ticket goes quiet, and nobody confirms anything until the next complaint. By that point, the whole history has to be rebuilt. Closing the loop is four short steps, and they are the steps that make the next incident faster.


  • Retest with the harness. Same action, same probes, same segments. Put the before-and-after decomposition side by side. If the fix worked, the segment that was the pie's biggest slice has shrunk and the others are unchanged. If it did not, you know that within a day instead of a month.
  • Document it in the finding format. Add a final block: what was found, what was changed, the numbers before and after, and the harness result. Annotate the joint dashboard with the change time.
  • Say thank you, in writing, and copy their manager. This is not politics; it is maintenance of a technical relationship you will need again. Specific thanks ("the query log turned it around in an hour") land better than general ones.
  • Keep the package. The vocabulary table, the dashboard, the harness, and the finding document now exist. The next "the app is slow" starts from them, not from zero.

Customer-open from warehouse (harness, median of 20 runs)
                 before fix      after fix
DNS                 30 ms           30 ms
connect             42 ms           41 ms
TLS                 90 ms           88 ms
think time       7,480 ms          610 ms
transfer           180 ms          175 ms
total            7,822 ms          944 ms

That table is the close. Every network segment is the same before and after. That is the last piece of exoneration your layer will ever need on this incident. The think-time row moved, which is where the fix was. And the harness, not anybody's opinion, produced the numbers. Sometimes the retest shows something else: the fix helped, and now transfer is the biggest slice because the response is still large. That is fine too, because the same process simply runs again with a new biggest slice.


Where to go next


Vocabulary, dashboard, handoff, harness, format, call, close: seven mechanisms that turn two teams with incompatible tools into one investigation. And not one of them needs anybody to blink first. The measurements they carry come from the rest of this topic. There is response-time decomposition for the six segments. There is server think time for the separator. And there is the layered diagnosis workflow for the sequence that produces the diagnosis log. The joint dashboard may show a healthy network and a slow application from one site only. In that case, chatty applications explains the design problem both teams will need to solve together. Everything here starts from the Application Slowness vs. Network Slowness pillar.


Put this into practice with SysPeep Monitor

The network half of a joint dashboard includes round trip and loss to the application server, plus WAN and server-port utilization. It also includes server CPU, memory, and disk over WMI and SNMP. That is exactly what SysPeep Monitor collects and displays from one web console. The history is kept so the incident window is still there when the call starts. The application team brings their own metrics. You bring the flat line. The trial is free.

Download Free TrialLearn About SysPeep