Home › Topics › Visualizing Network Health: Maps, Weathermaps, and Heatmaps › Monitoring Heatmaps

Heatmaps: A Hundred Devices or a Month of Latency at a Glance


There is a point at which a map stops working, and it arrives sooner than you expect. A status map with twenty boxes is a picture. With a hundred, it is a crowd, and with three hundred it is a texture. The same is true of line graphs: one is a story, five is a conversation, and thirty is a rug. Somewhere past that point you stop being able to answer "which of these is different?" by looking. That is the question a heatmap was built for.


I came to heatmaps late, after a year of scrolling through ninety access-point graphs one at a time looking for the one that misbehaved. The heatmap showed it in a second, as a row that was a different colour from its neighbours. Then it showed something the graphs never had. The misbehaviour happened at the same hour every day. That second finding was the useful one. It came free with the grid.


By the end of this guide you will be able to say what a heatmap is. You will be able to build the two grids that matter most in monitoring plus one for a big switch stack. You will be able to read one for rows, columns, and diagonals. You will be able to choose a colour scale that does not lie about missing data. And you will be able to turn a spreadsheet export into a heatmap with a script short enough to type. There is a specification template you can copy and a story about a floor full of access points and a microwave.


In this guide

  • What a heatmap is
  • The two grids that matter most
  • The port heatmap
  • How to read one
  • Choosing the colour scale
  • Building one from exported data
  • The heatmap spec template
  • A war story: the twelve-thirty stripe
  • The heatmap checklist
  • Where to go next

What a heatmap is


A heatmap is a grid where each cell's colour is a value. Rows are one kind of thing, columns are another, and the number where they meet is painted rather than printed. Pale means low, dark means high, and the eye does the rest. That is the entire mechanism. Everything that makes a heatmap useful comes from choosing what the rows and columns are.


A heatmap answers "which of many things is unusual, and when?" It is the picture for scale: a hundred devices, a thousand ports, a month of hours. It does not tell you an exact value, and it does not tell you why. When you have found the unusual row, the next step is the line graph for that one thing. The guide choosing the right picture explains where each picture hands off to the next.


A heatmap is a summary, and every cell is a bucket. A cell for "switch 14, Tuesday 09:00" holds one number that stands for the whole hour: an average, a maximum, or a count, depending on what you chose. The bucket hides everything that happened inside it. That is the price of fitting a month on one screen, and it is a fair price as long as you remember you paid it. Rollups and retention covers how monitoring data gets bucketed on its way to disk. A heatmap is what those rollups look like when you paint them.


The two grids that matter most


Two arrangements of rows and columns cover most of what a network administrator needs, and it is worth knowing them by name.


Device by time

Rows are devices, columns are hours or days, and the colour is one metric: CPU, latency, error rate, client count, whatever you are hunting. This grid answers "which device is unusual, and when?" in one glance. A device that is different shows up as a row that is a different colour from the rows around it. A time that is different shows up as a column. Sixty switches over a week is sixty rows by a hundred and sixty-eight columns, and it fits on a screen. Sixty line graphs do not.


Sort the rows so that similar devices are neighbours. Group by site, then by role, then by name, and the rows for one comms room sit together. A pattern that affects one site shows as a block of rows. One that affects every core switch shows as scattered rows that all have the same role. Random row order hides both, and the default in most tools is random, or alphabetical, which for this purpose is the same thing.


Value by time

Rows are ranges of a value, columns are time, and the colour is how many samples fell into each range. Think of a histogram, the bar chart that shows how many measurements landed in each bucket. Then stack one histogram per hour side by side. For latency, the rows might be "under 5 ms," "5 to 10," "10 to 20," "20 to 50," and "over 50." Each hour's column shows where that hour's pings landed. This grid answers "what does the distribution look like over time?", which is the question an average hides.


The latency case is the one to learn on. A line graph of average latency for a WAN link shows a flat line at 12 ms, and the users say it is slow. The value-by-time heatmap shows the same 12 ms as a dark band, and a faint but persistent band at "over 50" that appears every afternoon. The average did not lie; it just had nowhere to put the afternoon. Measuring latency covers where the samples come from, and expected latency baselines covers what the bands should be for links like yours.


The port heatmap


A port heatmap draws a switch the way the switch looks. Rows and columns are the physical port layout, one cell per port, and the colour is utilisation, errors, or power draw. A stack of six forty-eight-port switches is 288 cells, arranged in six blocks of two rows by twenty-four. And a glance shows which ports are busy, which are erroring, and which are dark. The same 288 numbers in a table take four screens of scrolling.


The port heatmap answers two questions the other grids do not. The first is "where is the busy port physically?", because the cell is where the port is, and the engineer with the cable can walk to it. The second is "which ports are unused?", which sounds trivial and is the answer to "do we need another switch?" more often than not. Colour the unused ports a distinct shade and count them before buying hardware. (The number is usually larger than the person asking for the switch expected.)


Errors are the more valuable colour on a port heatmap, because errors are rare and utilisation is not. A grid of 288 ports coloured by utilisation is mostly a pleasant mid-tone. The same grid coloured by input errors per hour is mostly white with two or three dark cells. Those are the bad cables and dying optics that produce the "it is slow sometimes" tickets. Healthy network metric ranges gives the numbers for what "a few errors" means; the grid shows you where they are.


How to read one


Read a heatmap for three shapes: the row, the column, and the diagonal. Each shape is a different kind of problem, and learning to see them is most of the skill.


A row that is different is one device with its own problem. Every other row is pale and one is dark all the way across. That device is always busy, always slow, or always erroring. It has been that way for as long as the grid runs. This is the easy one, and it is also the one that a status map never shows, because the device is up. It is just worse than its neighbours, and "worse than its neighbours" is not a state any map paints.


A column that is different is one time when everything changed. Every row is pale except for a vertical stripe of dark cells at the same hour. Something happened to the whole network at once, or to every device the grid shows. A backup job, a scheduled scan, a shift change, a broadcast storm. The column tells you when. The width of the column tells you how long. And the rows it does and does not touch tell you who was affected. A column that repeats at the same hour every day is a schedule, and somebody wrote that schedule. Seasonal patterns covers the daily and weekly rhythms that are normal. A heatmap is the fastest way to see them and the fastest way to spot the one that is not.


A diagonal is a problem that is moving. Each row goes dark one column later than the row above it. It might be a rolling restart or a firmware upgrade working its way down a list. Or it might be a loop propagating through a ring, or a backup job that runs against each server in turn. Diagonals are rare and worth stopping for, because the slope is the speed of whatever is moving. The direction tells you which way it went. I have seen a diagonal exactly once in a device-by-hour grid, and it was the maintenance window nobody had told the monitoring team about.


Tip: when a heatmap shows nothing, that is a finding. A grid that is uniformly pale for a month says the metric is not where the problem is. Change the metric before you change the grid.


Choosing the colour scale


Use a sequential scale with few steps. Sequential means one hue running from pale to dark, so that darker always means more. The eye reads that ordering without a legend. A rainbow scale, where low is blue and high is red with green and yellow in between, looks like more information and is less. That is because nobody can say without the legend whether yellow is more than green. Five steps is plenty. Beyond seven the steps stop being distinguishable and the grid becomes a gradient, which is pretty and unreadable.


Fix the scale to meaningful boundaries rather than to the data. If the scale stretches to fit whatever the maximum happened to be this week, a quiet week looks alarming and a bad week looks normal. In that case, the same colour means different numbers on two grids side by side. Pin the darkest step to a value that matters (eighty percent utilisation, fifty milliseconds, ten errors an hour), and let anything above it be the darkest step. The grid then reads the same way every week, and a dark cell always means the same thing.


Give "no data" its own colour, and make it nothing like "zero." A cell with no samples is not a cell with a value of zero. It is a cell where nobody measured. Painting it the palest shade of the scale says "this device was fine at this hour" when the truth is "this device was not polled at this hour." Use a grey that is outside the scale, or a hatched pattern, and put it in the legend with the words "no data." A month-long grid with a grey column is a month-long grid that shows you exactly when the collector was down, which is worth knowing. Keeping maps truthful is about that principle across every picture in this pillar.


Building one from exported data


Most monitoring tools draw heatmaps, and if yours does, use that. But the fastest way to understand a heatmap is to build one from an export. And the export is easy: a CSV file. That is comma-separated values, plain text, one row per line. It has three columns: device, hour, and value. Every monitoring tool can produce something like it, and a spreadsheet can produce it from anything else.


The script below reads that file and writes an HTML table with coloured cells, which any browser will open. It uses only the Python standard library. A plotting library would produce a prettier picture, and if you have one installed, use it. The point here is that a heatmap is a table with background colours and nothing more mysterious. The script is illustrative: it assumes the file has a header row and that every hour is a whole number.


# heatmap.py (illustrative): CSV of device,hour,value -> HTML heatmap table
import csv, sys, html
from collections import defaultdict
rows = defaultdict(dict)          # rows[device][hour] = value
hours = set()
with open(sys.argv[1], newline="") as f:
    for device, hour, value in csv.reader(f):
        if device == "device":    # skip the header row
            continue
        rows[device][hour] = float(value)
        hours.add(hour)
hours = sorted(hours, key=int)
# five-step sequential scale, pale to dark; grey is reserved for "no data"
steps = ["#f7f6ea", "#e6e3b9", "#c9c37a", "#9a9455", "#5c5812"]
lo = min(v for d in rows.values() for v in d.values())
hi = max(v for d in rows.values() for v in d.values())

def colour(v):
    if v is None:
        return "#d9d9d9"          # no data: not the same as zero
    i = int((v - lo) / ((hi - lo) or 1) * (len(steps) - 1))
    return steps[i]

out = ["<table>", "<tr><th>device</th>" + "".join("<th>%s</th>" % h for h in hours) + "</tr>"]
for device in sorted(rows):
    cells = []
    for h in hours:
        v = rows[device].get(h)
        label = "" if v is None else "%.0f" % v
        cells.append('<td style="background:%s" title="%s">%s</td>' % (colour(v), v, label))
    out.append("<tr><th>%s</th>%s</tr>" % (html.escape(device), "".join(cells)))
out.append("</table>")
open("heatmap.html", "w").write("\n".join(out))

Run it with python3 heatmap.py export.csv and open heatmap.html. Two things to change once it works. First, replace the lo and hi lines with fixed numbers, so the scale is pinned to boundaries that mean something rather than to this week's extremes. Second, sort the devices by site and role instead of alphabetically, which means adding a column to the export or a lookup to the script. Both changes are five minutes and both are the difference between a grid you can read and a grid you can admire.


The heatmap spec template


Write one of these for every heatmap before you build it, and pin it next to the grid. It forces the question to be said out loud, and it stops the "no data" colour from being forgotten, which is the mistake I have made most often. The example is the access-point grid from the story below.


FieldExample
Question it answersWhich access points have unusual channel utilisation, and at what hour?
GridDevice by time
RowsOne per access point, sorted by building, then floor, then name
ColumnsOne per hour, seven days (168 columns)
Value per cellAverage 2.4 GHz channel utilisation over the hour, percent
ScaleSequential, five steps; darkest step pinned at 70% and above
No-data colourMid grey, hatched, labelled "no data" in the legend
RefreshHourly, after the rollup runs
SourceWireless controller over SNMP, per-AP utilisation counter
Owner and review dateName; first working day of each month

A war story: the twelve-thirty stripe


Harbourview Clinic had a month of tickets about slow Wi-Fi on the second floor, all vague, all around lunchtime. Its network team had spent three weeks reading firmware release notes. The first device-by-hour heatmap of access-point channel utilisation showed a vertical amber stripe at twelve-thirty, every weekday. It ran across exactly the eleven rows that belonged to the second floor. The engineer who built the grid walked to the second floor at twelve-thirty and stood next to the staff kitchen. There, a microwave older than most of the staff was heating soup. It was replaced the following week, and the stripe went with it. The incident record lists the root cause as "kitchen appliance," which is accurate and does not do the month justice.


The graphs had held the same data all along. The stripe was in every one of the eleven access-point graphs, as a small bump at lunchtime. Nobody had noticed it because nobody had looked at eleven graphs side by side with the other seventy. The heatmap did not find new data; it arranged old data so the pattern was the first thing the eye landed on. That is all a heatmap ever does, and it is enough.


The heatmap checklist


Run this list before the grid goes anywhere near a report, and again when you change the metric.


  • The question is written down: which of many things, unusual how, and over what time span.
  • The grid type is chosen: device by time, value by time, or port layout.
  • Rows are sorted so that similar things are neighbours (site, then role, then name).
  • Each cell's value is defined: average, maximum, or count, over a stated bucket size.
  • The colour scale is sequential, with five to seven steps.
  • The darkest step is pinned to a fixed value that matters, not to this week's maximum.
  • "No data" has its own colour, outside the scale, named in the legend.
  • The legend shows the step boundaries in numbers and the bucket size in minutes or hours.
  • The refresh matches the rollup that produces the buckets.
  • Hovering a cell shows the exact value and the time range.
  • The grid has an owner and a review date.

Where to go next


A heatmap is a table with its numbers painted, and its whole value is in the arrangement. Put devices in rows and hours in columns and you can see which device is different and when. Put value ranges in rows and you can see the distribution the average hid. Put ports where the ports are and you can walk to the bad cable. Keep the scale sequential and pinned. Give missing data its own colour. And sort the rows so neighbours are neighbours. Then the pattern will find you.


A heatmap tells you which thing and when; the pictures earlier in this pillar tell you where. The status map shows what is down. And the weathermap shows which link is busy. And keeping maps truthful is the discipline that keeps all three, and this grid, honest about what they do not know. They live together under Visualizing Network Health: Maps, Weathermaps, and Heatmaps.


Put this into practice with SysPeep Monitor

A heatmap needs one number per device per hour, and that means polling that runs reliably across the whole estate. SysPeep Monitor auto-discovers devices and interfaces. It collects metrics over SNMP, WMI, and SSH from a central Windows service with optional remote probes. It shows the results on live dashboards and maps in one web console. It has reports you can export. The free trial takes an afternoon to install.

Download Free TrialLearn About SysPeep