Teaching Judgment, Not Buttons: Training the Next Admin to Think
At some point someone will sit next to you and you will be expected to turn them into an administrator. You will have a week, or a month, and a monitoring console full of things to show them. The temptation is to show them all of it. Here is where you add a device, here is where you set a threshold, and here is the acknowledge button. By Friday they can drive the tool. By the first outage it turns out that driving the tool was never the job.
The job is judgment: knowing what to look at first and what a graph is telling you. It means knowing whether an alert deserves the action it demands, and when "down" is not actually down. None of that is in a menu. I learned it by being wrong in front of patient people. I have since taught it to others mostly by asking them questions I already knew the answers to and refusing to say so.
By the end of this guide you will have a method for teaching judgment that works in any tool. It uses questions instead of clicks and a daily five-minute drill with one real graph. It includes a set of eight scenario cards you can copy and a way of reviewing someone's reasoning rather than their result. It also includes a lab where being wrong costs nothing. You will also know how to tell when it is working, which is easier than you would think.
In this guide
- Where the button is, and whether to press it
- Why button-training fails
- The method
- Eight scenario cards
- How to know it is working
- The trainee who knew every menu
- The checklist
- Where to go next
Where the button is, and whether to press it
There are two kinds of knowledge in this work, and they are learned in different ways. Button knowledge is knowing where things are: which menu adds a device, which screen holds the threshold, where the acknowledge button sits. Judgment is knowing whether to use them. Should this alert be acknowledged or escalated? Does this device belong in the monitor at all? Is the threshold wrong or the network? The first is learned in a week from the documentation. The second is the job.
Button knowledge is not worthless. You cannot act on a judgment without it, and a trainee who cannot find the graph cannot read it. But it is cheap, the tool's own documentation teaches it better than you will, and it expires the day the tool is replaced. Judgment is expensive to acquire, nobody's documentation teaches it, and it never expires. Spend your time on the expensive thing.
A simple test tells the two apart. Ask the trainee "how do I acknowledge this?" and they will show you. Ask "should I?" and watch what happens next. The tool will always tell you how. It has never once told anyone whether.
Why button-training fails
Button-training fails for two reasons, and the first is that the tool changes. Every hour spent learning where the button is in this console is an hour that vanishes when the vendor is acquired. The guide Surviving vendor churn explains why that is a matter of when. The trainee who learned the tool has to start again. The trainee who learned to ask "what would I check first?" has a question that works in any console, including one they have never seen.
The second reason is that the situation is never the one in the manual. The manual describes one alert at a time, cleanly labelled, with the cause where the label says it is. Real outages arrive as a pile. There are forty alerts, a user on the phone, a graph doing something nobody has seen before, and a manager in the doorway. Nothing in the pile matches a page in the manual. The trainee who only knows pages has nothing to do except click the buttons they know, in the order they learned them.
That is the dangerous outcome of button-training: confidence without competence. It produces a person who can do everything and decide nothing, and under pressure that person does something. The something is usually the button they know best. I once watched a trainee, faced with a slow file server, reboot it, because rebooting was the procedure they had been taught most thoroughly. The file server was slow because a backup was running. The backup then started again from the beginning.
There is a longer-term cost too. Careers in this field are built on judgment, because judgment is what gets someone trusted with the next bigger decision. That mechanism is the subject of the skills and careers pillar. A trainee who learns only buttons is being prepared for a job that will not exist by the time they are good at it.
The method
The method has five parts, and none of them takes more than a few minutes a day once it is running. All five share one rule: the trainee commits to an answer before you reveal one.
Questions instead of clicks
When a trainee brings you a problem, do not show them what to do. Ask "what would you check first, and why?" and wait. Then ask "what would you expect to see if you are right?" and "what would you expect if you are wrong?" Only after they have committed do you look together. The looking is the same either way; the difference is that they now have a prediction to compare it with. And a prediction that turns out wrong teaches more in a minute than a demonstration teaches in an hour. Waiting is the hard part. The silence after "and why?" feels long. Let it.
The daily five-minute drill
Once a day, show the trainee one real graph from yesterday with its title covered and ask "what is this telling you?" Five minutes, no more, and no context beyond what the axes show. They describe the shape, guess what metric it might be, and say what they would look at next. Then you uncover the title and talk about the gap. Rotate the material: a graph one day, an alert email the next, a page of log lines the day after. A month of this builds an eye for shapes that no amount of menu-walking produces. Those shapes are the sawtooth of a memory leak, the flat line of a dead sensor, and the step of a change. The drill is also the cheapest training there is, because the material arrives free every night.
Scenario cards
A scenario card is a described situation with no console in front of anyone. The trainee says what they would look at, in what order, and what they expect to see at each step. You are listening for a method, not an answer: a first hypothesis, the check that would confirm or kill it, and the next hypothesis if it dies. Hypothesis-driven diagnosis is the method by its proper name. The habit of halving the search space means choosing the check that rules out the most. A trainee who can walk a card without a console can walk the console when it matters. The eight cards later in this guide are enough to start with.
Review reasoning, not results
A right answer for a wrong reason is a near-miss, and you should treat it as one. When the trainee solves something, ask "how did you know?" before you say "well done". If the reasoning was sound, say so. If the answer was right and the reasoning was a guess, say that too, gently. The guess will not be right next time, and they need to know which of the two got them there. The same courtesy applies to a wrong answer with sound reasoning: that is a good day, and the trainee should hear it. Blameless reviews describe the same discipline at the scale of an incident. (The person being reviewed is the reasoning, not the trainee. Keep that distinction and they will keep bringing you their mistakes.)
Let them be wrong safely
Judgment is learned by being wrong, and the only cheap place to be wrong is a lab. A small monitoring install on a spare machine, watching a handful of lab devices, lets the trainee break things and watch the monitor react. They can be wrong about what the reaction means without anyone's payroll noticing. The homelab pillar covers building one from generic parts. Runbooks work as training material in the same spirit. The guide to runbooks as training shows how to use them as scripts to walk through rather than instructions to obey.
One lab exercise is worth more than the rest, because it teaches the lesson that trips up almost every new administrator. Absence of data is not the same as "down". On a lab Windows server that the monitor polls over SNMP, block the poller and watch what the console does. On the lab server:
netsh advfirewall firewall add rule name="Lab: hide from poller" ^ dir=in action=block protocol=UDP localport=161 (wait for the next few polls, then ask the trainee what the console shows and what it means) netsh advfirewall firewall delete rule name="Lab: hide from poller"
The server is fine. Its users, if it had any, would notice nothing. The console shows it grey, or unknown, or, in a badly configured tool, down. The question for the trainee is "is the server down?" The good answer is "I don't know yet; the monitor has stopped hearing from it, which is not the same thing". That should be followed by a plan to find out by another route. That distinction is principle five in the ten principles. And it is much easier to learn from a firewall rule than from a night shift.
Tip: keep the lab honest by leaving one thing subtly wrong in it at all times. It could be a threshold with no reason, an alert with no action, or a device with no dependency. Never say which. A trainee who finds it unprompted has just demonstrated the thing you are teaching.
Eight scenario cards
Copy these, put each on its own card or page, and use one a week. Each has the situation, the first question a good administrator asks, and what a good answer sounds like. There is no single right answer; there is a right shape of answer, and the third line describes it.
CARD 1 - The green dashboard
Situation: Users report the intranet is slow. Every tile on the dashboard is green.
First question: What does "green" actually test for this service?
Good answer: Names the check behind the tile, asks whether it does what a user
does, tries the page from a user's segment, and compares the
monitor's vantage point with the users'.
CARD 2 - Forty alerts at once
Situation: Forty device-down alerts arrive within a minute, all from one building.
First question: What do those forty devices have in common?
Good answer: Looks for the shared parent (a switch, an uplink, a power feed),
checks the parent first, and expects one cause. Does not start
with the first alert in the list.
CARD 3 - The disk that is always at 91 per cent
Situation: A disk-space alert fires every night at 02:00 and clears by 03:00.
First question: What happens on that server at 02:00?
Good answer: Suspects a scheduled job, checks the baseline graph, and proposes
fixing the threshold's duration or the job, not acknowledging
the alert every night.
CARD 4 - Silence
Situation: No alerts have arrived for nine days. The network is usually noisier.
First question: Is the monitoring system itself still running?
Good answer: Checks the poller, the last-sample time on a few sensors, and the
mail path. Treats silence as missing data, not as health.
CARD 5 - Latency doubled
Situation: Latency on the head-office link doubled at 14:10 and stayed there.
First question: What changed at 14:10?
Good answer: Looks for a change (configuration, routing, a new scheduled job,
the provider), compares against the baseline and against other
links, and halves the path to find which segment moved.
CARD 6 - The device that was never in the monitor
Situation: A user says the label printer in dispatch is down. No such device
exists in the monitoring system.
First question: Is it on the network, and who else depends on it?
Good answer: Finds it (neighbour table, DHCP lease), checks it directly, fixes
the printer, and then raises the coverage gap as its own item.
CARD 7 - Up for the monitor, down for the users
Situation: The web check for a service is green from the monitoring server.
Two users on the same floor cannot load it.
First question: What is different between the users' path and the monitor's path?
Good answer: Compares vantage points (segment, DNS answer, address family,
firewall), tests from the users' segment, and does not conclude
"user error" until the paths have been shown to be the same.
CARD 8 - The threshold someone wants to raise
Situation: A colleague asks you to raise a server's CPU threshold from 80 to
95 per cent "because it keeps alerting".
First question: Why was it 80, and what does the graph say the server normally does?
Good answer: Finds the written reason (or notes there is none), looks at the
baseline, asks what action the alert demands, and then either
fixes the cause, fixes the duration, or raises it with a reason.
You will notice that the first question on every card is a question about the situation, not about the tool. That is deliberate, and it is the whole lesson. The trainee who asks "what do these forty devices have in common?" will find the answer in any console. The trainee who asks "where is the acknowledge-all button?" will find that too.
How to know it is working
You will know the method is working when the trainee starts asking "why" about your alerts. They do not ask in a challenging way. They ask in the way of someone who has understood that every alert is supposed to have a reason. They have noticed that one of yours does not. This is uncomfortable the first time and it is the best evidence you will get. Other signs, in roughly the order they appear:
- They say "I don't know yet, but I would check the uplink first" instead of guessing an answer.
- They describe a graph by its shape before they read its title.
- They push back on a threshold change and ask for the reason.
- They tell you an alert has no action and suggest making it a report instead.
- They rewrite a runbook because a step in it was wrong, and tell you why.
- They are wrong in interesting ways: a sound method that met an unusual network, rather than a guess.
- They find the thing you left wrong in the lab.
The last sign is the one that decides when they are ready for an on-call rotation. The first on-call rotation covers how to make that first week survivable. The point here is that readiness for it is a judgment milestone, not a button one. Someone who can walk all eight cards and finds the broken thing in the lab is ready. Someone who can add a device in forty seconds is fast.
The trainee who knew every menu
Bellweather Logistics hired a junior administrator who had completed every module of the monitoring tool's own training course. They could navigate to any screen without looking and could add a device in under a minute. In their first real incident, a core switch reboot, they acknowledged thirty-eight alerts in four minutes, which the incident review described as "efficient". Asked afterwards what "down" meant for the warehouse management system, they said "when the tile is red". They could not say which check turned it red or whether the warehouse had actually stopped. The team stopped teaching the tool for a month and taught only questions, one card a week and one graph a day. Six weeks later the same trainee pointed out, unprompted, that the warehouse tile went red whenever the switch rebooted whether or not the warehouse was fine. They asked why nobody had written down what "down" meant. Nobody had. Defining "down" was the next thing the team did, and the trainee wrote the first draft.
The trainee had not become cleverer in six weeks. They had been given a different question to carry around, and the question found the gap that the menus had walked past thirty-eight times.
The checklist
This is for the trainer. Read it at the start of a trainee's first month and again at the end.
- The trainee commits to a first check, and a reason, before you show them anything.
- You ask "what would you expect to see?" before you look together.
- One real graph, alert, or log excerpt is reviewed every working day, for five minutes.
- One scenario card is walked every week, with the tool closed.
- Every solved problem gets "how did you know?" before "well done".
- A right answer for a wrong reason is recorded as a near-miss and discussed.
- A lab exists where the trainee can break things without a ticket.
- The lab has one deliberate fault in it that you have not named.
- The blind-the-poller exercise has been run, and the trainee explained what "unknown" means.
- At least one runbook has been walked as a script rather than followed as an order.
- The trainee has asked "why" about one of your alerts, and you answered honestly.
- Readiness for on-call is being judged on the cards and the lab, not on speed in the console.
Where to go next
Buttons are learned from the manual and lost with the tool. Judgment is learned from questions, graphs, cards, honest reviews, and a lab where mistakes are free. It is kept for a career. The method here costs five minutes a day and one card a week. And its output is an administrator who can sit in front of a console they have never seen and know what to look at first. That is the only kind of administrator a tool change cannot reset.
The questions on the cards are the ten principles turned into situations. And a trainee who has walked all eight has met most of the principles without being lectured on any of them. When everything changes is the checklist that trainee will one day open on their first morning at a new job. Both are part of The Fundamentals That Outlive Every Tool.
Put this into practice with SysPeep Monitor
Judgment is taught with real graphs and real alerts, which means a tool the trainee can safely be let loose on. SysPeep Monitor auto-discovers devices and interfaces and polls them over SNMP, WMI, and SSH from a central Windows service. It shows the graphs, alerts, dependencies, and maps in one web console, so the daily drill has somewhere to come from. The free trial installs on a lab machine as easily as on a production one.
Download Free TrialLearn About SysPeep