Monitoring BMCs: iLO, iDRAC, IPMI and Friends
Somewhere in your server room right now, a fan is spinning slightly slower than it should. Or a power supply is running on its last legs, or a disk is quietly logging the early symptoms of its own death. The operating system on that server has no idea. Your users have no idea. And unless something is reading the hardware's own diagnostics, you have no idea either. That lasts until the day the second fan gives up, the server cooks itself, and a perfectly predictable failure becomes a surprise outage.
The something that should be reading those diagnostics is the BMC — the baseboard management controller, a small independent computer built into nearly every business-grade server. You may know it by a vendor name like iLO or iDRAC, or through the standard protocol many of them speak, IPMI (Intelligent Platform Management Interface). Whatever the badge says, it is the same wonderful idea: a tiny machine whose only job is to watch, report on, and control the big machine it lives inside.
This guide shows you how to put BMCs to work. You will learn what a BMC actually is and the hardware health data it can feed into your monitoring. You will learn the two distinct directions of BMC monitoring (watching through it and watching it). You will learn how the common polling methods behave and how to secure BMCs without cutting off your own view. And you will learn how to avoid the classic tragedy of the BMC that turns out to be dead on the one day you need it.
In this guide
- The computer inside the computer
- What the BMC can tell you
- Two directions: monitoring via the BMC, and monitoring the BMC
- How the polling actually works
- What to alert on
- Securing BMCs while you watch them
- The dead-BMC problem
- Where to go next
The computer inside the computer
A BMC is a genuinely separate computer. It has its own processor, its own memory, its own firmware (the built-in software it runs), and its own network connection. All live on the server's motherboard. The BMC powers up the moment the server is plugged into the wall, and it keeps running whether the server is booted, crashed, mid-BIOS-update, or switched off entirely. The only way to kill a BMC is to remove power completely — which is exactly what makes it the perfect crisis door and the perfect health reporter. When the operating system has frozen so hard it cannot even log its own death, the BMC is still calmly answering questions.
Through the BMC you can typically do four crisis-grade things. You can see the server's screen and type at it as if you were standing in front of it (remote console). You can power the machine off, on, or through a hard reset. You can watch the boot process from the very first firmware messages. And you can read the hardware's sensors and event history. In quieter times, that same sensor access is what turns the BMC into a monitoring goldmine.
Two everyday superpowers deserve a mention before we get to monitoring. First, there is remote power-on. After a building power cut, the servers that did not come back on their own can be started from your desk instead of a drive across town. Second, virtual media — most BMCs let you present a disk image over the network as if you had walked up and plugged in an installation drive. That means you can rebuild a machine with a scrambled boot disk without touching it. Neither of these is monitoring, strictly speaking, but both are reasons the BMC has to be alive and reachable — which very much is.
One physical detail matters more than it seems: how the BMC reaches the network. Most servers offer a choice. One option is a dedicated management port — a separate Ethernet jack wired only to the BMC. The other is a shared mode, where the BMC piggybacks on one of the server's normal network ports. Shared mode saves a cable and a switch port. The price is that some of the failures you bought the BMC for (a dead network card, a misconfigured link) can take the BMC's connectivity down with them. Use the dedicated port where it exists, and cable it to your management network. The reasoning lives in The Management Plane: Your Network's Service Entrance.
What the BMC can tell you
The BMC sits closer to the hardware than any agent or operating system ever will, and it sees things nothing else sees. The catalogue varies by server, but the core set is remarkably consistent:
- Temperatures — intake (ambient) air, CPU, memory, sometimes individual components. The intake reading doubles as a free room thermometer. A rising inlet temperature across every server in a rack is your air conditioning failing, reported by machines that have no idea they are doing environmental monitoring.
- Fan speeds — each fan's actual rotation. A failed fan is obvious; more useful is the pattern where the remaining fans spin faster to compensate, which is the sound of your redundancy quietly being spent.
- Power supply state — presence, health, and whether redundancy is intact. A server with two power supplies runs happily on one; the whole point of monitoring is learning about the first failure while it is still boring.
- Disk pre-failure warnings — on servers with hardware RAID (multiple disks acting as one fault-tolerant volume), the BMC or its controller often reports a drive as "predicted to fail" before it actually does. That prediction is based on the drive's own self-diagnostics. This is the difference between a scheduled hot-swap and a degraded array.
- Memory errors — server memory can correct small errors on the fly (ECC, error-correcting code). A rising correction rate on one module is a strong hint about which stick to replace before it graduates to crashing the host.
- The hardware event log — a timestamped diary of hardware events (often called the SEL, system event log): power losses, over-temperature moments, component failures. It survives reboots and OS reinstalls, which makes it the closest thing a server has to a flight recorder.
- Power draw — many BMCs report the server's actual electrical consumption in watts. Graphed over time, it is free capacity-planning data for your UPS and rack circuits. A sudden change can flag a hardware problem or a runaway workload before anything else does.
Here is what a typical sensor query against a BMC looks like — the exact format varies by tool, but the shape is always name, reading, and a health verdict:
# sensor query against the BMC at 10.20.0.11 Inlet Temp | 24 C | ok CPU1 Temp | 47 C | ok Fan1A | 5880 RPM | ok Fan2B | 0 RPM | CRITICAL PSU1 Status | ok | ok PSU2 Status | failed | CRITICAL DIMM B2 Status | correctable errors rising | warning Drive Bay 4 | predictive failure | warning
Read that sample the way your monitoring should: this server is up, serving users, and completely fine from the operating system's point of view. And it has a dead fan, a dead power supply, a sick memory module, and a dying disk. Every one of those is a cheap fix today and an outage next month. That is the entire value proposition of BMC monitoring in one code block.
Two directions: monitoring via the BMC, and monitoring the BMC
BMC monitoring is really two jobs that people constantly blur together, so let us pull them apart. Monitoring via the BMC means using it as a data source. You poll its sensors so hardware health flows into your monitoring system alongside CPU, disk, and service checks. Monitoring the BMC means treating the BMC itself as a device to watch: is the service entrance reachable, responsive, and healthy? The first job makes your monitoring smarter. The second makes your crisis plan real. You want both, and they fail independently. A BMC can be happily reporting sensors in shared-port mode and still be unreachable the day the server's network card dies.
| Monitoring via the BMC | Monitoring the BMC | |
|---|---|---|
| Question asked | Is the server's hardware healthy? | Is my emergency door open? |
| Data collected | Temperatures, fans, PSUs, disks, event log | Reachability, response time, login works, its own health |
| Failure it catches | Hardware dying while the OS looks fine | The crisis path dying while nobody needs it (yet) |
| When it pays off | Every week, in small prevented outages | One terrible night, all at once |
There is one more argument for doing both jobs in your central monitoring rather than in per-vendor tools. Server vendors each ship their own management console. A mixed fleet quickly means three dashboards, three alert formats, and three places for a warning to go unread. Hardware alerts earn their keep when they land in the same pipeline as everything else — same dashboard, same notification rules, same on-call flow. That way, a dying power supply is triaged exactly like a full disk instead of languishing in a console nobody opens.
How the polling actually works
Monitoring systems talk to BMCs in three broad ways. You do not need to memorize commands here. You need to recognize the behavior of each style so you can pick what your servers and your monitoring both support.
SNMP-style polling
Many BMCs speak SNMP (Simple Network Management Protocol), the same question-and-answer protocol your switches use. The monitoring system asks for a numbered reading, and the BMC answers with a value. If your monitoring already polls network gear, adding BMCs this way feels familiar — same scheduler, same threshold logic. The catch is that vendors organize their sensor numbering differently. So expect a little initial fiddling to map which numbers mean "inlet temperature" on which server family.
IPMI-style queries
IPMI is the classic management protocol built specifically for this job. A query returns the whole sensor table — names, readings, and thresholds the hardware itself declares — in one go, like the sample above. Its strength is that the health verdicts come from the vendor's own limits rather than numbers you guessed. Its weakness is age: older IPMI implementations have a poor security reputation, which is one more reason BMCs belong on an isolated network.
API-style polling
Newer BMCs also offer an API (application programming interface) over HTTPS. The monitoring system requests a structured document and gets back the server's health as neatly labeled fields, human-readable names included. Where both ends support it, this is the most pleasant option. It is encrypted by default, consistent across vendors that follow the common standard, and rich enough to include things the older methods never exposed.
Which to use? The honest answer: whichever pairing your servers and monitoring system both speak fluently. A sensor feed you actually set up beats a theoretically better one you never finish. Poll somewhere between every one and five minutes for sensors; the hardware event log can be checked less often, but alert on any new critical entry.
What to alert on
A BMC exposes dozens of sensors; alert on the handful that predict or announce real trouble, and let the rest be data you graph. Starting points:
| Signal | What it usually means | Starting rule |
|---|---|---|
| Inlet temperature rising | Room cooling problem | Warn above your room's normal band; alert 5 degrees above that |
| Any fan at 0 RPM | Failed fan, redundancy spent | Alert; replace promptly |
| PSU failed / redundancy lost | One supply (or feed) down | Alert; server now has one life left |
| Drive predictive failure | Disk announcing retirement | Alert; swap while the array is healthy |
| Correctable memory errors rising | Memory module going bad | Warn; schedule replacement |
| New critical event-log entry | Hardware logged something serious | Alert and read it |
| BMC unreachable | Your crisis door is closed | Alert after a few failed checks |
Notice the last row: the BMC's own reachability sits in the same table as the hardware alerts. That is deliberate. The service entrance is part of the server's health, not an optional extra.
Securing BMCs while you watch them
A BMC is the most powerful login on the server. It can watch the screen, type keystrokes, and pull the virtual power cord, all beneath the operating system's security. That makes it a target, and it means monitoring and security have to be designed together rather than traded off. The checklist:
- Isolate the network. BMCs belong on a management network that user traffic and the internet cannot reach. Use a dedicated VLAN at minimum, reachable only from admin machines and your monitoring system. Never, under any circumstances, expose a BMC to the internet. (Design details: A Management Network You Can Trust.)
- Kill the default credentials. Factory-default BMC passwords are the oldest unlocked door in the industry. Set strong, unique credentials per server, stored in your password manager — with an offline copy for outages.
- Give monitoring its own account. Create a read-only account for polling wherever the BMC supports roles. Your monitoring needs to read sensors, not power-cycle servers; if the monitoring host is ever compromised, the blast radius stays small.
- Keep the firmware current. BMCs run software, software has flaws, and vendors ship fixes. Treat BMC firmware like any other patching obligation — the Firmware & Lifecycle pillar covers building that habit.
- Turn off what you do not use. BMCs ship with every remote-access feature enabled. Disable the protocols you never poll and the services you never use; every closed door is one less thing to patch.
- Mind the BMC's own certificate. The BMC's web console sits behind a TLS certificate like any other admin page. An expired one greets you with browser warnings at the worst moment. Track its expiry the same way you track the rest — the Certificate Monitoring pillar shows how.
Tip: the isolation rule and the monitoring rule collide in one practical spot — your monitoring system must be able to reach the management network to poll BMCs. Give it a deliberate, firewalled path (or a small probe host on that network) rather than "solving" it by flattening the isolation. The narrow path is the design; the flat network is the shortcut you will regret.
The dead-BMC problem
Now for the failure mode this pillar exists to prevent. An admin sets up BMCs during server installation, tests them once, and moves on. Years of uneventful uptime pass. Then one night a server freezes solid, the admin triumphantly browses to the BMC to grab the console — and nothing answers. The BMC has been dead for months: hung firmware, an unplugged cable from that switch migration, an expired certificate, a password nobody wrote down. It was found dead on the exact day it was needed, because that was the first day anyone looked. The crisis door rusted shut, silently, while everyone watched the front of the building.
The shared-port configuration from earlier deserves a second mention here, because it produces the cruelest version of this story. The BMC is alive and well but unreachable for exactly as long as the server is, because both ride the same dead network card. If any of your BMCs share a port with production traffic, write that fact in your inventory in large letters. That configuration means "this crisis door only opens when there is no crisis."
The fix for the silent-death problem is almost insultingly simple: look on a schedule instead. A reachability check against every BMC, every few minutes, with an alert when one stops answering: that alone converts "discovered dead during the crisis" into "fixed within days of dying." BMCs do genuinely hang now and then. Most recover with a BMC-only reset, and the stubborn ones need the server's power fully drained at the next maintenance window. All fine — on a Tuesday. Beyond reachability, poll a sensor or two so you know the BMC is answering meaningfully. Watch its certificate expiry if you use its web console. Once in a while have a human actually log in and open the remote console. The full rehearsal discipline is covered in Crisis-Path Drills.
If you take one sentence from this whole guide, take this one: an unmonitored BMC is not a crisis plan, it is a superstition. The check that proves your emergency door still opens costs your monitoring system a ping every few minutes. The absence of that check costs you the door, at night, during the fire.
Where to go next
Set this up in the boring order: get every BMC cabled and on the management network. Kill the default passwords. Point your monitoring at the sensors. And add the reachability check that watches the watcher's little brother. From there, the story widens beyond servers: switches, firewalls, and power gear have service entrances of their own. Consoles remain the path that works when even the BMC cannot help. That is picked up in Console Servers and OOB Paths. For the full map of the management plane, head back to Out-of-Band & Management-Plane Monitoring.
Put this into practice with SysPeep Monitor
If your servers have BMCs, they are already collecting hardware health data — something just needs to ask for it. SysPeep Monitor polls BMCs and other management interfaces over SNMP and similar collectors. It graphs temperatures, fans and power state and alerts both when hardware sickens and when a BMC itself stops answering. The free trial runs on any Windows machine.
Download Free TrialLearn About SysPeep