Home › Topics › Migrating Between Monitoring Platforms Without Losing Your History › Parallel Running

Parallel Running: Two Monitors, One Truth


There is only one way to know whether a new monitoring system sees what the old one sees. It is not reading the documentation, not running a demo against a lab, and not trusting the device count on the overview page. It is pointing both systems at the same real network, at the same time, for long enough that the network does something interesting, and then comparing notes. This is parallel running: both systems watch the same devices, but only one of them is believed. It is the phase that turns a migration from a hope into a decision backed by evidence.


It is also the phase people cut short, because it is dull. Nothing new is being built. Every day looks the same: check both systems, note where they differ, chase the reason. I have run three of these, and by the second week of each I wanted to declare it done. Every difference found in that second week was an outage or a missed alert that would otherwise have surfaced after cutover. That is the day the new system becomes the one people act on. By then, the old system would no longer have been there to catch it.


This guide covers how to set the two systems up side by side without hurting the devices they poll. It explains what to compare and how closely it needs to match. It covers how to route alerts so that only one system is ever the truth people act on. It explains how long to keep going, who does the daily watching, and how to treat the disagreements you will find. It ends with a daily checklist and a readiness table for deciding when the phase is over.


In this guide

  • Why nothing else proves parity
  • The polling load caution: two pollers, every device
  • Comparing what each system sees
  • Routing: one alerting truth, one quiet channel
  • How long to run both
  • Who watches which
  • Duplicates and disagreements
  • The daily comparison checklist
  • The cutover-readiness table
  • Where to go next

Why nothing else proves parity


Parity, for a monitoring migration, means the new system detects the same conditions as the old one and tells the same people about them. Notice what it does not mean. It does not mean the new system is configured the same way, has the same number of checks, or shows the same numbers on its overview page. Two systems can have identical device counts and still differ on what they would say when the mail server's disk fills at three in the morning.


Parity is a claim about behavior, and behavior can only be observed by letting the system behave. So both systems monitor the same devices with the same intended thresholds, both raise whatever alerts they raise, and a human compares them. When they agree for long enough, across enough kinds of event, you have evidence. When they disagree, you have a finding to chase — and every one of those is a gift, because you found it while the old system was still watching.


This holds in every direction. Migrating to a tool with a longer feature list does not mean it will see everything the old one saw. And migrating to a simpler tool does not mean it will miss things. The only way to know is to run them side by side.


The polling load caution: two pollers, every device


Before the new system is pointed at production, think about the devices. Every check is a question sent to a device, and answering costs the device CPU. During parallel running every device gets asked everything twice: once by the old poller (the part of a monitoring system that sends the questions), once by the new. For a modern switch or a server, that is fine. An older access switch with a small management processor, a busy firewall, or a printer is a different case. So is a UPS network card that was slow to answer one poller. Two pollers can tip those devices into timeouts. The timeouts then show up as false "down" alerts in one or both systems, which is exactly the kind of disagreement you will waste a morning on.


Three precautions cover most of it.


  • Stagger the intervals. If the old system polls every device at the top of the minute, do not set the new one to do the same. A different interval or an offset spreads the load so no device answers two pollers in the same instant.
  • Watch device CPU for the first few days. Most managed devices expose their management CPU over SNMP. Add that check to both systems if it is not already there, and look at it after the new poller starts. A device whose CPU climbs noticeably is one to poll less often, from at least one side.
  • Bring the new system up in batches. Core and servers first, watch for a day, then the access layer, then the long tail of sensors and odd devices. If something struggles, you know which batch caused it.

The arithmetic behind all this — how many checks per second a device and a network can comfortably absorb — is worked through in the polling load budget guide. It is worth reading before you double everyone's load. One more note for SNMP traps and syslog: a device can usually send those to more than one destination. So add the new system as a second receiver rather than switching the destination over. Both systems then see the same events, and the old one keeps receiving them until the day you decide otherwise.


Comparing what each system sees


Comparison happens at three levels, and each level is a different kind of "same."


Same things

Does the new system have the same devices, and for each device the same interfaces, disks, services, and sensors? This is the inventory check. The new system's discovery will find things the old one did not have, which is interesting but not a parity problem. The concern is the other direction — anything in the old system that is missing from the new. Interface lists are the usual trouble spot. Two tools can index the same switch differently, and a check that says "port 24" in one may be "GigabitEthernet1/0/24" in the other. (The switch, asked directly, will call it something else again.) Match by description, not by index number.


Same values, within tolerance

For a sample of checks, compare the numbers. They will not be identical. The two systems poll at different moments, calculate rates over different intervals, and may round differently. A disk that shows eighty-seven point one percent full in one and eighty-seven point three in the other agrees. A CPU that shows twelve percent in one and sixty in the other does not. And the reason is worth finding — usually one is reading a different value or averaging over a different period. Decide what tolerance counts as "the same" for each kind of metric before you start comparing, so it is not argued about afterwards. A few percent for gauges, and a matching shape on the graph for rates, is a reasonable start.


Same alerts, within minutes

This is the one that matters most. When the old system raises an alert, did the new one raise the equivalent — same device, same condition, same severity — and how long after? A new system that fires four minutes later because its interval is longer is a configuration note; one that fires an hour later, or never, is a finding. Keep a simple log with one line per old-system alert: time, device, condition, and whether the new system matched, with the delay.


And the reverse. When the new system raises an alert the old one did not, was it right? Sometimes the new system has caught something real the old one was missing, which is worth celebrating and worth adding to the parity list. Celebrate briefly, then find out how long the old one had been missing it. Sometimes it is noise, and it needs a threshold, a dependency, or a suppression the old system had and the new one does not yet.


Routing: one alerting truth, one quiet channel


The single most important rule of parallel running: at any moment, exactly one system is the alerting truth. That is the system whose alerts people act on, whose pages wake someone up, whose tickets get worked. Until cutover, it is the old system. The new system's alerts must go somewhere the migration team can see them for comparison but nobody will confuse with the real thing.


The usual arrangement is a quiet channel: a separate chat room, a mailbox, or a distribution list that only the migration team reads. The new system's notifications point there and nowhere else. No pages, no tickets, no on-call. The migration team compares that channel against the real alerts every day. When the parity proof is complete and cutover is scheduled, the routing flips. The new system's notifications then go to the real channels and the old system's go to the quiet one. So the old system is still watching, just silently.


Two mistakes to avoid. Do not route both systems to the real channels "so nothing gets missed." The duplicates will train everyone to ignore alerts, and that lesson takes a long time to unlearn. And do not silence the old system before cutover because the duplicates in the quiet channel are annoying. The annoyance is the point.


Tip: On day one, use the new system's test-notification feature to prove the quiet channel works. A quiet channel that is quiet because nothing is arriving is not evidence of anything.


How long to run both


Long enough for the network to do the things you need to see it do. The minimum is one full business cycle, which for most organizations means all of the following:


  • At least one weekend. Weekend behavior is different: backups run, nobody is around to notice things, and the alerts that fire at three on a Saturday morning are the ones that matter most.
  • At least one patch window or planned maintenance. That is when maintenance windows, suppressions, and dependencies get exercised, and when both systems should go quiet together and come back together.
  • At least one month-end, or whatever your busiest recurring period is, because load-related thresholds only prove themselves under load.
  • Ideally, a real incident. You cannot schedule one, but if the parallel run is long enough one usually turns up. It will be the most informative day of the whole migration. Did both systems see it? Which saw it first? Did the dependency logic collapse it to one alert in both?

For most mid-sized networks that adds up to four to six weeks. Set the timebox — the maximum length you fix before the phase starts — in advance. And set it as a maximum as well as a minimum. A parallel run with no end date becomes the "two systems forever" failure described in Why Monitoring Migrations Fail. If a real incident does not turn up, manufacture some. That is what the parity proof phase is for, and it is covered in The Alert Parity Checklist.


Tollgate Foods started its parallel run with no end date, because "we'll switch when we're confident". Confidence never quite arrived. The new system took over alerting after four months. But the old one was kept polling "just in case", and every new device was added to both. Eighteen months later the old system was still running on a server that had been due for retirement twice. The old system was still under license, and still the only place the monthly availability report came from, because nobody had ever rebuilt the report. The migration was finally finished by an audit, which asked why the company had two monitoring systems and received two different answers.


Who watches which


The old system keeps its existing watchers. Whoever was on call stays on call; whoever looked at the dashboard keeps looking. Nothing about their routine changes during parallel running, and that is deliberate: they are the control group. Their experience of the old system during the run is what the new system is being compared to.


The new system is watched by the migration team, and specifically by a named person each day who does the comparison. Rotate it if you can, because a second person notices things the first has stopped seeing. But do not make it "whoever has time," because on a busy morning nobody does, and the comparison is the first thing dropped. Fifteen to thirty minutes a day is what the routine costs once it is established.


Stakeholders get invited in during the second half. The help desk is shown the new dashboard and asked to keep it open beside the old one for a week. The person who reads the monthly report gets a draft from the new system to compare against the real one. Their feedback is a finding like any other, and their eventual sign-off is a row in the readiness table.


Duplicates and disagreements


You will have duplicates: every real alert appears in the real channel and, a few minutes later, in the quiet one. That is the system working. Do not try to suppress duplicates; log them as matches and move on.


You will also have disagreements, and the mindset here matters. A disagreement is a finding, not a failure. It is not evidence that the new system is broken, nor that the old one was. It is a difference between two configurations that needs explaining. Each one gets a line in the findings log with a reason, once you have one, and a resolution.


DisagreementCommon reasonsTypical resolution
Old alerted, new did notCheck not built; threshold left at default; alert rule not enabled; dependency wrongly suppressed itFix in the new system; add the alert to the parity list
New alerted, old did notNew system found something real; or a dependency or suppression the old system had is missing in the newVerify the condition; add or fix the suppression; occasionally fix the old system too
Both alerted, minutes apartDifferent intervals or confirmation countsNote it; decide whether the delay is acceptable and record the decision
Values differ widelyDifferent value being read, different averaging period, counter versus gauge mismatchAlign the check definition
Device missing in newDiscovery missed it; wrong credentials; firewall rule for the new poller not addedAdd the device; fix access
Device missing in oldNever monitored; an inventory gapDecide whether it belongs in scope, and record why

Aim for "every disagreement explained," not "no disagreements." Some will be explained as "the new system does this differently and we accept that." That is a legitimate resolution, as long as it is written down and somebody with authority has agreed to it. An accepted difference needs a reason, a name, and someone who will still be here when it matters.


The daily comparison checklist


This is the routine the named watcher runs every morning. It is deliberately short, because a long checklist stops being done by the second week.


  • 1. Are both systems up and polling? Check each system's own health first. A poller that stopped overnight makes everything below meaningless.
  • 2. Device counts, old versus new. Any change since yesterday explained? A device added to one and not the other is a finding.
  • 3. List the real alerts since yesterday from the old system's history.
  • 4. For each, find the match in the quiet channel. Record the delay, or "no match."
  • 5. Quiet-channel alerts with no real counterpart. Verify each; classify as real catch, noise, or missing suppression.
  • 6. Sample five checks and compare values within tolerance, rotating through device types across the week.
  • 7. Was a maintenance window active? Did both systems go quiet and resume together?
  • 8. Update the findings log and assign an owner to anything new.
  • 9. Update the spreadsheet. Mark rows verified once a check has been seen to match on value and, where it fired, on alert.
  • 10. Note polling-load observations: any device CPU rising, any new timeouts.

The cutover-readiness table


The parallel run ends when every row of this table says ready, and not before. It is the gate between parallel running and cutover, and it is meant to be filled in with evidence, not feelings.


CriterionEvidenceReady when
DurationThe calendarOne full business cycle including a weekend and a patch window has passed
Inventory matchSpreadsheet status columnEvery non-pruned row is built; unexplained missing devices: none
Value parityDaily samplesSampled checks are within tolerance across every device type
Alert parityAlert logEvery real alert in the period matched in the new system within the accepted delay
DisagreementsFindings logEvery finding resolved or accepted in writing
Routing testedTest notificationsEvery real channel and contact has received a test from the new system
Stakeholder viewsSign-off emailsEach dashboard and report owner has confirmed theirs works
Parity proofSigned checklistEvery alert on the parity list signed off or documented as untestable
Rollback planA written pageSteps to flip routing back to the old system within minutes, tested once
Cutover dateThe calendar, againAnnounced to every team, with the owner available for the following week

When every row says ready, cutover is a scheduled decision. When any row does not, you are not done, and the fix is to finish, not to declare. The rollback row deserves a word. Flipping routing back is usually a matter of minutes if the old system was left polling silently. That is exactly why the guidance is to silence it at cutover rather than stop it.


Where to go next


Parallel running is where a migration earns the right to cut over. Both systems watch the same devices. The old one stays the alerting truth while the new one reports into a quiet channel. A named person compares them every day at the level of things, values, and alerts. Every disagreement is logged as a finding and explained. And the phase ends when the readiness table is full, not when it feels done. Stagger the polling so the devices survive the attention. Run through a weekend and a patch window, and treat a real incident as the best test you will get. Then end it, on the date you wrote down. A parallel run that outlives its timebox is not caution. It is a second monitoring system, and now you own both.


The inventory that this phase compares against is built in Inventorying What You Monitor Before You Move It. The deliberate tests that fill the parity-proof row are in The Alert Parity Checklist. And what happens to the old system's history after cutover is in Data Export Realities. If you are still choosing the candidate for the parallel run, the Fair Proof of Concept pillar covers evaluating tools without stacking the deck. The whole topic is collected at Migrating Between Monitoring Platforms Without Losing Your History.


Put this into practice with SysPeep Monitor

If the new system in your parallel run is still to be chosen, the free trial of SysPeep Monitor is one candidate. You could point it at the same devices. It has a central service on one Windows machine, optional remote probes for other sites, and SNMP, WMI, and SSH collection. Its dependencies and alert rules are designed to keep noise down. Route its alerts to your quiet channel and let the daily comparison decide.

Download Free TrialLearn About SysPeep