Home › Topics › Firewall Log Analysis Fundamentals › Logs to Better Rules

From Logs to Better Rules: Closing the Loop


Every firewall rulebase drifts. Rules get added for a project and never removed. A broad "allow anything from the office to the servers" rule goes in during a crisis. It stays because nobody is sure what depends on it. Someone adds a narrow rule below a broad one. The narrow rule never fires because the broad one catches everything first. None of this is anyone's fault. It is what happens when rules are added under pressure and reviewed never. After a few years the rulebase is longer than anyone can hold in their head. The honest answer to "what does this rule do?" is "I'd rather not touch it."


Your logs can fix this, because every log line names the rule that made the decision. Read enough of them and the rulebase stops being a mystery. You can see which rules carry traffic, which carry nothing, which are far broader than the traffic they carry, and which are silently dead. This guide turns those observations into a safe, repeatable cleanup process a small team can run every quarter. The steps are propose, review, apply, verify in the logs, and record. The guide also shows how to tell whether it is working.


In this guide

  • The loop: logs teach, rules change, logs confirm
  • Rules that never fire
  • Overly broad allows revealed by logs
  • Shadowed rules
  • Temporary rules that became permanent
  • Documenting every change
  • Change control that fits a small team
  • The quarterly cleanup checklist
  • Measuring improvement
  • The finding-to-action table
  • Where to go next

The loop: logs teach, rules change, logs confirm


The mental model is a loop with three parts. Logs teach: aggregated by the rule field, they show what each rule actually does — how much traffic, between which hosts, on which ports. Rules change: based on what you learned, you remove, tighten, reorder, or expire rules. Logs confirm: after the change, the same aggregations show whether the new rules carry the traffic you expected. They also show whether anything you did not expect is now being denied. Then the loop repeats.


The third part is what makes this safe. Most firewall changes go wrong not because the idea was bad but because nobody looked afterward. A rule tightened on Friday breaks a batch job that runs at month-end. Without the confirming step, the connection between change and breakage is lost. Without that step, the fix is another broad rule added in a hurry. With the confirming step, the new deny appears in the next review. The rule name points straight at your change. With that confirming step, the fix is a narrow exception. Everything in this guide is in service of running the loop deliberately instead of by accident.


Rules that never fire


The first finding is the easiest to see and the most dangerous to act on. Group allow logs by rule name over a long window. List every rule in the rulebase, and note which rules appear in the log zero times. Your firewall's own per-rule hit counters give the same answer more directly. The rule-hit analysis guide covers reading those counters, including what resets them. Logs add something counters lack: a timeline. A rule with zero hits in the counter but three hits in last quarter's logs is not dead. Logs tell you when it last mattered and for whom.


"Never fires" is only meaningful relative to an observation window. The window has to be long enough to include the rare things. Disaster-recovery rules fire during a failover test. Year-end rules fire at year end. A rule allowing a vendor to reach a system fires when the vendor does maintenance, which might be twice a year. Before you conclude a rule is dead, ask three questions. Does the log window cover at least one full business cycle? Does the rule's name or description hint at an occasional purpose? And does anyone remember why it exists? If all three come back empty, the rule is a removal candidate. Even then, the safe move is disable first, delete later. Disable the rule, note the date, and leave it disabled through the next cycle. If nothing breaks and no deny referencing that flow appears in the reviews, delete it. If something breaks, re-enable with one click. In that case, you have learned what the rule was for. That is worth writing in its description.


Overly broad allows revealed by logs


A broad rule is one whose match criteria admit far more traffic than actually flows through it: source "any," destination "the server subnet," service "any." Such rules are usually born in a crisis ("just make it work, we'll tighten it later") and survive because nobody knows what they are carrying. The logs know. Take the rule's allow records over a long window and group them by source, destination, and destination port:


rule: office-to-servers-any     (window: one full quarter)

sessions   src              dst             dport
  418220   10.20.0.0/22     10.10.0.12      445     file server
  201877   10.20.0.0/22     10.10.0.20      443     intranet web
   66104   10.20.0.0/22     10.10.0.7       1433    finance app -> database
     212   10.20.4.51       10.10.0.31      22      one workstation -> one server
       3   10.20.6.9        10.10.0.40      5900    one workstation -> one server

A rule that permits anything from the office to any server on any port turns out to carry three real flows plus a long tail. The three flows are the rule's actual purpose. Each can become its own narrow rule. These are office to the file server on 445, office to the intranet web server on 443, and office to the finance database on 1433. The long tail needs a decision per row. The single workstation using SSH to one server is probably an admin who needs a specific rule of their own. The three sessions on a remote-desktop-style port are probably an accident, and denying them costs nothing.


The method generalizes: the traffic a broad rule actually carries is the specification for the narrow rules that should replace it. Build the narrow rules above the broad one. Watch the logs for a cycle to confirm the narrow rules are carrying everything. Then disable the broad one and watch the deny log for anything that falls through. Two cautions. First, the window must be long enough to catch the rare flows. Otherwise, the tail you did not see becomes the deny you did not expect. Second, do not be tempted to write one narrow rule per row. Group by purpose, so that "office to the file server" is one rule rather than dozens.


Shadowed rules


Firewalls evaluate rules top to bottom and stop at the first match. A shadowed rule is one that can never be reached because a rule above it already matches everything it would match. The usual shape has a broad allow near the top. Below it is a specific deny (or a specific allow with logging, or a specific allow with inspection) that someone added later. They did not notice it would never be consulted. The specific rule looks fine in the rulebase, and it is doing nothing at all.


Logs reveal shadowing in a way that reading the rulebase does not. Take the traffic the specific rule was written for — say, "deny the guest network to the printer subnet" — and search the logs for it. If the lines exist and name the broad rule instead of the specific one, the specific rule is shadowed. In that case, the traffic is being handled by the wrong policy. The fix is either to move the specific rule above the broad one or to narrow the broad one so it no longer overlaps. Reordering is the quicker fix and the riskier one, because the moved rule now sees traffic it never saw before. A specific deny moved to the top will deny things that were previously (perhaps accidentally) allowed. Treat a reorder as a change like any other, and watch the deny log afterward.


Temporary rules that became permanent


Every rulebase contains rules named something like temp-vendor-access or migration-weekend-allow. They were added for a specific window and never removed. The migration finished. The vendor's contract ended. The rule remains, wide open, because removing it was nobody's task. These are the rules a reviewer worries about most, because their purpose has expired but their permission has not.


Two habits prevent the problem and one process cures it. First habit: every temporary rule gets an expiry date in its name or description and, where the firewall supports it, a scheduled disable. Second habit: temporary rules go in a dedicated section of the rulebase so they are easy to find. The cure: the quarterly cleanup lists every rule in that section. It also lists every rule whose name or description contains "temp," "test," "migration," or a date. It checks each against the logs. A temporary rule carrying no traffic is disabled on the spot. One still carrying traffic gets a conversation with its owner. Either the need is permanent (write a proper narrow rule with a proper description) or it is not (set a real expiry and hold to it).


Documenting every change


A rule without a description is a rule whose purpose you will be guessing at later. The cheapest, most durable documentation lives in the rule itself. Most firewalls give every rule a description or comment field. And the discipline is that no rule is saved without one. A useful description answers four questions: who asked for it, why, which ticket or change record covers it, and when it should be reviewed. For example:


office-to-fileserver-smb
  Office workstations to FS01 on 445. Replaces office-to-servers-any (tightened, change CHG-0412).
  Owner: infra team. Review: next quarterly cleanup.

vendor-hvac-remote
  HVAC vendor maintenance access to BMS controller on 443, from vendor's fixed address only.
  Requested by facilities (ticket 8817). Expires end of maintenance contract; disable then.

Alongside the in-rule descriptions, keep a change log: a shared document or ticket queue with one entry per change. Record the date, the person, the rules touched, the reason, and the verification result. The change log is what you consult when a deny appears in the weekly review and you need to know what changed last week. Many firewalls keep a configuration audit trail automatically. It is worth retaining for the life of the device. But it records what changed and rarely why; the change log carries the why. The retention side of audit records is covered in what to keep, review, and ignore.


Change control that fits a small team


Formal change management, with its boards and forms, was designed for large organizations. A team of two still needs the substance of it. That means a second pair of eyes, a known time, a way back, and a check afterward — without the ceremony. Five steps deliver that substance in minutes rather than meetings.


  1. Propose. Write down what you intend to change and why, in a form someone else can read. Include the rule (or rules) and the finding from the logs that motivated it. Include the expected effect ("these three flows move to narrow rules; the broad rule is disabled"), and what would tell you it went wrong. Attach the log aggregation that supports it. Even when the proposer and the approver are the same person, writing it down catches half the mistakes.
  2. Review. Someone other than the proposer reads it. In a two-person team that is the other person. In a one-person team it is a colleague from another discipline, or yourself the next morning. The reviewer's questions are simple: is the log window long enough? Does the change touch anything not mentioned? What is the rollback? Rollback is usually trivial — re-enable the disabled rule, move the reordered rule back — but say it out loud.
  3. Apply in a window. Make the change when breakage is noticed quickly and matters least, and when you are available to roll it back. Early in the working day is often better than late at night. A batch job that breaks at night is not discovered until morning. By contrast, a user who cannot reach the file server at 9 AM tells you in seconds. Save a copy of the configuration before you begin.
  4. Verify in the logs. This is the step most teams skip and the one that makes everything else safe. Immediately after the change, confirm that the new narrow rules are logging allows for the traffic you expected. Over the next cycle, watch the deny log for the flows that used to match the old rule. Every one of them is a flow you did not know about, and each needs a decision. Note that "verify" means looking at the logs on purpose, not waiting for a complaint.
  5. Record. Update the rule descriptions and the change log with what was done and what the verification showed. If the verification turned up a surprise, record that too; it is the most valuable line in the entry.

Tip: Batch the cleanup into small changes rather than one grand reorganization. Tightening one broad rule and verifying it for a week teaches you more, and risks less, than rewriting the rulebase in an afternoon. If a change goes wrong, you want to be able to say exactly which of the three things you did this week caused it.


The quarterly cleanup checklist


Once a quarter, block two hours, ideally with the second person, and work through the list. Each item produces proposals, not changes; the changes go through the five steps above over the following weeks.


  1. Confirm the log window. Check that the log data you are about to analyze covers the whole quarter for every firewall, with no gaps from a collector outage. A gap turns "never fires" into "never fires that we know of."
  2. List rules with no traffic. Compare the rulebase against allow and deny logs grouped by rule. For each rule with no hits, apply the three questions (cycle coverage, occasional purpose, human memory) and mark the survivors as disable candidates.
  3. Profile the broadest allows. Sort allow rules by how broad their criteria are (any-source, any-service, large subnets). Aggregate each one's traffic by source, destination, and port. Draft the narrow replacements for the top few.
  4. Check for shadowing. For each specific rule with no hits, search the logs for the traffic it was written for and see which rule actually handled it. Draft reorders or narrowings.
  5. Sweep temporary rules. Everything in the temporary section, plus every rule whose name or description contains "temp," "test," "migration," or a date. Disable the idle ones; chase owners for the rest.
  6. Read the weekly notes. The "tune" findings from the quarter's deny-log reviews are proposals already; make sure each has an entry.
  7. Fill in missing descriptions. Any rule without a description gets one, even if it says "purpose unknown; carries traffic between X and Y; investigate before removing." An honest placeholder beats a blank.
  8. Record the metrics. Count the rulebase before you begin (see below), so next quarter has a comparison.

Measuring improvement


A cleanup process that cannot show its results will not survive its first busy quarter. Keep a handful of numbers, recorded at each cleanup, and watch their direction. Trends matter; absolute values do not.


  • Total rules. Fewer is better, up to a point; the goal is a rulebase someone can read end to end.
  • Broad rules. The count of allow rules with any-source, any-service, or a whole large subnet as the destination. This is the number that should fall fastest.
  • Rules with no description. Should reach zero and stay there.
  • Temporary rules past their expiry. Should reach zero and stay there.
  • Unexplained denies per week. From the weekly review notes: how many internal-source deny findings had no explanation at the end of the week. As the rulebase and the environment get cleaner, this falls.
  • Changes rolled back. A rollback is not a failure; it is the process working. But a rising rollback count means the log windows are too short or the reviews too thin.

Write the numbers at the top of each quarter's cleanup notes. Four quarters later you will have a small table showing the rulebase getting shorter, narrower, and better explained. That is satisfying to look at, and persuasive to whoever approves your time.


The finding-to-action table


What the logs showWhat it meansRule actionVerify by
Rule with no hits across a full business cycleProbably dead, possibly seasonalDisable; delete after a further cycleNo new denies for that flow; nobody complains
Rule with rare hits from one sourceA specific need hiding in a general ruleKeep, but narrow to that source and purposeAllow logs show the narrow rule carrying it
Broad allow carrying a few distinct flowsOver-permissive; real purpose is knownAdd narrow rules above; disable the broad oneNarrow rules log allows; deny log shows no fallout
Broad allow with a long tail of one-off flowsAccidental permissionsNarrow rules for the purposeful flows; let the tail be deniedReview each new deny; add exceptions only with an owner
Specific rule with no hits; its traffic logged under a broad rule aboveShadowedReorder, or narrow the broad ruleTraffic now logs under the specific rule
Temporary rule still carrying trafficNeed outlived its windowReplace with a proper, described, narrow rule, or set a real expiryOwner confirms; description updated
Temporary rule carrying nothingForgottenDisable now; delete next cleanupNo denies; no complaints
Internal deny from a legitimate flow (weekly review)Policy is missing a real needAdd a narrow allow with an owner and descriptionThe deny disappears from next week's review
Deny on a tripwire rule from a known health checkLegitimate exceptionAdd a narrow allow above the tripwireTripwire list is empty again
Rule with no descriptionNobody knowsDocument from the logs; investigate before touchingDescription present at next cleanup

Where to go next


The rulebase stops being frightening the moment you can see what each rule actually carries. The log's rule field gives you exactly that. Dead rules are disabled and then deleted. Broad rules are replaced by the narrow rules their own traffic specifies. Shadowed rules are surfaced and reordered. Temporary rules get real expiries. Every change is proposed, reviewed, applied in a window, verified in the logs, and recorded. Quarter by quarter the firewall becomes something you understand rather than something you avoid.


The findings this process consumes come mostly from the weekly deny-log review. The aggregations it depends on are built in the tooling guide. If any of the fields used here are unfamiliar, what a firewall log contains is the place to start. All of it lives under Firewall Log Analysis Fundamentals.


Put this into practice with SysPeep Monitor

Verifying a rule change means watching for what broke, and not everything that breaks shows up in a deny log. SysPeep Monitor keeps an eye on the servers and services behind the rules. That includes reachability, service checks, and health over SNMP and WMI. It alerts you when a tightened rule quietly cuts off something that mattered. The free trial runs on any Windows machine.

Download Free TrialLearn About SysPeep