Home › Topics › Config Backup and Drift Detection Alongside Monitoring › Automating Config Backups

Automating Config Backups: SSH, APIs, and Scheduled Pulls


Everyone agrees configuration backups are a good idea. The reason so many networks still do not have them is the chore: "log into forty devices and save the output." That gets done enthusiastically once, half-heartedly twice, and then never again. A backup that depends on a human remembering is not a backup. It is a good intention with a file attached.


The fix is to make a machine do it. A configuration backup is one login, one read-only command, and one file write — about the simplest thing you can automate. Once it runs on a schedule, drops the result into a version-controlled repository, and complains when a device does not answer, it becomes quiet infrastructure. Nobody thinks about it until the day it saves the network.


This guide walks through the ways a configuration can get from a device into a file. It shows a safe script skeleton you can adapt. It covers the parts people skip: scheduling, repository layout, retention, failed devices, and proving a backup actually restores. If you are still deciding whether this is worth the effort, the sibling guide on why configuration backups belong next to monitoring makes the case. This one assumes you want it running by Friday.


In this guide

  • The shape of every backup job
  • Method 1: SSH and CLI pulls
  • Method 2: device-triggered exports
  • Method 3: management APIs
  • Method 4: built-in backup features
  • Choosing a method per device
  • Scheduling: nightly plus a post-change hook
  • Storing backups in version control
  • Retention
  • Handling failures
  • Verifying restores on a lab device
  • The backup job checklist
  • Where to go next

The shape of every backup job


However you collect configurations, the job has the same steps. It reads a list of devices. For each one it authenticates, asks for the configuration, and receives text (or a structured file). It compares what it received with the last copy. If the two differ, it stores the new copy. If a device did not answer, it says so loudly. Every method below is just a different way of doing the "asks for the configuration" step.


Two principles apply to all of them. First, the job must be read-only: it runs a show-style command or calls an export endpoint, and never sends a configuration command. Use an account with read-only privileges wherever the device supports one, so that a bug in the script cannot become an outage. Second, no credentials in the code. The password, key, or API token comes from a protected store and is fetched at run time. That store can be a secrets vault or an operating-system credential manager. At minimum, it is an encrypted file readable only by the account that runs the job. A password in a script will end up in a repository, a screenshot, or a chat message.


Method 1: SSH and CLI pulls


The workhorse. The script opens an SSH session, runs the command that prints the running configuration, captures the output, and closes the session. It works on nearly every device with a command line. It needs nothing configured on the device beyond SSH access for the backup account. The output is exactly what you would see on the screen. The skeleton below is in Python because it reads like English; the same structure in PowerShell looks nearly identical.


#!/usr/bin/env python3
"""Nightly configuration pull. Read-only: log in, run one show command,
save the text, commit if it changed. Credentials come from the secret store."""

import csv, datetime, pathlib, subprocess, sys

REPO = pathlib.Path("/srv/config-backups")
INVENTORY = REPO / "inventory.csv"     # hostname,address,role,show_command

def get_secret(name):
    """Fetch a credential from the protected store at run time.
    Never put a password or key in this file."""
    return secret_store.read(name)      # your vault / credential-manager client

def pull_config(address, username, key, command):
    """Open an SSH session, run ONE read-only command, return its output."""
    ...

def normalize(text):
    """Drop volatile lines (timestamps, 'last changed' headers). See the diffing guide."""
    return text

def main():
    today = datetime.date.today().isoformat()
    creds = get_secret("network-backup-readonly")
    failures = []
    with open(INVENTORY) as f:
        for dev in csv.DictReader(f):
            target = REPO / dev["role"] / (dev["hostname"] + ".cfg")
            try:
                text = normalize(pull_config(dev["address"], creds.user,
                                             creds.key, dev["show_command"]))
                if not target.exists() or target.read_text() != text:
                    target.parent.mkdir(parents=True, exist_ok=True)
                    target.write_text(text)
                    subprocess.run(["git", "-C", str(REPO), "add", str(target)], check=True)
                    subprocess.run(["git", "-C", str(REPO), "commit", "-q", "-m",
                                    dev["hostname"] + ": pull " + today], check=True)
            except Exception as exc:
                failures.append((dev["hostname"], str(exc)))
    if failures:
        report_failures(failures)       # e-mail, chat, or a status file for monitoring
        sys.exit(1)

if __name__ == "__main__":
    main()

Notice what the skeleton does and does not do. It reads an inventory file, so adding a device means adding a line, not editing code. It fetches credentials from a store you already protect. It runs one command per device and never writes to the device. It only commits when the text changed, so the history is a list of real changes rather than a nightly heartbeat. And it treats any failure as reportable and exits non-zero, which is how a scheduler or a monitoring check learns that something went wrong.


Three practical details. Many devices paginate long output with a "more" prompt. Send the command that disables paging for the session first (something like terminal length 0 or a "no paging" equivalent in your switch's CLI). Some devices need a moment after login before they accept a command, so build in a timeout and a retry. And record each device's SSH host key on first contact so that the script fails safely if the key changes. A changed key means either the device was replaced or something is impersonating it. Both deserve a human's attention.


Method 2: device-triggered exports


Many devices can send their own configuration to a server, on a schedule or automatically every time the configuration is saved. The transport is usually SCP or SFTP (encrypted, authenticated) or TFTP (neither, and older). The device does the pushing; your side runs a file server that accepts the upload and a small script that commits whatever arrives.


The appeal is that the device knows exactly when its configuration changed and can archive immediately. That gives you a copy from the moment of every save. The cost is that each device has to be configured to do it. The receiving server needs hardening (it now accepts connections from every device and stores sensitive files). And the device chooses the filenames, so your commit script has to map them back to hostnames. TFTP in particular should be confined to an isolated management network if it is used at all. Anything on the path can read the configuration in flight.


A sensible hybrid: use the device's save-triggered push as the "after every change" copy. Keep the nightly SSH pull as the safety net that catches devices whose push silently stopped working.


Method 3: management APIs


Firewalls, wireless controllers, and load balancers increasingly expose a web API alongside or instead of a CLI. Somewhere in it there is nearly always an export call that returns the full configuration as a file. The script authenticates with an API token (read-only scope, from the protected store), calls the endpoint over HTTPS, and saves the response.


What comes back is often structured — JSON or XML — rather than the line-by-line text a CLI produces. That is fine for restore, but for diffing you want it consistently formatted. Pretty-print it with fixed indentation and sorted keys before saving. Otherwise, two exports of an identical configuration may differ only in whitespace and ordering and generate a false change every night. Some appliances return a compressed archive; store the archive for restore, and extract the readable configuration inside it for the diff.


Method 4: built-in backup features


Some monitoring platforms and most configuration-management tools include a configuration backup module. Point it at your device list and credentials, and it pulls configurations on a schedule and shows diffs in its own interface. If you already have such a tool, use it. It has solved the pagination prompts and odd device behaviors that eat a week of your own script's development time.


Two cautions. Check what the tool keeps and where: some hold configurations in their own database, which means your backups now share fate with that tool. Export the files into your own repository on a schedule so that the search and diff payoffs hold even when the tool is down. A repository of plain files is the most durable format there is. And be honest about coverage: a tool that supports your switches beautifully may not support the load balancer, so the odd devices still need another method.


Choosing a method per device


MethodProsConsFits best
SSH / CLI pullWorks on almost anything with a CLI; nothing to configure on the device; readable textPagination and prompt quirks per platform; needs a login to every deviceSwitches, routers, most firewalls; the default method
Device-triggered export (SCP/SFTP/TFTP)A copy at the moment of every save; no inbound login neededPer-device setup; file server to harden; device chooses filenames; TFTP is insecureCore devices; a complement to nightly pulls
Management APIScriptable, token-based auth; returns a complete exportStructured output needs consistent formatting for diffs; token scope must be checkedFirewalls, wireless controllers, load balancers
Built-in tool moduleHandles platform quirks for you; diffs in a UICoverage gaps; backups may live inside the toolNetworks that already run such a tool; still export to a repository

Scheduling: nightly plus a post-change hook


Run the full pull once a night, at a quiet hour, from a scheduler on the backup host. That is the built-in task scheduler on a Windows machine or cron on Linux. One login per device per night is a negligible load. But stagger the job across a few minutes rather than opening forty SSH sessions in the same second. That matters especially if the devices share an authentication server that would see the burst as suspicious.


Then add the post-change hook: a way to run the same script against one device on demand, immediately after a planned change. The simplest version is a command-line switch:


# nightly, all devices
python3 pull-configs.py

# right after a change to one device
python3 pull-configs.py --only sw-floor2-01

Make it easy enough that running it becomes reflexive: change made, config saved, backup pulled. The sibling guide on the change workflow makes that "backup before, backup after" pair a formal step. The hook is what makes the step painless.


A third, optional schedule is a weekly consistency check. For devices that distinguish running from startup configuration, pull both and flag any device where they differ. That catches the unsaved change that will vanish at the next reboot.


Storing backups in version control


Store the files in a git repository. Git is a version-control tool that records every version of every file and who committed it when. Comparing any two versions is a built-in operation. Git was designed for source code. But a device configuration is text that changes occasionally and whose history matters, which is precisely the case it handles well. Push the repository to a second location on a different machine so that it survives the backup host. Keep both locations access-controlled: these files contain secrets.


Layout matters, because the repository will outlive the script and the person who wrote it. Keep it boring:


config-backups/
  README                 what this is, how the job runs, who to call
  inventory.csv          hostname,address,role,show_command
  core/
    sw-core-01.cfg
    sw-core-02.cfg
  access/
    sw-floor1-01.cfg
    sw-floor2-01.cfg
  edge/
    fw-edge-01.json      API export, pretty-printed
    rtr-wan-01.cfg
  wireless/
    wlc-main-01.xml
  golden/                role templates (see the golden configs guide)
  status/
    last-run.txt         written by the job; watched by monitoring

  • One file per device, named by hostname, with an extension that says what kind of file it is. Never put the date in the filename; the date lives in the commit history.
  • One folder per role or site, whichever you will search by. Role tends to age better, because sites move and roles do not.
  • One commit per changed device per run, with the hostname and date in the message. Ten small commits are easier to read later than one commit touching ten files.
  • Commit only when changed. If the job commits every night regardless, the history becomes noise. The normalize step (dropping timestamps and other volatile lines) is what makes "unchanged" mean unchanged.
  • The inventory lives in the repository too, so that its history is tracked and the job cannot run against a stale list nobody can find.

Retention


Keep everything. A configuration file is a few kilobytes. A network of a few hundred devices changing occasionally accumulates history at a rate that will not trouble any disk you own. The questions you will ask of old history — "what did this port do before?", "when did we remove that route?" — are exactly the ones you cannot predict. The cost of keeping the answer is nothing. Never rewrite or prune the history; the value of a version-control store is that it is complete.


The one thing that does grow is binary exports from appliances that only offer compressed archives. If those are large, keep the extracted readable configuration in the repository and the archives on a file share with a keep-the-last-few-months rule. Text goes in git forever; big binaries get a retention policy.


Handling failures


A device that did not back up is not a log line; it is an alert. The failure modes are mundane and every one of them, left alone, results in a device with no recent backup on the day it dies:


  • The backup account's password was rotated and the secret store was not updated.
  • Someone tightened a management access list and the backup host is no longer allowed in.
  • The device was replaced and its SSH host key changed, so the script (correctly) refused to connect.
  • A slow device timed out; a new device joined the network but not the inventory.
  • The repository host ran out of disk, or the push to the second location has been failing quietly for weeks.

Build the job so that these surface. Have it write a status file at the end of every run. That file holds a timestamp, the count of devices attempted, succeeded, and failed, and the names of the failures. Point a monitoring check at that file. Alert if it is older than a day and a half, or if the failure count is above zero. The script's non-zero exit code does the same job if your scheduler can act on it. Silence means success; anything else reaches a human. A device that has failed three nights running should become a ticket, because by then it is a real gap in your recovery plan.


Tip: also alert on devices that the monitoring system knows about but the backup inventory does not. Monitoring discovers new devices; the backup job only knows the ones it was told about. Comparing the two lists weekly keeps the inventory from quietly falling behind reality.


Verifying restores on a lab device


The job is not finished when the files land. It is finished when you have proven that a file can bring a device back. Keep a spare of each critical model, or borrow one during a quiet period, and a few times a year run the restore drill:


  • Pick a recent backup of a production device of the same model.
  • Factory-reset the spare, connect by console, and confirm its firmware is close to the production device's.
  • Load the configuration — paste it, or copy the file over and apply it, whichever the platform expects.
  • Watch for rejected lines: interface names that do not exist on this hardware, commands the firmware does not know, password hashes that did not transfer.
  • Confirm the management address responds and that a handful of representative settings are in effect.
  • Write down every fix you had to make. That document is the restore runbook.

If the drill finds that the restore needs information the backup does not contain, extend the job to capture that too. Examples are the hardware model, the firmware version, or a certificate that is referenced but not included.


The backup job checklist


Run down this list once the job is in place, and again whenever a new device type joins the network.


  • Read-only account on every device, with the smallest privilege the platform allows.
  • Credentials from a protected store, fetched at run time; nothing in the script or the repository.
  • Inventory file in the repository, one line per device; a weekly comparison against the monitoring system's device list.
  • A method per device type from the table above, and a test that each returns a complete configuration, not a truncated page.
  • Normalization that strips volatile lines so unchanged configurations compare as unchanged.
  • Nightly schedule at a quiet hour, staggered; a post-change hook for one device on demand.
  • One file per device, commit only on change, pushed to a second location; both locations access-controlled and encrypted at rest.
  • Status file and exit code after every run, with a monitoring check that alerts on staleness or failures.
  • Restore drill on a lab device a few times a year, with a written runbook that the drill keeps honest.
  • A README that explains all of the above to the person who inherits it.

Where to go next


Automating configuration backups comes down to a small script that logs in read-only and asks for the configuration. The script stores the configuration in a version-controlled repository only when it changed, and complains when a device does not answer. SSH pulls cover most devices; device-triggered exports, APIs, and built-in tool modules fill the gaps. A nightly schedule plus a post-change hook gives you a safety net and a clean before-and-after record for every change. A restore drill a few times a year turns the files from a hope into a plan.


With the files flowing, the repository becomes the raw material for everything else in this topic. The sibling guide on diffing configs turns the nightly comparison into a drift alert. The guide on golden configs checks each backup against a per-role standard. And the change workflow ties the post-change pull to a change record. If you like the idea of monitoring configuration itself living as versioned files, the Monitoring as Code topic follows that thread. Everything here lives under Config Backup and Drift Detection Alongside Monitoring.


Put this into practice with SysPeep Monitor

Configuration backup is a companion practice to monitoring, and SysPeep Monitor does not do it for you — the script in this guide does. What SysPeep adds is the watching. It collects health and performance metrics from the backup host and from every device the job pulls from. It uses SNMP, SSH, and similar collectors. Its alert rules tell you when the backup host or one of those devices goes quiet. So a device that stops answering the backup job is usually already on your dashboard. The free trial installs on a Windows machine in minutes.

Download Free TrialLearn About SysPeep