Monitoring What Customers Actually Touch: Beyond the Home Page
Here is an outage that happens somewhere every day. The home page loads perfectly. The monitor is green. And for six hours nobody can log in, because the login form posts to a service that fell over at midnight. The only check anyone ever set up was the home page. The customers do not care that the home page was fine. They came to do something, and the something was broken.
This guide is about closing that gap. You will learn how to find the paths customers actually use (logins, searches, forms, checkouts, downloads). You will learn how to rank them by what breaking them would cost. The guide covers how to watch the API endpoints behind your apps and the third-party widgets embedded in your pages. It shows how to write content checks that notice a page that loads but does not work. And it explains how to keep the list of checks matched to the business as both change. The result is a short, prioritized inventory of checks that covers what matters instead of what was easy.
In this guide
- The home-page-only trap
- Finding and prioritizing the critical paths
- APIs: the site behind the site
- Third-party embeds that break your page from outside
- Content checks that catch silent failures
- Redirects and canonical hostnames
- The critical-path inventory
- Keeping the list aligned with the business
- Where to go next
The home-page-only trap
The home page is the first thing everyone monitors, and it is the worst single page to rely on. It is usually the most cached page on the site, sometimes served entirely from an edge cache without touching your server at all. It is often static or nearly static, so it does not exercise the database, the session store, the search index, or the payment integration. And it is the page most likely to have a maintenance version that returns a friendly success code while everything behind it is dark.
So a green home-page check proves that a copy of one HTML file can be delivered. It says nothing about whether a customer can do the thing they came for. Real failures are usually partial. The search index rebuild failed and results are empty. Or the session store is full and logins time out. Or the checkout's payment call is failing. Or the download server's disk is full. Every one of those leaves the home page untouched. Monitoring has to follow the customer past it.
Finding and prioritizing the critical paths
A critical path is a sequence of pages and requests a customer follows to get something done. The way to find yours is not to list every URL on the site. It is to ask one question of the people who own it: "If this stopped working, would the phone ring?" Then walk the site as a customer, from arrival to the thing they wanted, and write down each step. Most sites end up with a list something like this:
- Login and session. The form, the request that checks the password, the page that loads afterward. Includes password reset, which is the path people use when they are already frustrated.
- Search. The query and the results page, with real results in it.
- Forms. Contact, quote request, sign-up, support ticket. The form displaying is not the path; the form submitting is.
- Checkout-style flows. Add to cart, address, payment, confirmation. Any multi-step flow where a customer gives you money or commits to something.
- Account and dashboard pages. Where logged-in customers see their own data. These hit the database in ways the public pages do not.
- Downloads and documents. Software packages, invoices, reports, manuals. Often served from a different host with its own disk and its own failures.
- Pages people link to. Pricing, status, documentation, the page your advertising sends people to. Not transactional, but loud when broken.
Two sources fill in what a walk-through misses. Traffic analytics show which pages customers actually visit, which is often not the ones the organization thinks matter. Support tickets show what people complain about when it breaks, which is the most direct definition of "critical" there is.
Prioritizing by impact
Not every path deserves a one-minute check from three locations with a page at three in the morning. Rank each one by what breaking it costs, and let the rank decide the check frequency and the alert severity. Three tiers are enough for most sites.
| Tier | Definition | Examples | Check frequency | On failure |
|---|---|---|---|---|
| 1: Money and access | Failure stops revenue or locks customers out | Login, checkout, payment, the API a mobile app needs to start | Every minute, several locations | Page immediately after confirmation |
| 2: Core function | Failure stops the main thing people come to do | Search, account dashboard, main forms, downloads | Every one to five minutes | Alert during business hours; page if it lasts |
| 3: Reputation | Failure embarrasses but does not block | Pricing, documentation, landing pages, status page | Every five to fifteen minutes | Ticket for the next working day |
Impact is a business judgment, so make it with the business. A support manager will tell you in one sentence which broken page generates the most calls. And a sales lead will tell you which form the pipeline depends on. Ten minutes with each of them produces a better list than a week of guessing from the server room.
APIs: the site behind the site
An API (application programming interface) is the set of web endpoints that programs, rather than people, call. Those programs include your mobile app, your single-page web application, partner systems, your own internal tools. The API usually lives on its own hostname, such as api.example.com, and it can be completely down while every human-facing page renders fine. If you have a mobile app, its users experience the API as the entire product, and a home-page check tells you nothing about it.
Monitor the endpoints the app calls first, in the order it calls them. That means the configuration or health endpoint it fetches on launch, the login call, and the first list it displays. For each one, check more than the status code. An API that answers with a valid-looking success code and an empty list is broken in a way only the body reveals. The same goes for an error wrapped in a successful response. A simple check pulls the endpoint and confirms a field that only exists when the backend worked:
# Fetch a read-only endpoint with a monitoring token and confirm real data came back
curl -sS -m 10 -H "Authorization: Bearer MONITOR_TOKEN" \
https://api.example.com/catalog/items?limit=1 \
| grep -q '"sku"' && echo "API OK" || echo "API BROKEN: no item data"
Some practical rules. Give the monitor its own account with read-only permissions and a token that can be revoked without affecting anyone else. Never monitor with an administrator's credentials. Prefer endpoints that read real data over synthetic "health" endpoints that return "ok" without touching the database, or check both and trust the real one. Keep API budgets tight, because an app screen typically makes several calls and the user waits for the sum. The guide on response-time budgets covers setting those numbers. And filter the monitor's requests out of usage analytics, or your busiest "customer" will be your own check.
Third-party embeds that break your page from outside
Look at what a modern page loads. There is a payment widget from the payment provider and a chat bubble from a support vendor. There is an analytics script, a tag manager, a map, a font, a video player. Each is a request to somebody else's server, made by the customer's browser. Each can fail in a way that breaks your page while your server reports nothing wrong. Here are the classic cases. The payment widget's script fails to load and the "Pay" button does nothing. The chat script throws an error that halts every other script on the page. A slow analytics host delays the page for seconds because it was loaded in a blocking way.
You cannot fix somebody else's outage, but you can know about it before your customers tell you, and you can limit the damage. Keep an inventory of every third party your critical pages load, and monitor the ones that can break a critical path:
| Third party | Loaded on | If it fails | What to check | Damage limit |
|---|---|---|---|---|
| Payment widget | Checkout | Customers cannot pay | Its script URL from outside; a transaction check that reaches the payment step | Fallback message; vendor status feed watched |
| Chat bubble | Every page | Script error may break other scripts | Its script URL; a content check that the page's own buttons still render | Load it last and asynchronously |
| Analytics and tag manager | Every page | Slow page if loaded synchronously; lost data otherwise | Its host's response time from outside | Load asynchronously; never block rendering |
| Fonts, maps, video | Specific pages | Ugly or missing content, occasionally slow render | Its host's reachability | Local fallbacks where possible |
The single most useful habit here is to make sure every third-party script loads asynchronously. That way, a slow or dead vendor cannot hold your page hostage. That is a change for the developers. But it is one the monitoring evidence will justify the first time a vendor has a bad afternoon.
Content checks that catch silent failures
The most dangerous web failure is the page that loads, returns a success code, and is wrong. The product list renders with zero products. The price table is empty because the pricing service timed out and the template hid the error. The search page says "0 results" for every query. A stack trace sits at the bottom of the page in small grey text. Every one of those passes a status check. A content check reads the body of the response and applies rules to it. And it is the only kind of external check that catches this class of failure.
Four rules cover most of the ground:
- Positive match. The body must contain a string that only appears when the page worked. Choose something rendered from real data, near the bottom of the page. That could be the last product's price, a "Showing 24 of 1,381 items" line, the footer's dynamically inserted copyright. Do not choose the page title or a static heading, which the template prints even when the data is missing.
- Negative match. The body must not contain any of a list of error signatures: "exception," "stack trace," "database error," "warning:," "undefined," "temporarily unavailable," "0 results." Build the list from the errors your platform actually prints; every framework has its own vocabulary.
- Size sanity. The body must be larger than a minimum. A product page that is normally 80 kilobytes and suddenly arrives as 3 kilobytes is an error page in disguise, whatever the status code says.
- Structure, for APIs. The response must parse as valid JSON and contain the expected fields with plausible values, such as a non-empty list where a list is expected.
Here is what a content check for a product listing page looks like when written out as rules:
Check: Catalog listing (Tier 2) URL: https://www.example.com/catalog/ Expect status: 200 Must contain: "Showing " and " of " # the results counter renders only with data Must contain: 'class="price"' # at least one priced item rendered Must not match: "0 results" | "exception" | "database" | "temporarily unavailable" Minimum size: 40 KB Locations: 3 nearby, every 2 minutes; alert after 2 of 3 fail twice
Content checks are more fragile than status checks, because a redesign can change the string you matched on. That is a feature, not a bug. A check that fails after a release is telling you either that the release broke the page or that the check needs updating. And both are things you want to know the same day. Put "update the content checks" on the release checklist and the fragility becomes routine.
Redirects and canonical hostnames
A site has more front doors than it thinks. Customers arrive at the bare domain and at the www version. They arrive at http and at https. They arrive at the old domain from before the rebrand, and at the short link on the printed brochure. Each of those is supposed to redirect to the one canonical address, the single official form of the URL. Redirects fail quietly. The cause might be a certificate that covers www but not the bare domain, a rule lost during a server migration, a loop introduced by a well-meaning edit. Nobody notices until the share of customers who type the address from memory quietly stops arriving.
Give every front door its own check, and make the check assert three things. Check where the chain ends and how many hops it took. And check that the final page passes its own content check. A command-line tool shows the pieces:
curl -sS -o /dev/null -w "%{http_code} -> %{redirect_url}\n" http://example.com/
301 -> https://www.example.com/
curl -sS -o /dev/null -L -w "final: %{url_effective} hops: %{num_redirects} status: %{http_code}\n" http://example.com/
final: https://www.example.com/ hops: 1 status: 200
The first command shows the immediate answer; the second follows the chain and reports where it landed and how many steps it took. Anything more than two hops is worth fixing, because each hop is a round trip customers pay for. A chain that grows over time is a sign nobody owns it. Note the certificate detail: a redirect from https://example.com to https://www.example.com requires a valid certificate on the bare name too. That is because the browser checks it before it reads the redirect. A check that only tests the final address will never see that failure.
Tip: use a permanent redirect (status 301) for hostname and protocol canonicalization, and check that the status is what you intended. A temporary redirect (302) in that role makes browsers and search engines re-ask every time. And it is often a leftover from a migration that was meant to be undone.
The critical-path inventory
Everything above lands in one table: the inventory of what you check, how, how often, and who cares. It is the document you hand to a new colleague, the thing you review after an incident. It is also the list that answers "are we monitoring that?" in one glance. A filled-in example for a small commerce site:
| Path | Entry point | Check | Tier | Frequency | Budget | Owner |
|---|---|---|---|---|---|---|
| Front doors | bare, www, http, old domain | Redirect chain ends at canonical, 1 hop, 301 | 2 | 15 min | n/a | Web team |
| Home | / | Status + content (featured items rendered) | 2 | 1 min | 1.5 s | Marketing |
| Login | /login then POST | Transaction: submit monitoring account, land on dashboard | 1 | 1 min | 1.5 s | Platform |
| Search | /search?q=widget | Content: results counter and at least one priced item | 2 | 2 min | 2 s | Platform |
| Checkout | cart, address, payment | Transaction to the payment step (no charge) | 1 | 2 min | 3 s per step | Commerce |
| Contact form | /contact then POST | Transaction: submit test message, confirm receipt page | 2 | 10 min | 3 s | Sales ops |
| Downloads | /downloads/latest | Status, first byte, minimum size | 2 | 5 min | 1 s first byte | Support |
| Mobile API | /api/config, /api/login, /api/items | JSON structure, real data, token auth | 1 | 1 min | 500 ms | Mobile team |
| Payment widget | vendor script URL | Reachable from outside; response time | 1 | 1 min | 1 s | Commerce |
| Certificates | all hostnames | Valid chain, name match, days to expiry | 1 | 1 hour | warn 30 days out | Platform |
A few things to notice. Transaction checks that submit forms use a dedicated monitoring account and test data that the business recognizes and filters out. Nobody wants the sales team calling a fake lead every ten minutes. The checkout check stops before any money moves. The "owner" column names who is called and who decides whether the check is still right, which is the column that keeps the inventory alive. And the certificate line covers every hostname, including the bare domain and the API, because the front-door and API checks depend on it. How to run these checks from outside, and from more than one place, is the subject of the guide on multi-location checks.
Keeping the list aligned with the business
An inventory drifts the moment it is written. Features launch without checks, and pages retire while their checks keep failing. The flow that was Tier 1 last year is replaced by a new one nobody told monitoring about. Three habits keep the list honest without much effort.
First, put monitoring on the release checklist. Any change that adds a page, a form, an endpoint, or a third party also adds or updates a check. And the developer who made the change is the one who knows what a healthy response looks like. Second, hold a short quarterly review with the owners in the inventory. Read the list aloud, ask what changed, ask what would make the phone ring now. Third, after every incident, ask the one question that improves inventories faster than anything else: did the phone ring before the alert did? If it did, the path that failed was missing from the list or checked too gently. In that case, the fix goes in that week.
Resist the urge to check everything. A list of two hundred checks with no owners is noise, and noise gets ignored. Twenty checks that each map to something the business would notice, each with a name on it, will catch the outages that matter. They will leave the pager quiet the rest of the time.
Where to go next
The home page is where customers arrive, not where they go. Follow them past it. Find the paths that would make the phone ring. Rank them by what breaking them costs. Watch the API behind the app and the third parties inside the page. Add content checks that read the body instead of trusting the status code. Check every front door's redirect. And write the whole thing down with an owner beside each line. Then keep it matched to the business through the release checklist and a quarterly conversation.
From here, the layers of a single web check shows what to do when one of these checks fails and you need to know which stage broke. And the ten-minute triage sequence turns that knowledge into a first-ten-minutes routine. Both belong to the Website & Web-Service Uptime Monitoring topic.
Put this into practice with SysPeep Monitor
Every path in your inventory ends at a server, a database, or a link somewhere in your own network. SysPeep Monitor watches those from one Windows machine. It has optional remote probes for other sites and segments, and auto-discovery to find what you forgot. Its dependency-aware alerting means a dead database raises one alert instead of one per page that needs it. The free trial installs in minutes.
Download Free TrialLearn About SysPeep