Home › Topics › Website & Web-Service Uptime Monitoring › Beyond the Home Page

Monitoring What Customers Actually Touch: Beyond the Home Page


Here is an outage that happens somewhere every day. The home page loads perfectly. The monitor is green. And for six hours nobody can log in, because the login form posts to a service that fell over at midnight. The only check anyone ever set up was the home page. The customers do not care that the home page was fine. They came to do something, and the something was broken.


This guide is about closing that gap. You will learn how to find the paths customers actually use (logins, searches, forms, checkouts, downloads). You will learn how to rank them by what breaking them would cost. The guide covers how to watch the API endpoints behind your apps and the third-party widgets embedded in your pages. It shows how to write content checks that notice a page that loads but does not work. And it explains how to keep the list of checks matched to the business as both change. The result is a short, prioritized inventory of checks that covers what matters instead of what was easy.


In this guide

  • The home-page-only trap
  • Finding and prioritizing the critical paths
  • APIs: the site behind the site
  • Third-party embeds that break your page from outside
  • Content checks that catch silent failures
  • Redirects and canonical hostnames
  • The critical-path inventory
  • Keeping the list aligned with the business
  • Where to go next

The home-page-only trap


The home page is the first thing everyone monitors, and it is the worst single page to rely on. It is usually the most cached page on the site, sometimes served entirely from an edge cache without touching your server at all. It is often static or nearly static, so it does not exercise the database, the session store, the search index, or the payment integration. And it is the page most likely to have a maintenance version that returns a friendly success code while everything behind it is dark.


So a green home-page check proves that a copy of one HTML file can be delivered. It says nothing about whether a customer can do the thing they came for. Real failures are usually partial. The search index rebuild failed and results are empty. Or the session store is full and logins time out. Or the checkout's payment call is failing. Or the download server's disk is full. Every one of those leaves the home page untouched. Monitoring has to follow the customer past it.


Finding and prioritizing the critical paths


A critical path is a sequence of pages and requests a customer follows to get something done. The way to find yours is not to list every URL on the site. It is to ask one question of the people who own it: "If this stopped working, would the phone ring?" Then walk the site as a customer, from arrival to the thing they wanted, and write down each step. Most sites end up with a list something like this:


  • Login and session. The form, the request that checks the password, the page that loads afterward. Includes password reset, which is the path people use when they are already frustrated.
  • Search. The query and the results page, with real results in it.
  • Forms. Contact, quote request, sign-up, support ticket. The form displaying is not the path; the form submitting is.
  • Checkout-style flows. Add to cart, address, payment, confirmation. Any multi-step flow where a customer gives you money or commits to something.
  • Account and dashboard pages. Where logged-in customers see their own data. These hit the database in ways the public pages do not.
  • Downloads and documents. Software packages, invoices, reports, manuals. Often served from a different host with its own disk and its own failures.
  • Pages people link to. Pricing, status, documentation, the page your advertising sends people to. Not transactional, but loud when broken.

Two sources fill in what a walk-through misses. Traffic analytics show which pages customers actually visit, which is often not the ones the organization thinks matter. Support tickets show what people complain about when it breaks, which is the most direct definition of "critical" there is.


Prioritizing by impact

Not every path deserves a one-minute check from three locations with a page at three in the morning. Rank each one by what breaking it costs, and let the rank decide the check frequency and the alert severity. Three tiers are enough for most sites.


TierDefinitionExamplesCheck frequencyOn failure
1: Money and accessFailure stops revenue or locks customers outLogin, checkout, payment, the API a mobile app needs to startEvery minute, several locationsPage immediately after confirmation
2: Core functionFailure stops the main thing people come to doSearch, account dashboard, main forms, downloadsEvery one to five minutesAlert during business hours; page if it lasts
3: ReputationFailure embarrasses but does not blockPricing, documentation, landing pages, status pageEvery five to fifteen minutesTicket for the next working day

Impact is a business judgment, so make it with the business. A support manager will tell you in one sentence which broken page generates the most calls. And a sales lead will tell you which form the pipeline depends on. Ten minutes with each of them produces a better list than a week of guessing from the server room.


APIs: the site behind the site


An API (application programming interface) is the set of web endpoints that programs, rather than people, call. Those programs include your mobile app, your single-page web application, partner systems, your own internal tools. The API usually lives on its own hostname, such as api.example.com, and it can be completely down while every human-facing page renders fine. If you have a mobile app, its users experience the API as the entire product, and a home-page check tells you nothing about it.


Monitor the endpoints the app calls first, in the order it calls them. That means the configuration or health endpoint it fetches on launch, the login call, and the first list it displays. For each one, check more than the status code. An API that answers with a valid-looking success code and an empty list is broken in a way only the body reveals. The same goes for an error wrapped in a successful response. A simple check pulls the endpoint and confirms a field that only exists when the backend worked:


# Fetch a read-only endpoint with a monitoring token and confirm real data came back
curl -sS -m 10 -H "Authorization: Bearer MONITOR_TOKEN" \
     https://api.example.com/catalog/items?limit=1 \
  | grep -q '"sku"' && echo "API OK" || echo "API BROKEN: no item data"

Some practical rules. Give the monitor its own account with read-only permissions and a token that can be revoked without affecting anyone else. Never monitor with an administrator's credentials. Prefer endpoints that read real data over synthetic "health" endpoints that return "ok" without touching the database, or check both and trust the real one. Keep API budgets tight, because an app screen typically makes several calls and the user waits for the sum. The guide on response-time budgets covers setting those numbers. And filter the monitor's requests out of usage analytics, or your busiest "customer" will be your own check.


Third-party embeds that break your page from outside


Look at what a modern page loads. There is a payment widget from the payment provider and a chat bubble from a support vendor. There is an analytics script, a tag manager, a map, a font, a video player. Each is a request to somebody else's server, made by the customer's browser. Each can fail in a way that breaks your page while your server reports nothing wrong. Here are the classic cases. The payment widget's script fails to load and the "Pay" button does nothing. The chat script throws an error that halts every other script on the page. A slow analytics host delays the page for seconds because it was loaded in a blocking way.


You cannot fix somebody else's outage, but you can know about it before your customers tell you, and you can limit the damage. Keep an inventory of every third party your critical pages load, and monitor the ones that can break a critical path:


Third partyLoaded onIf it failsWhat to checkDamage limit
Payment widgetCheckoutCustomers cannot payIts script URL from outside; a transaction check that reaches the payment stepFallback message; vendor status feed watched
Chat bubbleEvery pageScript error may break other scriptsIts script URL; a content check that the page's own buttons still renderLoad it last and asynchronously
Analytics and tag managerEvery pageSlow page if loaded synchronously; lost data otherwiseIts host's response time from outsideLoad asynchronously; never block rendering
Fonts, maps, videoSpecific pagesUgly or missing content, occasionally slow renderIts host's reachabilityLocal fallbacks where possible

The single most useful habit here is to make sure every third-party script loads asynchronously. That way, a slow or dead vendor cannot hold your page hostage. That is a change for the developers. But it is one the monitoring evidence will justify the first time a vendor has a bad afternoon.


Content checks that catch silent failures


The most dangerous web failure is the page that loads, returns a success code, and is wrong. The product list renders with zero products. The price table is empty because the pricing service timed out and the template hid the error. The search page says "0 results" for every query. A stack trace sits at the bottom of the page in small grey text. Every one of those passes a status check. A content check reads the body of the response and applies rules to it. And it is the only kind of external check that catches this class of failure.


Four rules cover most of the ground:


  • Positive match. The body must contain a string that only appears when the page worked. Choose something rendered from real data, near the bottom of the page. That could be the last product's price, a "Showing 24 of 1,381 items" line, the footer's dynamically inserted copyright. Do not choose the page title or a static heading, which the template prints even when the data is missing.
  • Negative match. The body must not contain any of a list of error signatures: "exception," "stack trace," "database error," "warning:," "undefined," "temporarily unavailable," "0 results." Build the list from the errors your platform actually prints; every framework has its own vocabulary.
  • Size sanity. The body must be larger than a minimum. A product page that is normally 80 kilobytes and suddenly arrives as 3 kilobytes is an error page in disguise, whatever the status code says.
  • Structure, for APIs. The response must parse as valid JSON and contain the expected fields with plausible values, such as a non-empty list where a list is expected.

Here is what a content check for a product listing page looks like when written out as rules:


Check: Catalog listing (Tier 2)
URL:            https://www.example.com/catalog/
Expect status:  200
Must contain:   "Showing " and " of "        # the results counter renders only with data
Must contain:   'class="price"'              # at least one priced item rendered
Must not match: "0 results" | "exception" | "database" | "temporarily unavailable"
Minimum size:   40 KB
Locations:      3 nearby, every 2 minutes; alert after 2 of 3 fail twice

Content checks are more fragile than status checks, because a redesign can change the string you matched on. That is a feature, not a bug. A check that fails after a release is telling you either that the release broke the page or that the check needs updating. And both are things you want to know the same day. Put "update the content checks" on the release checklist and the fragility becomes routine.


Redirects and canonical hostnames


A site has more front doors than it thinks. Customers arrive at the bare domain and at the www version. They arrive at http and at https. They arrive at the old domain from before the rebrand, and at the short link on the printed brochure. Each of those is supposed to redirect to the one canonical address, the single official form of the URL. Redirects fail quietly. The cause might be a certificate that covers www but not the bare domain, a rule lost during a server migration, a loop introduced by a well-meaning edit. Nobody notices until the share of customers who type the address from memory quietly stops arriving.


Give every front door its own check, and make the check assert three things. Check where the chain ends and how many hops it took. And check that the final page passes its own content check. A command-line tool shows the pieces:


curl -sS -o /dev/null -w "%{http_code} -> %{redirect_url}\n" http://example.com/
301 -> https://www.example.com/

curl -sS -o /dev/null -L -w "final: %{url_effective}  hops: %{num_redirects}  status: %{http_code}\n" http://example.com/
final: https://www.example.com/  hops: 1  status: 200

The first command shows the immediate answer; the second follows the chain and reports where it landed and how many steps it took. Anything more than two hops is worth fixing, because each hop is a round trip customers pay for. A chain that grows over time is a sign nobody owns it. Note the certificate detail: a redirect from https://example.com to https://www.example.com requires a valid certificate on the bare name too. That is because the browser checks it before it reads the redirect. A check that only tests the final address will never see that failure.


Tip: use a permanent redirect (status 301) for hostname and protocol canonicalization, and check that the status is what you intended. A temporary redirect (302) in that role makes browsers and search engines re-ask every time. And it is often a leftover from a migration that was meant to be undone.


The critical-path inventory


Everything above lands in one table: the inventory of what you check, how, how often, and who cares. It is the document you hand to a new colleague, the thing you review after an incident. It is also the list that answers "are we monitoring that?" in one glance. A filled-in example for a small commerce site:


PathEntry pointCheckTierFrequencyBudgetOwner
Front doorsbare, www, http, old domainRedirect chain ends at canonical, 1 hop, 301215 minn/aWeb team
Home/Status + content (featured items rendered)21 min1.5 sMarketing
Login/login then POSTTransaction: submit monitoring account, land on dashboard11 min1.5 sPlatform
Search/search?q=widgetContent: results counter and at least one priced item22 min2 sPlatform
Checkoutcart, address, paymentTransaction to the payment step (no charge)12 min3 s per stepCommerce
Contact form/contact then POSTTransaction: submit test message, confirm receipt page210 min3 sSales ops
Downloads/downloads/latestStatus, first byte, minimum size25 min1 s first byteSupport
Mobile API/api/config, /api/login, /api/itemsJSON structure, real data, token auth11 min500 msMobile team
Payment widgetvendor script URLReachable from outside; response time11 min1 sCommerce
Certificatesall hostnamesValid chain, name match, days to expiry11 hourwarn 30 days outPlatform

A few things to notice. Transaction checks that submit forms use a dedicated monitoring account and test data that the business recognizes and filters out. Nobody wants the sales team calling a fake lead every ten minutes. The checkout check stops before any money moves. The "owner" column names who is called and who decides whether the check is still right, which is the column that keeps the inventory alive. And the certificate line covers every hostname, including the bare domain and the API, because the front-door and API checks depend on it. How to run these checks from outside, and from more than one place, is the subject of the guide on multi-location checks.


Keeping the list aligned with the business


An inventory drifts the moment it is written. Features launch without checks, and pages retire while their checks keep failing. The flow that was Tier 1 last year is replaced by a new one nobody told monitoring about. Three habits keep the list honest without much effort.


First, put monitoring on the release checklist. Any change that adds a page, a form, an endpoint, or a third party also adds or updates a check. And the developer who made the change is the one who knows what a healthy response looks like. Second, hold a short quarterly review with the owners in the inventory. Read the list aloud, ask what changed, ask what would make the phone ring now. Third, after every incident, ask the one question that improves inventories faster than anything else: did the phone ring before the alert did? If it did, the path that failed was missing from the list or checked too gently. In that case, the fix goes in that week.


Resist the urge to check everything. A list of two hundred checks with no owners is noise, and noise gets ignored. Twenty checks that each map to something the business would notice, each with a name on it, will catch the outages that matter. They will leave the pager quiet the rest of the time.


Where to go next


The home page is where customers arrive, not where they go. Follow them past it. Find the paths that would make the phone ring. Rank them by what breaking them costs. Watch the API behind the app and the third parties inside the page. Add content checks that read the body instead of trusting the status code. Check every front door's redirect. And write the whole thing down with an owner beside each line. Then keep it matched to the business through the release checklist and a quarterly conversation.


From here, the layers of a single web check shows what to do when one of these checks fails and you need to know which stage broke. And the ten-minute triage sequence turns that knowledge into a first-ten-minutes routine. Both belong to the Website & Web-Service Uptime Monitoring topic.


Put this into practice with SysPeep Monitor

Every path in your inventory ends at a server, a database, or a link somewhere in your own network. SysPeep Monitor watches those from one Windows machine. It has optional remote probes for other sites and segments, and auto-discovery to find what you forgot. Its dependency-aware alerting means a dead database raises one alert instead of one per page that needs it. The free trial installs in minutes.

Download Free TrialLearn About SysPeep