Availability & performance

Why agents are less patient than people

A person who hits a slow page waits a second, maybe refreshes. They give you the benefit of the doubt.

An agent doesn’t. It runs on a timeout and a budget. If your server is slow or down at the moment it calls, the agent doesn’t sigh and wait — it records a failure and moves on. Worse, a site that’s repeatedly slow or flaky gets quietly called less often next time.

So Level 0 reachability has two halves: are you up, and are you fast enough. Let’s handle both, plus one gotcha that’s unique to non-browser clients.

Step 1 — Stay up (and know the moment you don’t)

Availability is just the share of time your site is actually answering requests. The common bar is 99%+ — but the real goal is simpler: you should never be the last to know your site is down.

You can’t fix what you can’t see, so put an external monitor in place. It pings the site every minute from outside your network and alerts you the moment it stops responding.

1. Pick a monitor — UptimeRobot (free tier is fine), Better Uptime, or Pingdom. If the site is on Cloudflare, it has health checks too.
2. Point it at a real content URL, not just the homepage, and have it check for a 200 response and a word you know should be on the page.
3. Set alerts to somewhere you’ll actually see them — email plus SMS or Slack.

That second point matters more than it looks. A server can return a cheerful “200 OK” while serving a blank error page. Checking for expected text catches the failures a simple ping misses.

Step 2 — Be fast to answer

The single clearest signal of “is the server keeping up” is time to first byte (TTFB) — how long from the request until the server sends back the very first byte of its response. It’s the part of speed an agent feels first, and slow TTFB eats straight into its timeout budget.

You can measure it with one line of curl:

# Time to first byte, plus total time, for one URL
curl -o /dev/null -s -w "TTFB: %{time_starttransfer}s  Total: %{time_total}s\n" https://yourdomain.com

As a rough guide: under 0.8s is good, under 1.8s is acceptable, and anything north of that needs attention. If it’s slow, the usual fixes, in order of impact: cache pages so the server isn’t rebuilding them on every request, put a CDN like Cloudflare in front, trim slow database or API calls, and — last — move off the cheapest tier of shared hosting.

Beyond raw speed, there’s a standard scorecard worth knowing: Core Web Vitals. They’re three numbers Google uses to describe how a page feels, and they double as a clean, shared definition of “this page is genuinely fast and stable”:

• LCP — Largest Contentful Paint: how fast the main content shows up. Aim for under 2.5s.
• CLS — Cumulative Layout Shift: how much the page jumps around while loading. Aim for under 0.1.
• INP — Interaction to Next Paint: how quickly the page reacts to a tap or click. Aim for under 200ms.

Check yours in PageSpeed Insights (free, just paste a URL) or Google Search Console. The highest-leverage fixes are usually the boring ones: compress and right-size images, set explicit width and height on them so nothing jumps, lazy-load anything below the fold, and defer JavaScript that isn’t needed to draw the page.

Step 3 — Don’t hide your content behind JavaScript

Here’s the gotcha that’s specific to agents. Many of them — and plenty of crawlers — don’t run JavaScript at all, or run it with tight limits.

If your page is a shell that fills itself in with client-side JavaScript after it loads (a typical single-page app), then a browser sees a full page… and an agent that doesn’t run JS sees almost nothing. You can be up, fast, and still effectively invisible.

Check it the way an agent would — look at the raw HTML the server sends, before any JavaScript runs:

# Does your real content exist before JavaScript runs?
curl -s https://yourdomain.com | grep -i "a phrase from your page"

If the phrase comes back, you’re fine — the content is in the HTML. If it doesn’t, your content is being assembled in the browser, and you’ll want server-side rendering or pre-rendering so the important text ships in the initial response. Frameworks like Next.js, Nuxt, and Astro do this out of the box; classic WordPress and static sites already do.

That distinction trips up experienced developers constantly. When in doubt, trust curl.

With “up” and “fast” both handled, here’s what to take with you.

Worth bookmarking as you go:

§Level 0 · Reachable — read the exact requirements

Up and quick. Next, we make sure the response itself looks like it came from a competently-run origin — baseline security headers.