Agent-Ready Websites: The 5-Layer Implementation Stack

Every website serves two audiences: humans and machines. Crawlers have always parsed content; agents go further, calling tools, mutating state, and chaining decisions across domains. Sites can now expose machine-readable discovery, structured content, and callable tools, yet many still serve only the human-facing layer, invisible to the automated discovery pipelines that increasingly mediate how users find and interact with content.

Agent readiness is distinct from SEO and from AEO (Answer Engine Optimization): SEO wants visibility, AEO wants citation, agent readiness wants addressability. They share signals (sitemap, robots.txt, structured data) but serve different callers. This post maps agent readiness as capability (read, parse, act) and as a five-layer implementation stack, each layer deployed, scored, and audited.

Table of contents

Why Does Your Website Now Have Two Audiences?

Agents split into readers and actors depending on which layer the site exposes. The Readable layer is ordinary HTML, styled and interactive, built for a human with eyes and a mouse. The Parseable layer is structured data that an agent can fetch without executing JavaScript. The Executable layer exposes callable functions in the browser through document.modelContext, the WebMCP (Model Context Protocol) Imperative API, a W3C Web Machine Learning Community Group Draft in Chrome origin trial. A parseable-only site lets an agent read; an executable site lets it act, subject to the author’s safety annotations.

Accessibility is agent readiness: a clean accessibility tree, ARIA labels, and programmatic names on interactive elements are how an agent reads a layout. Semantic HTML is not optional polish; it is machine-readable semantics at zero marginal cost (Google Chrome, 2026).

Readable, parseable, and executable describe capability. Table 1 maps the implementation stack that realizes those outcomes. Scoring platforms now grade sites against that stack, so “agent-friendly” is a number a CTO can track quarter over quarter instead of a marketing claim.

What Does “Fully Agent-Ready” Look Like?

Five layers stacked in sequence let an agent discover, fetch, and act without human relay. Each resolves to a concrete endpoint backed by a published or draft standard:

Three production deployments demonstrate that full agent readiness is achievable across different site types:

• joost.blog runs a stateless /mcp endpoint, giving agents a persistent Model Context Protocol server alongside in-page tools.
• WorkOS generates its MCP tool definitions from the same source files as its HTML documentation, so the two surfaces cannot drift apart.
• Coinranking exposes live cryptocurrency chart data through WebMCP, proving the pattern works for data that updates every few seconds, not just static content.

Cloudflare’s One-Click Bridge and Its Ceiling

One-click agent bridges change the cost calculus for smaller sites (Cloudflare, 2026). Cloudflare’s bridge uses HTMLRewriter to inject a baseline WebMCP surface automatically. That free baseline provides basic tool registration and content negotiation. A hand-rolled implementation adds domain-specific tools tied to a curated knowledge graph, human-in-the-loop annotations tuned per action, and a semantic layer that the bridge cannot infer from generic page content. The scoring gap is that domain-specific layer, a pattern that extends into multi-agent governance as examined in AI SDLC Governance.

How Is the WebMCP Tool Layer Built?

Competitive differentiation concentrates in the executable layer. Any site can publish /llms.txt (llmstxt.org, 2026) in an afternoon, but domain-specific tools backed by a curated knowledge graph take as long to build as the content itself.

Four tools register via AgentReady.astro, each wired to its own AbortController for explicit cleanup on component unmount. Each page registers only the subset that fits its kind: the home page, a post, or an archive.

The API entry point is document.modelContext, exposed by the W3C Web Machine Learning Community Group Draft (W3C, 2026); Chrome deprecated the older navigator.modelContext getter in Chrome 150. Feature detection guards the registration: if document.modelContext is undefined, the component skips tool registration entirely and logs a warning to the console.

const mc = document.modelContext;
const ac_search = new AbortController();
mc.registerTool(  
  {
    name: "search_posts",
    description: "Search blog posts by topic or keyword.",
    inputSchema: {
      type: "object",
      properties: { query: { type: "string" } },
      required: ["query"],
    },
    annotations: { readOnlyHint: true },
    execute: async function (input) {
      return (await window.__pagefindInit).search(input.query);
    },
  },
  { signal: ac_search.signal }
);src/components/AgentReady.astro

Read-Only vs. Mutating: The Annotation Design

Two safety classes govern WebMCP tools. All four tools on this site are read-only and carry annotations: { readOnlyHint: true }, signaling that they cannot mutate state. A write tool omits the annotation deliberately; its description instead states the side effect and requires user confirmation before the call. Figure 1 shows the decision flow for classifying a new tool.

Applying the Pattern

Cloudflare’s bridge cannot infer which page actions mutate state and which do not. A hand-rolled tool layer is the only place that judgment can currently live.

Start by classifying every action as read-only or state-mutating before writing a tool. For write-path tools, the description string, not the schema, carries the confirmation requirement, because it is the one field an agent reads before deciding whether to call. Server-side rate limiting on write-path endpoints and input validation on inputSchema arguments remain the trust boundary; client-side annotations are advisory, not enforcement. Chrome’s proposed Decisions API would add one more client-side pre-screen: an on-device “Semantic If” that scores predefined candidate options without token generation, so a page can route ambiguous input or gate a mutating call before anything leaves the browser (Chrome Platform Status, 2026).

What Is the Knowledge Graph Layer and Why Does It Matter?

Exposing content relationships as structured data makes a static site queryable. /knowledge-graph.json captures typed edges (citation, concept_shared, thematic_chain, and section_reference), concept clusters, and curated reading chains, built at Astro build time from curated data in src/data/knowledge-graph.ts and frontmatter extracted from every post.

Clusters, Edges, and Reading Chains

Two of the four WebMCP tools expose this graph directly: get_related_posts walks the typed edges from a given slug, and get_posts_by_concept returns every post tagged into a given cluster. The context isolation patterns that make multi-agent graphs reliable are covered in Context Engineering; the orchestration layer that coordinates agents across those graphs is examined in Orchestrating AI Agents.

Beyond the tools, every post publishes a /posts/{slug}.json agent contract summarizing its own graph position. The site’s RSS feed carries Dublin Core plus graph metadata for agents that prefer feed polling over live queries. At call time, get_related_posts fetches /knowledge-graph.json, filters edges by slug, and returns each connected post’s title and edge type.

Cloudflare’s generic bridge can fetch pages but cannot infer which posts cite one another or share concepts. That editorial judgment lives in the curated knowledge-graph.ts; the suggest-cluster CLI keeps it consistent as content grows.

When Does a Knowledge Graph Pay Off?

In production testing, the graph starts to pay off once cross-links become dense, past roughly two dozen posts. Below that point, a curated /llms.txt reading chain covers most of the agent value.

How Do Agents Actually Discover All of This?

An agent lands on the domain with zero prior knowledge and follows a deterministic chain through every layer above. Discovery can start at DNS: a DNS-AID index at _index._agents.<domain> advertises agent capability before HTTP. DNS-AID is an advancing IETF draft (IETF, 2026); the wire format may still shift. Without it, agents still enter via robots.txt and Link headers. From there, robots.txt declares crawl policy and Content-Signals opt-in or opt-out for named AI bots, categorized by function rather than left as one undifferentiated block.

Link Headers as Connective Tissue

Link response headers carry the next hop: the describedby, api-catalog, mcp-server-card, agent-skills, and ai-catalog relations point an agent directly at machine-readable resources without requiring it to guess well-known paths. Table 3 lists the endpoints reachable from each relation.

Content negotiation is the most consequential signal in the stack. Honor RFC 9110 q-values at the root and return structured JSON even on 404. Media type is part of the signal: a route that answers HTML where XML or JSON is expected reads as absent to a grader, not present.

Per-post markdown at /posts/{slug}.md and per-post JSON contracts at /posts/{slug}.json are the concrete endpoints that satisfy this signal; the markdown frontmatter carries a canonical url, author, and tags, so an agent that fetches only the markdown can still cite the HTML page. The api-catalog relation follows RFC 9727 (Smith, 2025), the well-known URI and link relation for API discovery, and the post page surfaces the markdown endpoint to readers through a Copy for LLM action.

An auth.md file and OAuth discovery metadata state explicitly that anonymous access is supported for read-only resources, removing a common ambiguity where agents cannot tell whether authentication is required or merely unimplemented. An HTTP signatures directory covers message-level integrity via Web Bot Auth, built on HTTP Message Signatures (Backman et al., 2024). Packaged skills at /.well-known/agent-skills/index.json carry SHA-256 digests so an agent can verify each SKILL.md bit-for-bit before loading it, which blocks silent tampering between publish and fetch.

{
  "$schema": "https://schemas.modelcontextprotocol.io/server-card/draft/schema.json",
  "serverInfo": {
    "name": "clouatre-ca",
    "title": "clouatre.ca",
    "version": "1.0.0"
  },
  "description": "Technical blog for CTO/CIO/CISO audiences covering AI, MCP, and agentic systems.",
  "iconUrl": "https://clouatre.ca/favicon.svg",
  "documentationUrl": "https://clouatre.ca/llms.txt"
}public/.well-known/mcp.json

Figure 2 places the server card within the broader discovery stack, showing how every layer from DNS to interactive tools connects through a deterministic chain.

How Do You Measure Agent Readiness?

Use scores to compare discovery, content, actionability, and trust gaps across sites. Four independent platforms now grade sites against these layers. Table 4 compares them.

What Do Scores Actually Tell You?

The average top-100 site scores 55% on AgentGrade. 65% lack even an /llms.txt file, and 100% fail to link agent-discovery files from their homepage (AgentGrade, 2026). Running two complementary scales in the same session separates discovery failures from content-structure failures. Scores are snapshots, since platforms add checks as standards settle and optional, low-cost signals can move the number without a new endpoint. The operational output is the next gap to close and the order for doing so. This site’s public scans demonstrate the full stack from Table 1 in production; the rubric can be inspected against a concrete example rather than taken on faith (isitagentready.com, 2026; AgentGrade, 2026). The same audit runs on demand for any domain through the site’s paid agent-readiness endpoint, described in x402 on Cloudflare Workers.

What Should Engineering Leaders Ship First?

Start with the cheap, reversible layers: /robots.txt, /llms.txt, and Link headers, which any static host can serve in an afternoon. Those discovery surfaces are durable. Then register read-only WebMCP tools with explicit readOnlyHint annotations so agents can act without mutating state. A follow-up eval of this site’s tools found that bounded responses and relevance ordering mattered more than how many tools each page exposes. A curated knowledge graph earns its keep once cross-links become sufficiently dense to make traversal worthwhile. Ship mutating tools last, with explicit confirmation language and a clear trust boundary separating what an agent may change from what requires a human. In-browser WebMCP tools remain a Community Group Draft; ship them behind feature detection, not as a load-bearing dependency.

A sixth category, payment protocols, applies to sites with metered or payable resources; the x402 HTTP 402 price-quote header is the reference implementation, and this site now runs two live x402 endpoints documented in x402 on Cloudflare Workers.

References

• AgentGrade, “Agent Readiness Audit” (2026) — https://agentgrade.com
• AgentScope, “Agent Readiness Scoring” (2026) — https://agentscope.pro
• Backman, A. et al., “HTTP Message Signatures” (2024) — https://www.rfc-editor.org/rfc/rfc9421
• Chrome Platform Status, “Decisions API” feature entry (2026) — https://chromestatus.com/feature/5155080092385280
• Cloudflare, “WebMCP” (2026) — https://blog.cloudflare.com/webmcp
• Coinranking, “Live Chart Data via WebMCP” (2026) — https://coinranking.com
• Google Chrome, “Lighthouse Agentic Browsing Scoring” (2026) — https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring
• IETF, “DNS for AI Discovery” (2026) — https://datatracker.ietf.org/doc/draft-mozleywilliams-dnsop-dnsaid/
• isitagentready.com, “Is It Agent Ready?” (2026) — https://isitagentready.com
• joost.blog, “Stateless MCP Endpoint” (2026) — https://joost.blog
• llmstxt.org, “The /llms.txt File” (2026) — https://llmstxt.org/
• Smith, K., “api-catalog: A Well-Known URI and Link Relation to Help Discovery of APIs” (2025) — https://www.rfc-editor.org/rfc/rfc9727
• W3C, “WebMCP” (2026) — https://webmachinelearning.github.io/webmcp/
• WorkOS, “Unified MCP and Documentation Source” (2026) — https://workos.com