How to Make Your Site Agent-Ready

What is agent-readiness, and what makes a site usable by AI agents?

Agent-readiness is the measurable degree to which a website lets AI agents discover, read, access, use, transact with, and trust it — achieved by implementing a defined set of machine-readable standards across six dimensions, each of which the Agent-Readiness Audit can verify. It is the engineering counterpart to the Almanac: the Almanac explains the agentic web; this page tells you how to ship it.

Agent-readiness is processual, not a switch. It is a sequence of concrete changes — files at your site root, HTTP headers, structured markup, capability manifests — each tied to a named standard. "Ready" is not a subjective judgement: each standard either resolves correctly or it does not, which is exactly what makes readiness machine-verifiable.

How does agent-readiness differ from SEO and accessibility?

Agent-readiness targets autonomous machine clients, which separates it from two adjacent disciplines. Classic SEO optimizes a site for human-ranking search engines; web accessibility (WCAG) optimizes it for human assistive technology. Agent-readiness optimizes it for AI agents acting on a user’s behalf — programs that fetch, parse, and act on your pages without rendering them. The signals overlap (clean structure helps all three), but the client is different, and so is the proof: an agent-ready site is judged by whether a machine can complete a task on it.

Why does agent-ready mean machine-verifiable?

Every readiness claim on this site resolves to a standard plus a pass/fail check. You do not assert that your site is agent-ready; you implement a named standard (llms.txt, Web Bot Auth, JSON-LD), and a check confirms it resolves. That standards-and-proof model is why each how-to names both the spec it implements and the audit check it satisfies, and why this page closes into the Audit rather than ending in claims. Every how-to here implements a standard specified in the Agent Protocol Atlas.

Which six dimensions structure agent-readiness?

Agent-readiness has six dimensions — Discoverability, Content, Access control, Capabilities, Commerce, and Quality — and each maps to a cluster of standards and to a category of checks in the Agent-Readiness Audit. The first five mirror the categories Cloudflare’s public Agent Readiness Score scans for: its announcement of 17 April 2026 names Discoverability, Content, Bot Access Control and Capabilities, and its scanner at isitagentready.com lists Discoverability, Content Accessibility, Bot Access Control, Protocol Discovery and Commerce as of September 2026 (Cloudflare, 2026). Agents Welcome adds Quality as the structured, fresh, self-demonstrating dimension — the extension that makes the whole set machine-verifiable.

Discoverability: declaring your site to agents

The discoverability dimension is the Layer-1 declaration: it tells agents your site exists and where its important content lives, before any reading or acting happens. Member standards: llms.txt, llms-full.txt, AGENTS.md, agents.txt, agents.json, .well-known discovery, and NLWeb. Formal registration is rare: the IANA well-known-URI registry, last updated 19 August 2026, has no entry for llms.txt, agents.txt or agents.json; the registered API-discovery name is api-catalog (RFC 9727), which this site serves at /.well-known/api-catalog and advertises with an RFC 8288 Link header (IANA, 2026; IETF, 2025; IETF, 2017). AGENTS.md is the repository instruction file for coding agents, used by over 60,000 open-source projects (Agentic AI Foundation, 2026). Start here — the discoverability dimension declares your site to agents so everything downstream has something to find.

Content: serving machine-readable, answer-first content

The content dimension serves machine-readable, answer-first pages an agent can lift without parsing layout. Member standards: markdown twins, content negotiation (text/markdown), answer-first structure, and FAQ schema. The GEO benchmark study reports that content-side optimizations can raise a source’s visibility in generative-engine answers by up to 40% (Aggarwal et al., 2023) — the content dimension serves machine-readable, answer-first pages that agents quote verbatim.

Access control: letting the right agents in and verifying them

The access-control dimension governs which agents you admit and how you confirm their identity. Member standards: Web Bot Auth (RFC 9421, Ed25519 signatures), an AI-aware robots.txt, pay-per-crawl, RSL, and Content Signals. As of September 2026, Web Bot Auth is an adopted IETF working-group draft — draft-ietf-webbotauth-httpsig-protocol-00, dated 1 September 2026, Standards Track — built on RFC 9421 HTTP Message Signatures (IETF, 2026). robots.txt states policy, not enforcement: RFC 9309 says its rules "are not a form of access authorization" (IETF, 2022). Cloudflare’s Content Signals Policy (24 September 2025) adds search, ai-input and ai-train preferences to robots.txt and applies search=yes, ai-train=no by default on the 3.8 million-plus domains using Cloudflare’s managed robots.txt (Cloudflare, 2025); RSL 1.0 (10 December 2025) augments robots.txt’s yes/no rules with machine-readable licensing terms (RSL Collective, 2025). Verifying crawlers such as GPTBot and ClaudeBot turns an anonymous user-agent string into a cryptographically proven caller.

Capabilities: exposing tools an agent can call

The capabilities dimension exposes callable tools so an agent can act on your site, not just read it. Member standards: MCP, WebMCP, and the agents.json action manifest. WebMCP is a W3C Web Machine Learning Community Group draft dated 4 September 2026 — not a W3C Standard and not on the Standards Track (W3C WebML CG, 2026) — so treat it as experimental. This dimension is optional — implement it only when agents genuinely need to do something (book, query, submit) rather than merely retrieve.

Commerce: accepting agent payments and checkout

The commerce dimension lets agents pay and check out on your site. Member standards: x402, AP2, ACP, and UCP. It depends on everything above it — an agent cannot transact on a site it cannot discover, read, or be trusted on — so the commerce dimension enables agent payments and checkout as the last layer, not the first.

Quality: structured, fresh, retrievable and self-demonstrating

The quality dimension is the machine-verifiable, structured-data layer: it makes your facts extractable, fresh, and provable. Member standards: JSON-LD / schema.org, freshness signals (last_verified), markdown twins, and self-demonstration. This is the Agents Welcome extension to the field’s five-category model — the dimension that turns "we serve agents" into something an audit can score.

Which agent-readiness step should you ship first?

Ship Discoverability first, then Content: implement agent-readiness in dependency order, because an agent must find and read you before it can be admitted, call a tool, or pay.

1. Discoverability first — ship llms.txt and AGENTS.md at your root so an agent can find your important pages. Prerequisite for everything else.
2. Content next — add markdown twins and answer-first structure so the pages it found are actually readable by a machine.
3. Access control — add Web Bot Auth and an AI-aware robots policy so you can admit and verify the right agents now that they can read you.
4. Capabilities — publish agents.json / an MCP surface only if agents need to call tools, not just read.
5. Commerce — wire x402 / AP2 once discovery, reading, and trust are in place, because payment is the highest-trust action.
6. Quality, throughout and last — JSON-LD, freshness, and self-demonstration harden every layer and make the whole set machine-verifiable.

The minimum viable agent-ready site is steps 1–2: discoverable and readable.

How do you implement each agent-readiness standard?

Each agent-readiness how-to implements one standard in three moves — add the artifact, point it at your content, and confirm it resolves — and each ends the same way: verify it with the Agent-Readiness Audit. The cards below are the on-hub summaries; the full step-by-step for each lives on its own /agent-readiness/<how-to> subpage.

How do you declare your content to agents with llms.txt?

Add an llms.txt file at your site root that lists your most important pages as a clean, linkable index for agents. It implements the llms.txt declaration standard — proposed by Jeremy Howard of Answer.AI on 3 September 2024, with an H1 as the only required section and H2-sectioned link lists as the payload (Answer.AI, 2024) — recorded at the llms.txt protocol entry. Keep expectations honest: SE Ranking found the file on 10.13% of nearly 300,000 studied domains and no measurable effect on AI citations (SE Ranking, 2025), so ship it because it costs minutes, not because it ranks. It satisfies audit check llms-txt. Self-demo: this site serves /llms.txt live. Then verify it with the Agent-Readiness Audit.

How do you publish an agents.json capability manifest?

Publish an agents.json manifest that declares the agent-facing actions and capabilities your site exposes. It implements the agents.json declaration pattern — agents.json declares your agent-facing capabilities — for which no standards body publishes a specification and IANA registers no well-known name (IANA, 2026); this site documents its own manifest shape at /.well-known/agents.json. It satisfies audit check agents-manifest. Then verify it with the Agent-Readiness Audit.

How do you serve markdown twins of every page?

Serve a .md copy of every HTML page so agents can read your content without parsing layout markup. It implements the markdown-twin content pattern (the content dimension; no standalone protocol entity), using the text/markdown media type registered by RFC 7763 (IETF, 2016). It satisfies audit check markdown-negotiation. Self-demo: every page on this site has a .md twin — append .md to this URL. Then verify it with the Agent-Readiness Audit.

How do you mark up entities with schema.org JSON-LD?

Mark up your entities with schema.org JSON-LD so an agent can extract structured facts, not just prose. It implements schema.org structured data — JSON-LD uses schema.org for agents to expose entities as machine-readable facts — on two stable foundations: JSON-LD 1.1 has been a W3C Recommendation since 16 July 2020 (W3C, 2020), and the current vocabulary release is schema.org 30.0 of 19 March 2026 (Schema.org, 2026). It satisfies audit check json-ld. Self-demo: this page ships an @graph JSON-LD block. Then verify it with the Agent-Readiness Audit.

How do you verify visiting agents with Web Bot Auth?

Verify visiting agents by checking their Ed25519-signed requests under RFC 9421 and publishing your own key directory at /.well-known/http-message-signatures-directory. It implements Web Bot Auth, specified at its identity-layer entry under RFC 9421: RFC 9421 defines the ed25519 signature algorithm (IETF, 2024), and the IETF Web Bot Auth working group published its first working-group draft of the protocol, dated 1 September 2026 (IETF, 2026). It satisfies audit check web-bot-auth and lets you verify the agents catalogued in the crawler registry. Follow the dedicated Web Bot Auth how-to to implement agent verification on your own site; see it defined as a term in the Lexicon first if needed. Then verify it with the Agent-Readiness Audit.

How do you return text/markdown to agents that ask?

Return text/markdown (or JSON) when an agent sends Accept: text/markdown, so the same URL serves humans and machines the right format. It implements HTTP content negotiation for agents (media type per RFC 7763; the content dimension). It satisfies audit check vary-accept. Self-demo: request this page with Accept: text/markdown and you get its .md twin. Then verify it with the Agent-Readiness Audit.

What is on the agent-readiness checklist, and what does each standard need to pass?

The agent-readiness checklist maps 14 standards to the six readiness dimensions, the how-to that ships each one, and its pass criterion — the engineering view, how you get ready. What the Agent-Readiness Audit (POST /api/audit) then scores is a related set: 18 weighted checks across four categories — Discovery (25), Content (39), Capability (17) and Trust (19), summing to 100 (Agents Welcome, 2026); the check ids include llms-txt, markdown-negotiation, json-ld, agents-manifest, web-bot-auth and agent-welcome-header. The table’s key takeaway: every row ends in a criterion a machine can test, which is what lets the checklist double as your implementation plan.

Unsure how the root files relate — whether llms.txt replaces robots.txt, and whether agents.txt is worth shipping at all? Read llms.txt vs robots.txt vs agents.txt compared: what each file controls, verified against the primary specs.

How can you verify live that this site is agent-ready?

You can verify every how-to on this page against this site itself: it ships an llms.txt, an agents.json, a markdown twin per page, schema.org JSON-LD, a Web Bot Auth verifier, and content negotiation, and each is fetchable before you trust the advice. Its own Agent-Readiness Audit score is rendered at build from these live artifacts, not hardcoded. The hard-to-replicate element behind these pages is the site’s own data: the AI Crawler Registry holds 41 crawler records (updated 6 Jul 2026) and the Agent Protocol Atlas 28 protocol records (updated 30 Jul 2026), each carrying its source URLs and a last_verified date (Agents Welcome, 2026), and every readiness claim resolves to a live endpoint — /api/audit, /.well-known/agents.json, /.well-known/api-catalog, /api/whoami.

• llms.txt — fetch /llms.txt (proves the llms.txt how-to).
• agents.json — fetch /.well-known/agents.json (proves the agents.json how-to).
• Markdown twin — append .md to this URL, or fetch /agent-readiness.md (proves the markdown-twins how-to).
• JSON-LD — read the @graph block in this page’s head (proves the JSON-LD how-to).
• Content negotiation — send Accept: text/markdown to this URL (proves the content-negotiation how-to).
• Web Bot Auth — sign a request with Ed25519 and check /api/whoami; trusted keys are listed at /data/trusted-keys.json (proves the Web Bot Auth how-to).

Because every check is what the Agent-Readiness Audit verifies (POST /api/audit), an agent can read what this site claims and test each claim against the matching live path — the honest version of "agent-ready."

Agent-readiness — frequently asked questions

Is agent-readiness the same as SEO?

No. Classic SEO optimizes a site for human-ranking search engines, and web accessibility (WCAG) optimizes it for human assistive technology. Agent-readiness targets autonomous machine clients: it implements machine-readable standards so AI agents can discover, read, access, use, transact with, and trust your site, and so each can be verified pass or fail.

Which agent-readiness step should I implement first?

Ship in dependency order. Start with Discoverability (llms.txt, AGENTS.md) and Content (markdown twins, answer-first) because an agent must find and read you before anything else; then add Access control (Web Bot Auth, robots policy); then Capabilities (MCP, agents.json) only if agents need to act; and reach Commerce (x402, AP2) and the full Quality layer last.

What does the Agent-Readiness Audit check?

The Audit (POST /api/audit) runs 18 weighted checks across four categories — Discovery, Content, Capability and Trust — and the six dimensions on this page are the implementation view those checks roll up into. Each check has a machine-checkable pass criterion — the standard either resolves correctly or it does not — so the checklist doubles as your implementation plan.

Is agent-readiness a single setting or a process?

It is a process, not a switch. Agent-readiness is a sequence of concrete changes — files, headers, markup, manifests — each tied to a named standard. 'Ready' is not subjective: each standard either resolves correctly or it does not, which is what makes readiness machine-verifiable.

Agent-readiness border: when do you stop implementing and start certifying?

Stop adding how-to detail once every standard on the checklist resolves; from there the open question — did you ship them correctly, and can you prove it to the agents and customers that matter? — is answered only by running the verification. This guide shows you how to ship each standard; the Agent-Readiness Audit checks every one, scores your site with 18 weighted checks across four categories (Discovery, Content, Capability, Trust), and — when you pass — certifies it as Agents Welcome.

Ship each step, then the Agent-Readiness Audit checks every standard and certifies your site. Every readiness step implements a standard specified in the Agent Protocol Atlas; unsure of a term? Every standard here is defined in the Agentic Web Lexicon first. An agent-ready site lets the frontier models in the Model Matrix complete tasks for users, and adoption of each readiness standard is measured over time in the State of the Agentic Web report.