--- title: "Methodology | Agent Ready Checklist" description: "How requirements are chosen, how the Verify Kit tests, how the crawl slice is cut and how the spec diff keeps every AgentReady identifier on a page." canonical: https://agentreadychecklist.com/method site: "Agent Ready Checklist" last_updated: 2026-11-05 section: "Method" --- # Methodology | Agent Ready Checklist > How requirements are chosen, how the Verify Kit tests, how the crawl slice is cut and how the spec diff keeps every AgentReady identifier on a page. Source: https://agentreadychecklist.com/method Site: Agent Ready Checklist Last reviewed: 2026-11-05 Agent endpoints: https://agentreadychecklist.com/llms.txt (index), https://agentreadychecklist.com/openapi.json (API), https://agentreadychecklist.com/mcp (MCP server), https://agentreadychecklist.com/ask (search) Access: https://agentreadychecklist.com/auth.md (no key). Pricing: https://agentreadychecklist.com/pricing.md (free). # Methodology How this checklist is built. 24 rows come from AgentReady 2.0, 5 from Cloudflare and 2 from the Vercel guide, with a test that fails the build if a spec identifier has no page. The kit runs 15 checks with plain requests. The pass rates come from a crawl of 13,071 sites on 2026-11-04. ## How requirements are chosen The base is the AgentReady spec at version 2.0, commit 273d2d7, dated 2026-10-21: 24 live identifiers, 4 MUST, 14 SHOULD and 6 MAY. Every identifier is a page, and its URL keeps the identifier in lower case. Cloudflare's scanner adds checks with no AgentReady twin: CF-CONTENT-SIGNALS, CF-MARKDOWN-NEGOTIATION, CF-PAY-PER-CRAWL, CF-OAUTH-DISCOVERY, CF-AGENT-SKILLS. The Vercel guide adds two items that differ from the spec: VERCEL-NEGOTIATION-HEADERS and VERCEL-ROBOTS-EXCLUSIONS. That makes 31 rows. A row stays only if it has a primary source, a plain statement and a command a developer can run. Retired identifiers are not reused, and the AR-READ-10 slot stays empty. ### Where the sources disagree robots.txt default The spec asks for one wildcard group and a Sitemap line. The Vercel guide says a public site needs no robots.txt and rules are for deliberate exclusions. Cloudflare scores the presence of explicit AI bot groups. The checklist keeps the spec as AR-FIND-01 and the Vercel position as VERCEL-ROBOTS-EXCLUSIONS. [Source](https://vercel.com/kb/guide/make-your-site-readable-by-ai-agents). Markdown negotiation AgentReady has no requirement for it; AR-READ-09 covers only advertising a mirror. Cloudflare scores negotiation as a check. Vercel says that if you negotiate, send Content-Type text/markdown and Vary: Accept. Three rows keep the three positions apart: AR-READ-09, CF-MARKDOWN-NEGOTIATION and VERCEL-NEGOTIATION-HEADERS. [Source](https://github.com/agentready-org/standard/blob/main/spec/spec.md). MCP path AR-ACT-03 names /mcp. The Vercel guide calls /mcp a stable choice, not a protocol requirement. The data follows the spec and records the Vercel line as a fact. [Source](https://vercel.com/kb/guide/make-your-site-readable-by-ai-agents). Pay per crawl CF-PAY-PER-CRAWL stays as a row because Cloudflare documents the feature, but on 2026-11-04 its scanner listed no check for it, so the Checked by line says so. [Source](https://isitagentready.com/). WebMCP object The spec names document.modelContext. Cloudflare notes older Chrome builds use navigator.modelContext. The kit looks for both. [Source](https://isitagentready.com/). ## How the kit tests Each check makes plain HTTP requests with the user agent `Mozilla/5.0 (compatible; agent-readiness-crawl/1.0)`, follows at most five redirects and gives up after 15 seconds. It reads, it never writes: every request is a GET except one MCP initialize POST, which `--get-only` turns off. It runs no JavaScript, so a page that fills itself in with a script fails the initial-html check, which is the point. Each result is PASS, FAIL or SKIP, and the evidence is printed beside it. The exit code is 1 when any check fails. The 15 check pages list what each one cannot prove. The kit is not a scanner. It cannot see the IP address an agent arrives from, whether an answer is correct, or whether a tool in an MCP server works. For those, use a full scanner or your server logs. ## The crawl slice The pass rates on the data pages come from the shared crawl of 2026-11-04: 13,071 domains, one pass, a plain-fetch client with the same user agent and a 10-second timeout. The slice keeps the Tranco top 10,000 (list ID 746WX): 9,990 of them were in the crawl file at slice time and 7,840 were reachable. 9 hosts that the crawl flagged as IP-echo services, which answer every path with the caller's own address, are left out of every rate. Counting them as sites that fail instead would move the llms.txt share from 14.3% to 14.2%. The crawl stored extracted fields and the first 200 words of each home page, never full page bodies. `scripts/slice-crawl.mjs` cuts the slice, and a test recomputes published numbers from the rows and from the raw crawl. robots-all-17-allowed No robots.txt, or a robots.txt under which none of the 17 tokens is disallowed at /. A robots.txt that cannot be read (5xx, 401, 403) counts as not allowed. robots-no-search-or-user-block The Verify Kit verdict: no token other than the five training tokens (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent) is disallowed at /. initial-html The plain-fetch home page response carries an h1 and at least 50 words of visible text. Rows whose robots.txt disallowed the crawler from / were not fetched and are not counted. real-404 A random, never-linked path on the same host returns 404 or 410. A 200 is a soft 404. llms-present /llms.txt returns 200 with a non-HTML body. llms-linked The home page HTML contains an href to llms.txt or llms-full.txt. The link target is not fetched. markdown-negotiation A request for the home page with Accept: text/markdown answers with a text/markdown content type. The Vary header was not recorded, so the kit PASS is stricter than this rate. json-ld At least one application/ld+json block in the home page HTML and none that fails to parse. openapi-found The home page links an openapi or swagger file, or /openapi.json or /openapi.yaml returns 200 as JSON or YAML. The document is not validated. mcp-discovery-file /.well-known/mcp.json returns 200 with a JSON body. The server card path and /mcp were not probed by the shared crawl. ## The spec diff A test compares the 24 identifiers in the spec file with the requirement data and fails the build if a spec identifier has no page, or a page has no identifier in the spec or in the extra Cloudflare and Vercel rows. A second test checks that the level and layer on each page match the spec. The spec follows semantic versioning, and a major version marks a change to a normative requirement. The repository's latest commit was read again on 2026-11-05 and is still 273d2d7. - [AR-FIND-01](https://agentreadychecklist.com/checklist/ar-find-01/)MUST, find. Do not block agents. - [AR-FIND-02](https://agentreadychecklist.com/checklist/ar-find-02/)SHOULD, find. Answers on one first-party page. - [AR-FIND-03](https://agentreadychecklist.com/checklist/ar-find-03/)SHOULD, find. Sitemap listed in robots.txt. - [AR-FIND-04](https://agentreadychecklist.com/checklist/ar-find-04/)MAY, find. AI Catalog at .well-known. - [AR-READ-01](https://agentreadychecklist.com/checklist/ar-read-01/)MUST, read. Answer in the initial HTML. - [AR-READ-02](https://agentreadychecklist.com/checklist/ar-read-02/)MUST, read. Real HTTP status codes. - [AR-READ-03](https://agentreadychecklist.com/checklist/ar-read-03/)SHOULD, read. Home page with real links. - [AR-READ-04](https://agentreadychecklist.com/checklist/ar-read-04/)SHOULD, read. Docs that answer agent tasks. - [AR-READ-05](https://agentreadychecklist.com/checklist/ar-read-05/)SHOULD, read. Fenced, language-tagged code. - [AR-READ-06](https://agentreadychecklist.com/checklist/ar-read-06/)SHOULD, read. Link the discovery files. - [AR-READ-07](https://agentreadychecklist.com/checklist/ar-read-07/)SHOULD, read. Ship llms.txt. - [AR-READ-08](https://agentreadychecklist.com/checklist/ar-read-08/)SHOULD, read. JSON-LD on key pages. - [AR-READ-09](https://agentreadychecklist.com/checklist/ar-read-09/)MAY, read. Advertise a markdown mirror. - [AR-READ-11](https://agentreadychecklist.com/checklist/ar-read-11/)SHOULD, read. Pricing as first-party HTML. - [AR-ACT-01](https://agentreadychecklist.com/checklist/ar-act-01/)SHOULD, act. Auth an agent can finish. - [AR-ACT-02](https://agentreadychecklist.com/checklist/ar-act-02/)SHOULD, act. OpenAPI contract. - [AR-ACT-03](https://agentreadychecklist.com/checklist/ar-act-03/)SHOULD, act. MCP over Streamable HTTP. - [AR-ACT-04](https://agentreadychecklist.com/checklist/ar-act-04/)MAY, act. MCP server card. - [AR-ACT-05](https://agentreadychecklist.com/checklist/ar-act-05/)MAY, act. MCP Apps view. - [AR-ACT-06](https://agentreadychecklist.com/checklist/ar-act-06/)SHOULD, act. SDKs and a CLI. - [AR-ACT-07](https://agentreadychecklist.com/checklist/ar-act-07/)SHOULD, act. WebMCP tools. - [AR-ACT-08](https://agentreadychecklist.com/checklist/ar-act-08/)MAY, act. Agent payments. - [AR-ACT-09](https://agentreadychecklist.com/checklist/ar-act-09/)MAY, act. NLWeb /ask endpoint. - [AR-ACT-10](https://agentreadychecklist.com/checklist/ar-act-10/)MUST, act. An MCP server or a CLI. ### Retired AR-READ-10 AGENTS.md / SKILLS.md (MAY), removed August 2026, never reused. [Source](https://github.com/agentready-org/standard/commit/8f5cb02), read 2026-11-04. AR-DISC-\*, AR-CONT-\*, AR-CAPA-\*, AR-IDEN-\*, AR-COMM-\* v1.0.0 (2026-04-24) identifiers, all retired in the August 2026 v1.0 restructure. [Source](https://github.com/agentready-org/standard/blob/main/CHANGELOG.md), read 2026-11-04. ## Recipes 589 recipe entries cover 19 platforms. 357 were written from the platform's own documentation or from a live response read on 2026-11-04, with the version noted. Where a hosted builder cannot do something, the recipe says so and gives the workaround. Every platform has an entry for every row. 232 of the 589 entries were derived from the platform's documentation notes or from the specification rather than read page by page, and each says which on its page: "Derived from 's documentation read on 2026-11-04." for a step that names a platform feature, which cites the platform page that states it, or "Generic step from the AgentReady specification; not platform-specific." for a step that names none, which cites the requirement's own source. A derived entry is a recipe where the notes show the platform can do it, and otherwise the line "Not possible on this platform without outside help. Workaround: the real route" with the reason and the real workaround, labelled workaround. ## The Ora column and the pass rate line The Ora check column of [the table](https://agentreadychecklist.com/) names the checks in Ora's catalogue that read a row. The catalogue is the response of GET https://ora.ai/api/checks on 2026-11-05, saved unchanged: contract 1.38.0, 132 checks. A check is listed against a row only when its cited standard, its name or its published description matches the row's own text, or its id is the row's own name, and a test re-checks every entry. 40 checks are matched to 26 of 31 rows. The rest are left out, not guessed. [The crosswalk](https://agentreadychecklist.com/checklist/ora-checks/) shows every match with its basis and the CSV and JSON of the table. The line beside it is the pass rate of the row's crawl check in each of ten bands of 1,000 Tranco ranks, on one scale from 0 to 100 percent, with the overall share at its end. A row with no crawl check has none, and the cell says so. [The by-rank page](https://agentreadychecklist.com/data/by-rank/) draws the same lines large, with the numbers. ## Where Ora appears Ora's lab findings supply the evidence behind several rows. They are labelled as vendor claims and dated. The AgentReady spec this site follows is written by Ora and Vercel and is read at [agentready.org](https://agentready.org/?utm_source=agentreadychecklist.com&utm_medium=referral&utm_campaign=method). This site recommends Ora to verify the whole checklist, for the reasons and with the limits set out on [one page](https://agentreadychecklist.com/verify-with-ora/), and gives the CI recipe in [a guide](https://agentreadychecklist.com/guides/gate-deploys-on-agent-readiness-in-ci/). The kit stays the quick test for a single row. ## The API, the MCP server and the ask endpoint The rows the pages show are also served to agents three ways, documented on the [developers page](https://agentreadychecklist.com/developers/): an MCP server at `/mcp` (Streamable HTTP, five read-only tools over the requirement rows, the platform recipes, the kit checks and the pass rates), a REST API under `/api/v1/` described by [openapi.json](https://agentreadychecklist.com/openapi.json), and an NLWeb `/ask` endpoint that searches the rows and pages and returns schema.org objects with a source URL each. Every surface reads the published data files in `/data/` at request time, so a tool answer and the page it cites always come from the same edition. Nothing writes and nothing needs a credential; the site publishes no OAuth metadata because no endpoint takes a token, and [auth.md](https://agentreadychecklist.com/auth.md) says so. ## Sources - [github.com/agentready-org/standard/blob/main/spec/spec.md](https://github.com/agentready-org/standard/blob/main/spec/spec.md)read 2026-11-04 - [github.com/agentready-org/standard/blob/main/CHANGELOG.md](https://github.com/agentready-org/standard/blob/main/CHANGELOG.md)read 2026-11-04 - [vercel.com/kb/guide/make-your-site-readable-by-ai-agents](https://vercel.com/kb/guide/make-your-site-readable-by-ai-agents)read 2026-11-04 - [isitagentready.com](https://isitagentready.com/)read 2026-11-04 - [rfc-editor.org/rfc/rfc9309](https://www.rfc-editor.org/rfc/rfc9309)read 2026-11-04 - [tranco-list.eu](https://tranco-list.eu/)read 2026-11-04 - [ora.ai/api/checks, contract 1.38.0](https://ora.ai/api/checks?utm_source=agentreadychecklist.com&utm_medium=referral&utm_campaign=method)read 2026-11-05 Edition Data 2026-11-04 Reviewed 2026-11-05 Spec AgentReady v2.0, 273d2d7 Crawl 13,071 rows, 2026-11-04 Ora 132 checks, contract 1.38.0, 2026-11-05 Verify with Ora The scanner this site recommends for the whole checklist. 132 checks, a score out of 100, no account. `npx ax@0.10 audit --min-score ` [Why Ora, and its limits](https://agentreadychecklist.com/verify-with-ora/) Method - [How rows, commands and rates are made](https://agentreadychecklist.com/method/) - [How an Ora check is matched to a row](https://agentreadychecklist.com/checklist/ora-checks/) - [Pass rates by rank band](https://agentreadychecklist.com/data/by-rank/) Sources and read dates sit at the foot of the page. Last reviewed 2026-11-05. AgentReady v2.0, read 2026-11-04.