--- title: "The five most failed rows and their fixes | Agent Ready Checklist" description: "The five checklist rows that apply to every site and fail most in a crawl of 7,840 sites, each with the fix on 19 platforms, the Ora check and the kit command." canonical: https://agentreadychecklist.com/data/most-failed-requirements site: "Agent Ready Checklist" last_updated: 2026-11-05 date: 2026-11-04 section: "Data" --- # The five most failed rows and their fixes | Agent Ready Checklist > The five checklist rows that apply to every site and fail most in a crawl of 7,840 sites, each with the fix on 19 platforms, the Ora check and the kit command. Source: https://agentreadychecklist.com/data/most-failed-requirements Site: Agent Ready Checklist Last reviewed: 2026-11-05 Agent endpoints: https://agentreadychecklist.com/llms.txt (index), https://agentreadychecklist.com/openapi.json (API), https://agentreadychecklist.com/mcp (MCP server), https://agentreadychecklist.com/ask (search) Access: https://agentreadychecklist.com/auth.md (no key). Pricing: https://agentreadychecklist.com/pricing.md (free). # The five most failed rows, and the fix for each Of the rows that apply to every site, five fail most often in the crawl of 2026-11-04: AR-READ-07 at 85.8%, AR-READ-08 at 52.6%, AR-READ-01 at 36.0%, AR-READ-02 at 33.5%, AR-FIND-01 at 28.8%. Each section gives the fix, the platforms where it is direct, the Ora check that reads it and the kit command that tests it. ## The ranking The ranking takes the rows whose "applies to" line begins with every, because a row that applies only to products with an API or an MCP server is not a fair thing to fail a news site for. Each row stands for its crawl check. A fail share is the share of reachable sites on which that check ran and did not pass. Crawl of 2026-11-04, 7,840 reachable sites in the Tranco top 10,000. Red is the share that fails, green the share that passes. Numbers behind the chart | Row | Fail | Pass | | --- | --- | --- | | AR-READ-07 | 85.8% | 14.3% | | AR-READ-08 | 52.6% | 47.4% | | AR-READ-01 | 36.0% | 64.0% | | AR-READ-02 | 33.5% | 66.5% | | AR-FIND-01 | 28.8% | 71.2% | Left out: AR-READ-06, AR-READ-09, AR-ACT-02, AR-ACT-03, AR-ACT-10, CF-MARKDOWN-NEGOTIATION, which apply to a narrower set of sites and are on [the pass rates page](https://agentreadychecklist.com/data/) with their rates. ## 1\. AR-READ-07: Ship llms.txt Fails in the crawl **85.8%**, 6,443 of 7,514 sites tested The crawl check /llms.txt exists. /llms.txt returns 200 with a non-HTML body. The row [AR-READ-07](https://agentreadychecklist.com/checklist/ar-read-07/) is SHOULD. Ship an llms.txt: a lean, described index of the key pages. Ora llms-txt-exists, llms-txt-formatting, llms-txt-links-resolve in Ora's catalogue of 2026-11-05. ### The fix on three platforms #### [Static hosting](https://agentreadychecklist.com/platforms/static-hosting/) recipe Upload llms.txt to the publish root; set Content-Type text/plain; charset=utf-8 in \_headers if the host guesses wrong. Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). [docs.netlify.com/manage/routing/headers/](https://docs.netlify.com/manage/routing/headers/) #### [Next.js](https://agentreadychecklist.com/platforms/nextjs-app-router/) recipe Write public/llms.txt (H1, summary blockquote, link lists) or generate it in app/llms.txt/route.ts with export const dynamic = "force-static" and Content-Type text/plain. Next.js 16.x (npm latest 16.4.5, read 2026-11-04); docs at nextjs.org/docs/app. [nextjs.org/docs/app/api-reference/file-conventions/route](https://nextjs.org/docs/app/api-reference/file-conventions/route) #### [WordPress, Elementor](https://agentreadychecklist.com/platforms/wordpress-elementor/) workaround Core has none. Turn on llms.txt in Yoast SEO (Settings > Site features > APIs) or Rank Math (General Settings > llms.txt), or upload a hand-written llms.txt to the web root. WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. [yoast.com/features/llms-txt/](https://yoast.com/features/llms-txt/) Across all 19 platforms, 14 have a direct recipe, 5 a workaround and 0 say it cannot be done on the platform. The row page lists every one: [AR-READ-07](https://agentreadychecklist.com/checklist/ar-read-07/). ### Check it ```bash curl -fsSL https://agentreadychecklist.com/kit/verify.sh | bash -s -- https://example.com --only=llms-linked ``` It cannot prove this: llms.txt does not control crawling or grant or deny access. ## 2\. AR-READ-08: JSON-LD on key pages Fails in the crawl **52.6%**, 3,097 of 5,890 sites tested The crawl check Home page carries JSON-LD that parses. At least one application/ld+json block in the home page HTML and none that fails to parse. The row [AR-READ-08](https://agentreadychecklist.com/checklist/ar-read-08/) is SHOULD. Put JSON-LD on key pages, the homepage above all, and repeat critical facts in visible text. Ora json-ld in Ora's catalogue of 2026-11-05. ### The fix on three platforms #### [Static hosting](https://agentreadychecklist.com/platforms/static-hosting/) recipe Put the JSON-LD script in the HTML at build time; the host serves it as uploaded. Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). [docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages](https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages) #### [Next.js](https://agentreadychecklist.com/platforms/nextjs-app-router/) recipe In the page component render '; });` or rely on the Yoast or Rank Math schema graph. Repeat key facts in visible text. WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. [developer.wordpress.org/reference/hooks/wp\_head/](https://developer.wordpress.org/reference/hooks/wp_head/) Across all 19 platforms, 15 have a direct recipe, 4 a workaround and 0 say it cannot be done on the platform. The row page lists every one: [AR-READ-08](https://agentreadychecklist.com/checklist/ar-read-08/). ### Check it ```bash curl -fsSL https://agentreadychecklist.com/kit/verify.sh | bash -s -- https://example.com --only=json-ld ``` It cannot prove this: JSON-LD is not a ranking or retrieval switch; it changed nothing in the 19-configuration experiment. ## 3\. AR-READ-01: Answer in the initial HTML Fails in the crawl **36.0%**, 2,123 of 5,890 sites tested The crawl check Home page has an h1 and 50 words in the initial HTML. The plain-fetch home page response carries an h1 and at least 50 words of visible text. Rows whose robots.txt disallowed the crawler from / were not fetched and are not counted. The same check stands for AR-READ-03. The row [AR-READ-01](https://agentreadychecklist.com/checklist/ar-read-01/) is MUST. Put the answer in the initial HTML: server-rendered or prerendered, not injected by a single-page app after load. Ora content-no-js in Ora's catalogue of 2026-11-05. ### The fix on three platforms #### [Static hosting](https://agentreadychecklist.com/platforms/static-hosting/) recipe Static HTML is the initial HTML. Pre-render every route at build time; if the build output is an SPA shell, the host cannot fix that. Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). [developers.cloudflare.com/pages/configuration/serving-pages/](https://developers.cloudflare.com/pages/configuration/serving-pages/) #### [Next.js](https://agentreadychecklist.com/platforms/nextjs-app-router/) recipe Keep pages as Server Components (the App Router default) so content is in the HTML response. Fetch data in the page, not in a useEffect. For fully static output set output: "export" in next.config.js. Next.js 16.x (npm latest 16.4.5, read 2026-11-04); docs at nextjs.org/docs/app. [nextjs.org/docs/app/getting-started/server-and-client-components](https://nextjs.org/docs/app/getting-started/server-and-client-components) #### [WordPress, Elementor](https://agentreadychecklist.com/platforms/wordpress-elementor/) recipe PHP templates render on the server, so post content is in the initial HTML. Check that no block, Elementor widget or plugin loads the answer by AJAX after load: `curl -s https://example.com/page/ | grep -c 'answer phrase'`. WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. [developer.wordpress.org/themes/classic-themes/basics/template-hierarchy/](https://developer.wordpress.org/themes/classic-themes/basics/template-hierarchy/) Across all 19 platforms, 19 have a direct recipe, 0 a workaround and 0 say it cannot be done on the platform. The row page lists every one: [AR-READ-01](https://agentreadychecklist.com/checklist/ar-read-01/). ### Check it ```bash curl -fsSL https://agentreadychecklist.com/kit/verify.sh | bash -s -- https://example.com --only=initial-html ``` It cannot prove this: Adding llms.txt or a markdown mirror does not fix a JavaScript-only page for a client that reads the HTML. ## 4\. AR-READ-02: Real HTTP status codes Fails in the crawl **33.5%**, 2,515 of 7,500 sites tested The crawl check A missing path returns 404 or 410. A random, never-linked path on the same host returns 404 or 410. A 200 is a soft 404. The row [AR-READ-02](https://agentreadychecklist.com/checklist/ar-read-02/) is MUST. Return accurate HTTP status codes: 404 or a real redirect for missing paths, 429 with Retry-After when throttling; never 200 for a missing page and never a silent challenge page in place of 429. Ora redirect-hygiene in Ora's catalogue of 2026-11-05. ### The fix on three platforms #### [Static hosting](https://agentreadychecklist.com/platforms/static-hosting/) recipe Ship a top-level 404.html. Netlify, Cloudflare Pages and GitHub Pages serve it with status 404. On Cloudflare Pages a missing 404.html switches on SPA mode (index.html with 200 for every path). On CloudFront set a custom error response with response code 404, not 200. Remove any "/\* /index.html 200" rule from \_redirects. Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). [developers.cloudflare.com/pages/configuration/serving-pages/](https://developers.cloudflare.com/pages/configuration/serving-pages/) #### [Next.js](https://agentreadychecklist.com/platforms/nextjs-app-router/) recipe Call notFound() from next/navigation when a record is missing; Next.js renders not-found.js with a 404. For throttling, return new Response("Too Many Requests", { status: 429, headers: { "Retry-After": "60" } }) from proxy.ts or a Route Handler. Next.js 16.x (npm latest 16.4.5, read 2026-11-04); docs at nextjs.org/docs/app. [nextjs.org/docs/app/api-reference/functions/not-found](https://nextjs.org/docs/app/api-reference/functions/not-found) #### [WordPress, Elementor](https://agentreadychecklist.com/platforms/wordpress-elementor/) recipe Keep a 404.php template; core sends 404 for unknown URLs. Redirect moved pages with 301. When a rate-limit plugin throttles, make it send `status_header(429); header('Retry-After: 120');` instead of a 200 challenge page. WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. [developer.wordpress.org/reference/functions/status\_header/](https://developer.wordpress.org/reference/functions/status_header/) Across all 19 platforms, 17 have a direct recipe, 2 a workaround and 0 say it cannot be done on the platform. The row page lists every one: [AR-READ-02](https://agentreadychecklist.com/checklist/ar-read-02/). ### Check it ```bash curl -fsSL https://agentreadychecklist.com/kit/verify.sh | bash -s -- https://example.com --only=status-codes ``` It cannot prove this: A correct 404 does not tell the agent where the content moved; add a redirect for that. ## 5\. AR-FIND-01: Do not block agents Fails in the crawl **28.8%**, 2,260 of 7,840 sites tested The crawl check robots.txt allows all 17 agent tokens at /. No robots.txt, or a robots.txt under which none of the 17 tokens is disallowed at /. A robots.txt that cannot be read (5xx, 401, 403) counts as not allowed. The row [AR-FIND-01](https://agentreadychecklist.com/checklist/ar-find-01/) is MUST. Do not block agent and crawler access to public content, in robots.txt or at the network layer (WAF, bot manager). Throttle with 429, not a challenge page. Ora robots-ai-policy-quality, robots-agent-user-policy, agent-crawler-reachability, bot-detection in Ora's catalogue of 2026-11-05. ### The fix on three platforms #### [Static hosting](https://agentreadychecklist.com/platforms/static-hosting/) recipe Upload robots.txt with "User-agent: \*" and "Allow: /". On Cloudflare Pages also check the zone Bot Fight Mode and AI Crawl Control settings, which apply before your file. Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). [developers.cloudflare.com/pages/configuration/serving-pages/](https://developers.cloudflare.com/pages/configuration/serving-pages/) #### [Next.js](https://agentreadychecklist.com/platforms/nextjs-app-router/) recipe In app/robots.ts return rules: \[{ userAgent: "\*", allow: "/" }\] and add a sitemap URL. Do not add per-bot disallow groups for GPTBot or ClaudeBot unless you mean to opt out of training. Check the WAF on your host separately. Next.js 16.x (npm latest 16.4.5, read 2026-11-04); docs at nextjs.org/docs/app. [nextjs.org/docs/app/api-reference/file-conventions/metadata/robots](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots) #### [WordPress, Elementor](https://agentreadychecklist.com/platforms/wordpress-elementor/) recipe Leave Settings > Reading > Search engine visibility unchecked. Keep the default `User-agent: *` group. Audit security and caching plugins and the host WAF for rules that return 403 or a challenge to ClaudeBot, GPTBot, OAI-SearchBot or Claude-User; fetch a page with `curl -A 'Claude-User/1.0'` and expect 200. WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. [developer.wordpress.org/reference/functions/do\_robots/](https://developer.wordpress.org/reference/functions/do_robots/) Across all 19 platforms, 19 have a direct recipe, 0 a workaround and 0 say it cannot be done on the platform. The row page lists every one: [AR-FIND-01](https://agentreadychecklist.com/checklist/ar-find-01/). ### Check it ```bash curl -fsSL https://agentreadychecklist.com/kit/verify.sh | bash -s -- https://example.com --only=reach-as-agent,robots-verdicts ``` It cannot prove this: robots.txt does not stop a browsing agent that runs in a user session; user-initiated fetchers may not consult robots.txt at all. ## Sources - [tranco-list.eu](https://tranco-list.eu/)read 2026-11-04 - [github.com/agentready-org/standard, spec/spec.md](https://github.com/agentready-org/standard/blob/main/spec/spec.md)read 2026-11-04 - [ora.ai/api/checks, contract 1.38.0](https://ora.ai/api/checks?utm_source=agentreadychecklist.com&utm_medium=referral&utm_campaign=data)read 2026-11-05 - [docs.netlify.com/manage/routing/headers/](https://docs.netlify.com/manage/routing/headers/)read 2026-11-04 - [nextjs.org/docs/app/api-reference/file-conventions/route](https://nextjs.org/docs/app/api-reference/file-conventions/route)read 2026-11-04 - [yoast.com/features/llms-txt/](https://yoast.com/features/llms-txt/)read 2026-11-04 - [docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages](https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages)read 2026-11-04 - [nextjs.org/docs/app/guides/json-ld](https://nextjs.org/docs/app/guides/json-ld)read 2026-11-04 - [developer.wordpress.org/reference/hooks/wp\_head/](https://developer.wordpress.org/reference/hooks/wp_head/)read 2026-11-04 - [developers.cloudflare.com/pages/configuration/serving-pages/](https://developers.cloudflare.com/pages/configuration/serving-pages/)read 2026-11-04 - [nextjs.org/docs/app/getting-started/server-and-client-components](https://nextjs.org/docs/app/getting-started/server-and-client-components)read 2026-11-04 - [developer.wordpress.org/themes/classic-themes/basics/template-hierarchy/](https://developer.wordpress.org/themes/classic-themes/basics/template-hierarchy/)read 2026-11-04 - [nextjs.org/docs/app/api-reference/functions/not-found](https://nextjs.org/docs/app/api-reference/functions/not-found)read 2026-11-04 - [developer.wordpress.org/reference/functions/status\_header/](https://developer.wordpress.org/reference/functions/status_header/)read 2026-11-04 - [nextjs.org/docs/app/api-reference/file-conventions/metadata/robots](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots)read 2026-11-04 - [developer.wordpress.org/reference/functions/do\_robots/](https://developer.wordpress.org/reference/functions/do_robots/)read 2026-11-04 Edition Data 2026-11-04 Reviewed 2026-11-05 Spec AgentReady v2.0, 273d2d7 Crawl 13,071 rows, 2026-11-04 Ora 132 checks, contract 1.38.0, 2026-11-05 Verify with Ora The scanner this site recommends for the whole checklist. 132 checks, a score out of 100, no account. `npx ax@0.10 audit --min-score ` [Why Ora, and its limits](https://agentreadychecklist.com/verify-with-ora/) Method - [How rows, commands and rates are made](https://agentreadychecklist.com/method/) - [How an Ora check is matched to a row](https://agentreadychecklist.com/checklist/ora-checks/) - [Pass rates by rank band](https://agentreadychecklist.com/data/by-rank/) Sources and read dates sit at the foot of the page. Last reviewed 2026-11-05. AgentReady v2.0, read 2026-11-04.