---
title: "CF-PAY-PER-CRAWL: Pay per crawl, HTTP 402 | Agent Ready Checklist"
description: "CF-PAY-PER-CRAWL (CLOUDFLARE, Find layer): Pay per crawl, HTTP 402. What it asks, the evidence, the command that verifies it and the fix on 19 platforms."
canonical: https://agentreadychecklist.com/checklist/cf-pay-per-crawl
site: "Agent Ready Checklist"
last_updated: 2026-11-05
section: "Checklist"
---

# CF-PAY-PER-CRAWL: Pay per crawl, HTTP 402 | Agent Ready Checklist

> CF-PAY-PER-CRAWL (CLOUDFLARE, Find layer): Pay per crawl, HTTP 402. What it asks, the evidence, the command that verifies it and the fix on 19 platforms.

Source: https://agentreadychecklist.com/checklist/cf-pay-per-crawl
Site: Agent Ready Checklist
Last reviewed: 2026-11-05
Agent endpoints: https://agentreadychecklist.com/llms.txt (index), https://agentreadychecklist.com/openapi.json (API), https://agentreadychecklist.com/mcp (MCP server), https://agentreadychecklist.com/ask (search)
Access: https://agentreadychecklist.com/auth.md (no key). Pricing: https://agentreadychecklist.com/pricing.md (free).

# CF-PAY-PER-CRAWL: Pay per crawl, HTTP 402

CF-PAY-PER-CRAWL is a Cloudflare scanner check on the find layer: If you charge crawlers, answer with HTTP 402 Payment Required and pricing headers, or 200 when the crawler presents payment intent. It applies to publishers on Cloudflare who sell crawl access. 19 of 19 platforms have a recipe below, 18 of them workarounds. Ora has no check that matches it.

## The requirement

Identifier

CF-PAY-PER-CRAWL

Level

CLOUDFLARE

Layer

Find

Applies to

publishers on Cloudflare who sell crawl access

Checked by

none listed by isitagentready on 2026-11-04

Ora check

none matched, see below

The statement

If you charge crawlers, answer with HTTP 402 Payment Required and pricing headers, or 200 when the crawler presents payment intent.

## Why

-   Pay per crawl is a feature of AI Crawl Control in closed beta; Cloudflare acts as Merchant of Record.primary source, 2026-07-28: [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/), read 2026-11-04
-   A crawler either presents payment intent in request headers for a 200 or receives 402 Payment Required with pricing.primary source, 2026-07-28: [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/), read 2026-11-04
-   WAF or Bot Management blocks override the pay per crawl charge rule.primary source, 2026-07-28: [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/), read 2026-11-04
-   The isitagentready check list read on 2026-11-04 shows Commerce checks for x402, MPP, UCP and ACP and no pay per crawl check.primary source, 2026-11-04: [isitagentready.com](https://isitagentready.com/), read 2026-11-04

## How to verify

No Verify Kit check covers this row, so the test is manual.

### The one-line version

```bash
curl -s -o /dev/null -w "%{http_code}" -A "GPTBot/1.0" <url>
```

curl -s -o /dev/null -w "%{http\_code}" -A "GPTBot/1.0" <url> returns 402 only where you meant to charge.

## Verify with Ora

Ora's catalogue of 132 checks, read 2026-11-05, has no check that this row's name, cited standard or published description matches, so this page names no Ora id. The audit below still scores the rest of the site. [The crosswalk](https://agentreadychecklist.com/checklist/ora-checks/) lists every row and its match.

### Or audit the whole site

```bash
npx ax@0.10 audit https://example.com --min-score 70
```

Exit code 1 means the score is below the number you passed, so the same line gates a deploy: [the CI recipe](https://agentreadychecklist.com/guides/gate-deploys-on-agent-readiness-in-ci/). [Why this site recommends Ora, and where it falls short](https://agentreadychecklist.com/verify-with-ora/).

## Fix it on your platform

### Next.js (App Router)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Next.js does not set it, and a crawler is answered 402 only on a zone that has it. The documentation read lists no built-in feature for this row. Workaround: put the domain on a Cloudflare zone, where Next.js allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Next.js 16.x (npm latest 16.4.5, read 2026-11-04); docs at nextjs.org/docs/app. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Next.js (App Router) recipes](https://agentreadychecklist.com/platforms/nextjs-app-router/#cf-pay-per-crawl)

### Astroworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Astro does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Astro allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Astro 7.x (npm latest astro 7.4.5, @astrojs/sitemap 3.7.4, read 2026-11-04). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Astro recipes](https://agentreadychecklist.com/platforms/astro/#cf-pay-per-crawl)

### Nuxtworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Nuxt does not set it, and a crawler is answered 402 only on a zone that has it. The documentation read lists no built-in feature for this row. Workaround: put the domain on a Cloudflare zone, where Nuxt allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Nuxt 4.x (npm latest nuxt 4.6.4, @nuxtjs/robots 7.0.2, @nuxtjs/sitemap 9.1.0, read 2026-11-04). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Nuxt recipes](https://agentreadychecklist.com/platforms/nuxt/#cf-pay-per-crawl)

### SvelteKitworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so SvelteKit does not set it, and a crawler is answered 402 only on a zone that has it. The documentation read lists no built-in feature for this row. Workaround: put the domain on a Cloudflare zone, where SvelteKit allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

SvelteKit (npm latest @sveltejs/kit 3.1.1, read 2026-11-04); docs at svelte.dev/docs/kit. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All SvelteKit recipes](https://agentreadychecklist.com/platforms/sveltekit/#cf-pay-per-crawl)

### Remix and React Router (framework mode)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Remix and React Router does not set it, and a crawler is answered 402 only on a zone that has it. The documentation read lists no built-in feature for this row. Workaround: put the domain on a Cloudflare zone, where Remix and React Router allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

React Router framework mode (npm latest react-router 8.5.1; Remix v2 @remix-run/node 2.17.5, read 2026-11-04); docs at reactrouter.com. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Remix and React Router (framework mode) recipes](https://agentreadychecklist.com/platforms/remix-react-router/#cf-pay-per-crawl)

### Plain static hosting (Netlify, Vercel static, Cloudflare Pages, GitHub Pages, S3 and CloudFront)recipe

On Cloudflare Pages the site already sits on a Cloudflare zone, so pay per crawl is set in Cloudflare's own controls (AI Crawl Control) with a price per zone, and Cloudflare answers HTTP 402 with the price to a crawler that presents no payment intent. The feature was in closed beta on 2026-11-04. On another static host, put the domain on a Cloudflare zone first. Test with curl -A "GPTBot/1.0".

Host documentation as published on 2026-11-04 (no version numbers; config files \_headers, \_redirects, vercel.json, 404.html). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Plain static hosting (Netlify, Vercel static, Cloudflare Pages, GitHub Pages, S3 and CloudFront) recipes](https://agentreadychecklist.com/platforms/static-hosting/#cf-pay-per-crawl)

### WordPress and Elementorworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so WordPress and Elementor does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where WordPress and Elementor allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

WordPress 7.1.4 (core release from the version-check API), Elementor 4.3.6 (plugin directory, updated 2026-11-02), Yoast SEO 28.8 noted for llms.txt; docs read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All WordPress and Elementor recipes](https://agentreadychecklist.com/platforms/wordpress-elementor/#cf-pay-per-crawl)

### Shopify (Online Store 2.0 themes)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Shopify does not set it, and a crawler is answered 402 only on a zone that has it. The documentation read lists no built-in feature for this row. Workaround: put the domain on a Cloudflare zone, where Shopify allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, Online Store 2.0 Liquid themes, docs read 2026-11-04; storefront behaviour probed live on one Shopify store the same day. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Shopify (Online Store 2.0 themes) recipes](https://agentreadychecklist.com/platforms/shopify/#cf-pay-per-crawl)

### Webflowworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Webflow does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Webflow allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, help center articles read 2026-11-04 (updated dates 2024-10-14 to 2026-10-29). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Webflow recipes](https://agentreadychecklist.com/platforms/webflow/#cf-pay-per-crawl)

### Framerworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Framer does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Framer allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, help articles read 2026-11-04 (marked updated 2026-09-15). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Framer recipes](https://agentreadychecklist.com/platforms/framer/#cf-pay-per-crawl)

### Wix (Wix Editor and Wix Studio)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Wix does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Wix allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, help center read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Wix (Wix Editor and Wix Studio) recipes](https://agentreadychecklist.com/platforms/wix/#cf-pay-per-crawl)

### Squarespace (version 7.1)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Squarespace does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Squarespace allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, version 7.1 sites, help center read 2026-11-04 (articles updated 2026-07-17 to 2026-11-03). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Squarespace (version 7.1) recipes](https://agentreadychecklist.com/platforms/squarespace/#cf-pay-per-crawl)

### HubSpot CMS (Content Hub)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so HubSpot CMS does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where HubSpot CMS allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, Content Hub, knowledge base and developer docs read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All HubSpot CMS (Content Hub) recipes](https://agentreadychecklist.com/platforms/hubspot-cms/#cf-pay-per-crawl)

### Docusaurusworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Docusaurus does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Docusaurus allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

3.10.2 (npm @docusaurus/core latest, read 2026-11-04). Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Docusaurus recipes](https://agentreadychecklist.com/platforms/docusaurus/#cf-pay-per-crawl)

### Mintlifyworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Mintlify does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Mintlify allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, docs read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Mintlify recipes](https://agentreadychecklist.com/platforms/mintlify/#cf-pay-per-crawl)

### GitBookworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so GitBook does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where GitBook allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, docs read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All GitBook recipes](https://agentreadychecklist.com/platforms/gitbook/#cf-pay-per-crawl)

### ReadMeworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so ReadMe does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where ReadMe allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS, docs read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All ReadMe recipes](https://agentreadychecklist.com/platforms/readme/#cf-pay-per-crawl)

### Salesforce B2C Commerce (SFRA and PWA Kit / Composable Storefront)workaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Salesforce B2C Commerce does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Salesforce B2C Commerce allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

Hosted SaaS (B2C Commerce help release 264.0.0); PWA Kit retail-react-app 10.3.0 and pwa-kit-runtime 3.21.0 on npm, read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Salesforce B2C Commerce (SFRA and PWA Kit / Composable Storefront) recipes](https://agentreadychecklist.com/platforms/salesforce-commerce-cloud/#cf-pay-per-crawl)

### Adobe Commerce and Magento Open Sourceworkaround

Pay per crawl is a feature of Cloudflare AI Crawl Control, set per Cloudflare zone and in closed beta on 2026-11-04, so Adobe Commerce and Magento Open Source does not set it, and a crawler is answered 402 only on a zone that has it. Not possible on this platform without outside help. Workaround: put the domain on a Cloudflare zone, where Adobe Commerce and Magento Open Source allows a proxy or custom DNS in front, and set the price there. Test with curl -A "GPTBot/1.0".

2.4.9 (GitHub tag published 2026-05-12; 2.4.10-alpha1 tagged 2026-10-13); release notes read 2026-11-04. Derived from Cloudflare's pay per crawl documentation read on 2026-11-04. [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/) and [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/). [All Adobe Commerce and Magento Open Source recipes](https://agentreadychecklist.com/platforms/adobe-commerce/#cf-pay-per-crawl)

## What it does not do

-   Charging is the opposite of AR-FIND-01 for crawlers that do not pay; decide per crawler.
-   Not available outside Cloudflare.

## Common mistakes

-   Charging user-initiated fetchers that act for a reader, not a training crawler.

## Sources

-   [developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/](https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/)read 2026-11-04
-   [isitagentready.com](https://isitagentready.com/)read 2026-11-04
-   [ora.ai/api/checks, contract 1.38.0](https://ora.ai/api/checks?utm_source=agentreadychecklist.com&utm_medium=referral&utm_campaign=requirement)read 2026-11-05
-   [registry.npmjs.org/ax, the ax README](https://registry.npmjs.org/ax)read 2026-11-05

Verify with Ora

The scanner this site recommends for the whole checklist. 132 checks, a score out of 100, no account.

`npx ax@0.10 audit <url> --min-score <n>`

[Why Ora, and its limits](https://agentreadychecklist.com/verify-with-ora/)

Fix it on

-   [Next.js (App Router)workaround](#p-nextjs-app-router)
-   [Astroworkaround](#p-astro)
-   [Nuxtworkaround](#p-nuxt)
-   [SvelteKitworkaround](#p-sveltekit)
-   [Remix and React Router (framework mode)workaround](#p-remix-react-router)
-   [Plain static hosting (Netlify, Vercel static, Cloudflare Pages, GitHub Pages, S3 and CloudFront)recipe](#p-static-hosting)
-   [WordPress and Elementorworkaround](#p-wordpress-elementor)
-   [Shopify (Online Store 2.0 themes)workaround](#p-shopify)
-   [Webflowworkaround](#p-webflow)
-   [Framerworkaround](#p-framer)
-   [Wix (Wix Editor and Wix Studio)workaround](#p-wix)
-   [Squarespace (version 7.1)workaround](#p-squarespace)
-   [HubSpot CMS (Content Hub)workaround](#p-hubspot-cms)
-   [Docusaurusworkaround](#p-docusaurus)
-   [Mintlifyworkaround](#p-mintlify)
-   [GitBookworkaround](#p-gitbook)
-   [ReadMeworkaround](#p-readme)
-   [Salesforce B2C Commerce (SFRA and PWA Kit / Composable Storefront)workaround](#p-salesforce-commerce-cloud)
-   [Adobe Commerce and Magento Open Sourceworkaround](#p-adobe-commerce)

Last reviewed 2026-11-05. AgentReady v2.0, read 2026-11-04.