Cloudflare AI Crawl Control reviewed: see, allow, block and charge AI crawlers
Cloudflare AI Crawl Control (formerly AI Audit) lets a site owner see which AI services fetch their content, allow or block individual crawlers with granular policies, and optionally charge for access via pay-per-crawl — all from the CDN edge in front of the site.
What Cloudflare AI Crawl Control does
Cloudflare AI Crawl Control is an edge control panel for AI bot traffic. Per Cloudflare's documentation it lets a site owner (1) see which AI services access their content, (2) control access with granular allow/block policies per crawler, and (3) explore monetization through pay-per-crawl pricing for content access. Because it runs at Cloudflare's edge, enforcement happens before a request reaches the origin.
Where it fits in the agentic web
This is an access-control and access-economics tool. It operationalizes two standards this Almanac documents: pay-per-crawl, which prices AI access at the edge, and identity checks that lean on Web Bot Auth to verify which crawler is real before a policy is applied. The crawlers it lists are the same entities catalogued in the AI Crawler & Agent Registry, and Cloudflare Radar's network-wide crawl figures feed the adoption data compiled in the State of the Agentic Web.
Our take
The default place to enforce an AI-access decision if your site already sits behind Cloudflare — and its Radar data is a canonical, citable adoption source. It tells you who is crawling and lets you gate them; it does not tell you whether your own site is agent-ready. Pair the enforcement with a readiness check, then decide allow / block / charge per crawler using the access-economics guide. Plans and exact features change — confirm current capabilities on the vendor page.