# Pay-Per-Crawl: Pricing AI Access at the Edge

> Pay-per-crawl prices each AI request at the edge: Cloudflare returns HTTP 402 to unpaid crawlers and settles the fee as merchant of record.

## Pay-per-crawl charges AI crawlers per request

Pay-per-crawl prices AI access at the network edge: a publisher sets a fee per request, and the CDN returns **HTTP 402 (Payment Required)** to AI crawlers that have not paid — so crawling becomes a metered transaction rather than a free take. Cloudflare's pay-per-crawl is the reference implementation: it sits in front of the origin, identifies known AI crawlers, and gates them on payment instead of an all-or-nothing block. The publisher keeps three outcomes per crawler — *allow free*, *charge*, or *block*.

The control point matters: pay-per-crawl is **CDN-controlled and enforced at the edge**, not a voluntary signal honored at the operator's discretion. Cloudflare acts as **merchant of record**, handling billing and settlement between crawler and publisher. The exact pricing model, recognised-crawler list, and GA/beta status must be verified against Cloudflare's primary pay-per-crawl docs at build — no transaction or pricing figure is asserted here.

## The HTTP 402 flow gates an unpaid crawl

An AI crawler requests a URL; the edge checks for a payment arrangement; if none, it answers **402 Payment Required** instead of 200 with the content. A paying crawler is charged the per-request fee and served normally. A 402 is a standard, parseable refusal a well-built agent can react to, unlike a silent block.

```
GET /article HTTP/1.1
User-Agent: GPTBot

HTTP/1.1 402 Payment Required
(crawler has no payment arrangement — fetch is gated until settled)
```

## Content-Signal declares how content may be used after access

Pay-per-crawl decides *whether* a crawler may fetch; the **Content-Signal** directive declares *how* the fetched content may be used afterward — search indexing vs. model training vs. AI input. Content-Signal is a Layer-6 licensing directive (robots.txt or HTTP header); its exact syntax should be verified against primary docs at build.

## Pay-per-crawl — frequently asked questions

### How does Cloudflare pay-per-crawl charge AI crawlers?

It prices access at the network edge: the publisher sets a per-request fee and the CDN returns HTTP 402 (Payment Required) to AI crawlers that have not paid, charging paying crawlers and serving them normally. Cloudflare acts as merchant of record for billing and settlement. Exact pricing and status should be verified against Cloudflare's primary docs.

Related: [pay-per-crawl Layer-6 spec](/protocols/pay-per-crawl) · [how x402, AP2, ACP and UCP settle and authorize agent payments](/commerce/x402-vs-ap2-vs-acp-vs-ucp) · [Content-Signal spec](/protocols/content-signal) · [pay-per-crawl in the Lexicon](/glossary/pay-per-crawl) · [crawler registry](/crawlers) · [AI access economics](/access-economics) · [audit your site](/services)