# Pay-Per-Crawl: Pricing AI Access at the Edge

> Pay-per-crawl prices each AI request at the edge: Cloudflare returns HTTP 402 to unpaid crawlers and settles the fee as merchant of record.

## Pay-per-crawl charges AI crawlers per request

Pay-per-crawl prices AI access at the network edge: a publisher sets a fee per request, and the CDN returns **HTTP 402 (Payment Required)** to AI crawlers that have not paid — so crawling becomes a metered transaction rather than a free take. Cloudflare's pay-per-crawl is the reference implementation: it sits in front of the origin, identifies known AI crawlers, and gates them on payment instead of an all-or-nothing block. The publisher keeps three outcomes per crawler — *allow free*, *charge*, or *block*.

The control point matters: pay-per-crawl is **CDN-controlled and enforced at the edge**, not a voluntary signal honored at the operator's discretion. Cloudflare acts as **merchant of record**, handling billing and settlement between crawler and publisher. The exact pricing model, recognised-crawler list, and GA/beta status must be verified against Cloudflare's primary pay-per-crawl docs at build — no transaction or pricing figure is asserted here.

## The HTTP 402 flow gates an unpaid crawl

An AI crawler requests a URL; the edge checks for a payment arrangement; if none, it answers **402 Payment Required** instead of 200 with the content. A paying crawler is charged the per-request fee and served normally. A 402 is a standard, parseable refusal a well-built agent can react to, unlike a silent block.

```
GET /article HTTP/1.1
User-Agent: GPTBot

HTTP/1.1 402 Payment Required
(crawler has no payment arrangement — fetch is gated until settled)
```

## Content-Signal declares how content may be used after access

Pay-per-crawl decides *whether* a crawler may fetch; the **Content-Signal** directive declares *how* the fetched content may be used afterward — search indexing vs. model training vs. AI input. Content-Signal is a Layer-6 licensing directive (robots.txt or HTTP header); its exact syntax should be verified against primary docs at build.

Related: [pay-per-crawl Layer-6 spec](/protocols/licensing/pay-per-crawl) · [Content-Signal spec](/protocols/licensing/content-signal) · [pay-per-crawl in the Lexicon](/glossary/pay-per-crawl) · [crawler registry](/crawlers) · [AI access economics](/access-economics) · [audit your site](/services)