# Should You Block AI Crawlers? A Decision Guide

> Block or welcome AI is one trade-off: blocking buys control but forfeits citations. Decide per-crawler: block training, price retrieval, welcome citation.

## Blocking trades citations for control

Whether to block or welcome AI comes down to a single trade-off — blocking gives you control over training and refuses no-referral crawlers, but it forfeits the citations and agent referrals that search and retrieval crawlers send back, so the right answer depends on which crawler and which content. This page steelmans both sides before recommending; deep per-bot data lives in the registry.

## The case for blocking AI is real

Training crawlers can ingest content with no referral and no payment, so for some publishers refusing them is rational: blocking protects proprietary or paywalled content, refuses uncompensated training, and is the one lever a publisher fully controls. For a premium archive or licensable dataset, *block-until-paid* is a coherent commercial position — and pay-per-crawl and RSL exist to turn that stance into revenue rather than a flat refusal.

## The case for welcoming AI is the zero-click reality

Search and retrieval crawlers cite and refer — blocking them removes a site from AI answers entirely, and in a **zero-click** world the AI answer may be the only surface a user ever sees. Welcoming and structuring content captures that citation surface. An agent that can transact is a buyer, not a cost. This is the site's stance — *welcome* — offered as a position, not the only rational choice.

## A neutral decision: block training, price retrieval, welcome citation

The defensible synthesis is per-crawler and per-content: **block uncompensated training, price high-value retrieval with pay-per-crawl or RSL, and welcome the citation crawlers that put your content in front of users.**

- **Training crawler, no referral, no pay** → block, or price it via licensing.
- **High-value retrieval crawler** → price it (pay-per-crawl / RSL) rather than block outright.
- **Citation / search crawler that refers users** → welcome it.

Related: [crawler registry](/crawlers) · [opt-out tokens](/access-economics/opt-out-tokens) · [pay-per-crawl](/access-economics/pay-per-crawl) · [RSL content licensing](/access-economics/content-licensing-rsl) · [adoption data](/state-of-the-agentic-web) · [AI access economics](/access-economics) · [audit your site](/services)