anthropic-ai
Anthropic · training. Anthropic's earlier training user-agent, widely blocked in AI robots.txt files. Anthropic's current, documented training crawler is ClaudeBot — prefer targeting ClaudeBot in new rules.
- name
- anthropic-ai
- operator
- Anthropic
- purpose
- training
- ua_substring
- anthropic-ai
- robots_token
- anthropic-ai
- respects_robots
- yes
- verify
- Legacy token; Anthropic's current crawler is ClaudeBot. Anthropic honors robots.txt and documents its crawlers (see documentation_url).
- notes
- Anthropic's earlier training user-agent, widely blocked in AI robots.txt files. Anthropic's current, documented training crawler is ClaudeBot — prefer targeting ClaudeBot in new rules.
- canonical_name
- anthropic-ai
- user_agent_token
- anthropic-ai
- ua_full
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- bot_type
- training
- bot_type_extension
- —
- opt_out_mechanism
- robots.txt disallow (User-agent: anthropic-ai); also block ClaudeBot for current Anthropic training
- published_ip_range_url
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- asn
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- reverse_dns_suffix
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- supports_web_bot_auth
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- signature_agent_domain
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- jwks_url
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- verification_methods
user-agent-string- crawl_traffic_share
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots
- targeted_content_type
- HTML, text
- documentation_url
- https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- first_seen_date
- — verify-against-primary-at-build ↗ https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- last_verified_date
- 2026-07-06
- block_vs_allow_recommendation
- conditional — legacy training token; block via robots.txt. Also target ClaudeBot, the current Anthropic crawler.
- citation_referral_value
- low (training)
- cloudflare_verified_category
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots
- status
- deprecated
- triples
["anthropic-ai","operated_by","Anthropic"]["anthropic-ai","has_bot_type","training"]["anthropic-ai","verified_via","user-agent-string"]- attribute_sources
{"claims":["user_agent_token","robots_token"],"source":"https://github.com/ai-robots-txt/ai.robots.txt","last_verified":"2026-07-06"}{"claims":["operator","purpose","documentation_url","respects_robots"],"source":"https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler","last_verified":"2026-07-06"}
last verified · by Özden Erdinc