# Meta-WebIndexer

> Meta · search

_The AI Crawler Registry · /crawlers/meta-webindexer · [JSON](/api/crawlers/meta-webindexer) · [all The AI Crawler Registry](/crawlers)_

- **name:** Meta-WebIndexer
- **operator:** Meta
- **purpose:** search
- **ua_substring:** Meta-WebIndexer
- **robots_token:** Meta-WebIndexer
- **respects_robots:** no
- **verify:** Meta publishes crawler IP ranges; confirm against those. Meta documents that allowing Meta-WebIndexer in robots.txt lets Meta AI cite and link your content.
- **notes:** Per Meta's documentation, the Meta-WebIndexer crawler navigates the web to improve Meta AI search result quality; allowing it in robots.txt helps Meta AI cite and link your content in its responses. Token and operator-doc reference recorded in the ai.robots.txt machine-readable registry.
- **canonical_name:** Meta-WebIndexer
- **user_agent_token:** Meta-WebIndexer
- **ua_full:** — (verify-against-primary-at-build)
- **bot_type:** search-index
- **bot_type_extension:** —
- **opt_out_mechanism:** robots.txt disallow (User-agent: Meta-WebIndexer)
- **published_ip_range_url:** — (verify-against-primary-at-build)
- **asn:** — (verify-against-primary-at-build)
- **reverse_dns_suffix:** — (verify-against-primary-at-build)
- **supports_web_bot_auth:** — (verify-against-primary-at-build)
- **signature_agent_domain:** — (verify-against-primary-at-build)
- **jwks_url:** — (verify-against-primary-at-build)
- **verification_methods:** published-IP-range
- **crawl_traffic_share:** — (verify-against-primary-at-build)
- **targeted_content_type:** HTML, text
- **documentation_url:** https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/
- **first_seen_date:** — (verify-against-primary-at-build)
- **last_verified_date:** 2026-06-15
- **block_vs_allow_recommendation:** allow — Meta documents that allowing Meta-WebIndexer lets Meta AI cite and link your content in its answers; blocking forfeits that citation/referral.
- **citation_referral_value:** high (powers Meta AI search answers; Meta documents that it cites and links allowed content)
- **cloudflare_verified_category:** — (verify-against-primary-at-build)
- **status:** active
- **triples:** ["Meta-WebIndexer","operated_by","Meta"], ["Meta-WebIndexer","has_bot_type","search-index"], ["Meta-WebIndexer","verified_via","published-IP-range"]
- **attribute_sources:** {"claims":["user_agent_token","robots_token","operator","documentation_url","block_vs_allow_recommendation"],"source":"https://github.com/ai-robots-txt/ai.robots.txt/blob/main/robots.json","last_verified":"2026-06-15"}