Google-Extended
A robots.txt user-agent token that lets a site opt out of having its content used to train and ground Google's Gemini models, while leaving Google Search indexing unaffected.
It separates 'be findable in search' from 'be used for AI training' — a core opt-out lever in the access-economics debate.
- term
- Google-Extended
- category
- identity
- short_def
- A robots.txt user-agent token that lets a site opt out of having its content used to train and ground Google's Gemini models, while leaving Google Search indexing unaffected.
- long_def
- Google-Extended is a control token, not a crawler. Adding 'User-agent: Google-Extended' with 'Disallow: /' to robots.txt tells Google not to use the site's content for training or grounding Gemini and Vertex AI generative models; normal Googlebot search crawling continues. Introduced by Google in September 2023.
- see_also
- —
- etymology_origin
- Introduced by Google in September 2023 as an opt-out control for generative-AI training/grounding.
- related_to
- —
- contrast_with
- Unlike a crawler token that blocks fetching, Google-Extended only governs AI training/grounding use — Googlebot search access is unaffected.
- example
- A publisher adds Google-Extended to robots.txt to keep its articles out of Gemini training while staying in Google Search.
- source
- https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
- status
- stable
- why_it_matters
- It separates 'be findable in search' from 'be used for AI training' — a core opt-out lever in the access-economics debate.
- sameAs
- —
- bridge_entity
- /access-economics
- last_verified
- 2026-06-15
- md_twin
- /glossary/google-extended.md
last verified · by Özden Erdinc