Google-Extended
A robots.txt user-agent token that lets a site opt out of having its content used to train and ground Google's Gemini models, while leaving Google Search indexing unaffected.
- term
- Google-Extended
- category
- identity
- short_def
- A robots.txt user-agent token that lets a site opt out of having its content used to train and ground Google's Gemini models, while leaving Google Search indexing unaffected.
- long_def
- Google-Extended is a control token, not a crawler. Adding 'User-agent: Google-Extended' with 'Disallow: /' to robots.txt tells Google not to use the site's content for training or grounding Gemini and Vertex AI generative models; normal Googlebot search crawling continues. Introduced by Google in September 2023.
- see_also
- —
- etymology_origin
- Introduced by Google in September 2023 as an opt-out control for generative-AI training/grounding.
- related_to
- —
- contrast_with
- Unlike a crawler token that blocks fetching, Google-Extended only governs AI training/grounding use — Googlebot search access is unaffected.
- example
- A publisher adds Google-Extended to robots.txt to keep its articles out of Gemini training while staying in Google Search.
- source
- https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
- status
- stable
- why_it_matters
- It separates 'be findable in search' from 'be used for AI training' — a core opt-out lever in the access-economics debate.
- sameAs
- —
- bridge_entity
- /access-economics
- last_verified
- 2026-06-15
- md_twin
- /glossary/google-extended.md