Google-Extended

Google-Extended is a robots.txt user-agent token introduced by Google in September 2023 that allows website owners to opt out of having their content used to train Google’s AI models, including Gemini, while still permitting Googlebot to crawl and index their pages for traditional search. This was a response to publisher concerns about their content being used for AI training without compensation or consent. Adding User-agent: Google-Extended followed by Disallow: / to a site’s robots.txt file signals to Google that the site’s content should be excluded from AI training datasets but should continue to appear in Search and Discover. Google-Extended is separate from the opt-outs for Googlebot (which controls search indexing) and for Google’s AI Overviews feature (which is governed by different mechanisms). The introduction of Google-Extended was broadly welcomed by media and publishing industries as a step toward clearer AI data-use controls.