Applebot-Extended

Applebot-Extended doesn’t crawl pages itself. Apple only uses it to decide how data crawled by Applebot may be used.

Apple says pages that disallow Applebot-Extended can still be included in search results, in Spotlight, Siri and Safari.

Apple may use crawled data to answer broad world knowledge questions in Siri and Search, with links to sources. To keep specific content out of those answers, Apple points to the nosnippet meta tag.

User-agent: Applebot-Extended
Disallow: /

At Apple, training and AI answers are two separate controls: Applebot-Extended for training, nosnippet for the answers.

Disallowing Applebot-Extended does not stop Applebot from crawling your pages.

• Add an Applebot-Extended group if you don’t want your content used for Apple’s model training.
• Use nosnippet on content you want kept out of Apple’s world knowledge answers.
• For paywalled pages, Apple supports isAccessibleForFree: false in your structured data; it won’t use that content as extra context for its AI output.

Try it: Free robots.txt generator

Applebot

Applebot is Apple’s crawler for Spotlight, Siri and Safari. Applebot-Extended is only a setting for training use.

Google-Extended

Both are opt-out tokens without a crawler of their own, for two different companies.