AI crawlers
Claude-SearchBot
Claude-SearchBot is Anthropic’s crawler that navigates the web to improve the quality of the search results Claude gives its users.
In plain words
Anthropic says it analyzes online content to make search responses more relevant and accurate.
It is separate from ClaudeBot, which is about training.
Example
A site wants Claude to find its pages for users’ questions, but doesn’t want its content used for training. It allows Claude-SearchBot and disallows ClaudeBot, as in the snippet below.
User-agent: Claude-SearchBot
Allow: /
User-agent: ClaudeBot
Disallow: /Why it matters for AI search
Anthropic says disabling Claude-SearchBot stops it from indexing your content for search, which may reduce your site’s visibility and accuracy in user search results.
That makes it the Anthropic bot to keep open if you want to be found through Claude.
What you can do
- Check that your robots.txt doesn’t block Claude-SearchBot through a broad rule for all bots.
- Make the training decision separately, with a ClaudeBot group.
- Check visits against Anthropic’s published IP list.
Try it: Free robots.txt generator
Often confused with
- ClaudeBot
- ClaudeBot collects content that could be used for training. Claude-SearchBot is for search.
- Claude-User
- Claude-User fetches a page when a user asks. Claude-SearchBot indexes content for search.
Quick facts
- Run by
- Anthropic
- robots.txt name
- Claude-SearchBot
- Official IP list
- crawling/bots.json
Sources
Facts and sources last checked on 19 October 2026.
Related terms
How does your page do in AI search?
GEO-Score checks one page on 22 signals, from AI crawler access to structured data, and shows what to fix first.
Analyze a page