AI crawlers
GPTBot
GPTBot is OpenAI’s web crawler that collects content which may be used to train OpenAI’s generative AI foundation models.
In plain words
OpenAI runs several bots, each with its own job. GPTBot is the one for training.
If you disallow GPTBot in robots.txt, OpenAI says that tells it your content should not be used for training.
Example
A publisher wants to appear in ChatGPT search, but doesn’t want its articles used for training. It allows OAI-SearchBot and disallows GPTBot. OpenAI says these settings are independent of each other.
User-agent: GPTBot
Disallow: /Why it matters for AI search
Blocking GPTBot does not remove you from ChatGPT search. That is controlled by OAI-SearchBot.
If you allow both bots, OpenAI says it may use the results of one crawl for both purposes, to avoid crawling twice.
What you can do
- Decide separately about training (GPTBot) and search (OAI-SearchBot).
- Add a GPTBot group to your robots.txt, or build one with our free robots.txt generator.
- Check that visits claiming to be GPTBot come from OpenAI’s published IP ranges.
Try it: Free robots.txt generator
Often confused with
- OAI-SearchBot
- OAI-SearchBot finds pages for ChatGPT’s search answers. GPTBot collects training data.
- ChatGPT-User
- ChatGPT-User visits a page when a person in ChatGPT asks for it. It doesn’t roam the web on its own.
Quick facts
- Run by
- OpenAI
- robots.txt name
- GPTBot
- Official IP list
- gptbot.json
Sources
Facts and sources last checked on 19 October 2026.
Related terms
How does your page do in AI search?
GEO-Score checks one page on 22 signals, from AI crawler access to structured data, and shows what to fix first.
Analyze a page