NodpickWhat is GPTBot?
GPTBot is OpenAI's web crawler. It collects publicly available pages that may be used to train OpenAI's models. Website owners can allow or block it with a GPTBot rule in robots.txt. It is separate from OAI-SearchBot, which supports ChatGPT search, and from ChatGPT-User.
What blocking does
A Disallow rule for GPTBot tells OpenAI not to crawl those paths for training. It does not on its own remove you from ChatGPT search, which uses a separate crawler, OAI-SearchBot.
Check your setup
A blanket Disallow for all user agents also covers GPTBot and the other AI crawlers. Use the free checker to see which AI crawlers your robots.txt allows.
Check which AI crawlers your site allowsFree AI crawler checker: GPTBot, ClaudeBot, PerplexityBot and more.Related terms
Be first when Nodpick opens
Nodpick opens soon. Join the waitlist for early access and the launch price.