NodpickFree AI crawler checker
Enter your domain to see which of 11 AI crawlers your robots.txt allows or blocks, including GPTBot, ClaudeBot, PerplexityBot and Google-Extended, and what each one is used for.
Frequently asked questions
Does blocking GPTBot remove me from ChatGPT?
Not on its own. GPTBot is used for training. ChatGPT search uses OAI-SearchBot, and ChatGPT-User fetches pages when a user asks. Each has its own robots.txt rule.
Does blocking Google-Extended affect Google Search?
No. Google-Extended is a control token for Gemini use of your content. Search crawling by Googlebot is separate and unaffected.
Should I allow AI crawlers?
It depends. Allowing search crawlers lets assistants find and cite your pages. Allowing training crawlers lets your content be used for model training, which some publishers prefer to prevent. Decide per crawler.
Does robots.txt guarantee a bot stays away?
No. It is a voluntary convention that reputable crawlers follow. Firewalls and bot management are separate controls, and this tool does not check them.
Is the check stored?
We store nothing about your domain or the result. We fetch your public robots.txt once to evaluate it. To limit abuse we keep a salted hash of your IP address, with a timestamp, for rate limiting.
Learn more
Next step: join the waitlist for the AI visibility check.